Repository Issue Activity (beta)

#49 Missing pyproject.toml

#48 用这个量化的时候报了这个错，求解决

#47 [Question] why fp8 kv output is dequantized before attention?

#46 [Qusetion] How can i see the net structure after i use autofp8

#44 How to Quantize KV Cache for the deepseek-v2 Model

#43 Floating point exception (core dumped)

#39 Qwen2-72B-Instruct-FP8 generate bad output using cutlass but be fine with torch._scaled_mm

#42 Quantize DeepSeek-Coder-V2-Instruct to W8A8(INT8)?

#40 Differences in Dynamic Quantization Speedup for Varying SFT Tasks on Qwen2-72b-Instruct Models

#37 LLaMA3 report

#41 User tokens masking?

#38 fp8 vs bf16 performance problem

#36 CUDA out of memory when quantizing llama3.1-405b on 80GiBx8 H100 instance

#35 [Qusetion] Calibration datasets

#34 Support for Vision models

#32 [Feature] ADD Support for DeepSeek-V2-Chat

#31 Runtime Error:The weights trying to be saved contained shared tensors.

#29 DeepSeek-Coder-V2-Lite-Instruct not working when quantized to FP8 using AutoFP8

#23 Can AutoFP8 quantized MOE model inferenced with vlllm?（kv_cache fp8 or kv_cache+weights fp8）

#30 Integration with Hugging Face transformers library

#27 I get empty response after Quantizing LLama2 70B using AutoFP8 with calibration

#28 error: RuntimeError: The weights trying to be saved contained shared tensors

#26 E5M2 or mix format trial

#24 When I use autofp8 to quantize the qwen32b model and test it, the accuracy drops significantly.

#10 FP8 KV cache support