Repository Issue Activity (beta)

#571 Curious about Exllama+TP

#817 Multiple out-of-bounds accesses in QMatrix quant-metadata handling via a crafted GPTQ/EXL2 model

#814 E8 lattice VQ for KV cache: 2-3 bit with asymmetric K/V — cross-model results

#813 [BUG] System freeze during inference with RTX 5060 Ti via Thunderbolt 5 eGPU on Linux (kernel freeze, SSH drop, GPU fan full speed)

#811 [REQUEST] Please add support for Qwen3VL

#808 [BUG] 2080ti outputs gibberish

#804 [REQUEST]Please support Torch 2.8

#809 [QUESTION] Inquiry about DynamicGenerator: EOS not returned until max_new_tokens reached, despite stop_conditions

#810 [REQUEST] Make a version that is on Cuda V13

#806 [QUESTION] I cannot achieve token/s speed as in your tests even with better GPU

#803 [BUG] [Cause Identified & Workarounds Proposed] Infinite Generation for Llama3 & Other Models with Multiple EOS IDs

#800 [BUG] Dry sampling over long contexts

#802 [BUG] Unable to run examples/chat.py

#795 [BUG] ExllamaV2 version >0.2.8 broken for mistral 7b(v0.2) models on Nvidia 2060

#784 [BUG] Runtime error when trying to load Qwen3 32B

#798 [REQUEST] Support for Hunyuan-A13B-Instruct

#797 [BUG] ExLlamaV2Generator import broken across WHL & source repo — Windows 11 + RTX 5090 + CUDA 12.8 build inconsistencies

#777 [BUG]gemma 3 27b exl2 loops nonsense afterwards 2-3 correct paragraphs

#749 [REQUEST] Please add support for Gemma3.

#793 [BUG] Exllamav2 quickly devolves into endless repetition in versions newer than 2.8.0

#789 [BUG] Remove Sentencepiece

#724 [REQUEST] Support new SOTA vision model: Qwen 2.5 VL (3B, 7B, 72B)

#792 [QUESTION] Estimate measurements and quantization peak VRAM use in advance

#790 [BUG] Silent crash in safetensors call when compiling shards

#780 [BUG] Blue Screen MEMORY_MANAGEMENT Error when trying to quantize Gemma3.