Repository Issue Activity (beta)

#413 qint4 (TinyGemmWeightQBitsTensor) is much slower than qint2 for bf16 CUDA training workloads

#327 issues with non-contiguous Tensor

#411 GPU Memory Leak During Layer-by-Layer Inference in QuantizedModelForCausalLM (or quantize & freeze) (Layers Not Released After Deletion)

#404 'TinyGemmWeightQBitsTensor' object has no attribute '_scale'

#405 is this project still maintained?

#406 wrapped tensor does not expose data pointer

#400 Is model parallelism supported?

#397 module incompatible

#398 Error loading quantized qwen2.5 vl model

#396 The quantized model does not support FSDP.

#393 Support Multi-GPU calibration

#391 The `bench/generation/metrics/perplexity.py` will produce a wrong result when n_ctx > n_batch

#339 Error when quantizing flux (already fixed in master)

#388 Some tensors share memory, this will lead to duplicate memory on disk and potential differences when loading them agai

#386 RuntimeError: Error building extension 'quanto_cuda':

#381 can "tests" be used instead of "test" for test folder?

#385 param.dtype of the model is torch.float32 after quantization

#379 bench/kernels/benchmark.py fails due to usage of pruned API disable_extensions

#378 When using `Calibration`, only the first batch returns `QTensor`

#361 ValueError: invalid literal for int() with base 10: '90a'

#311 quantize(model, weights=qint4, activations=qint8) produce weights with dtype = torch.uint8

#375 When will we be getting full Quanto support for WAN 2.1?

#376 Cannot use pipe.enable_model_cpu_offload() to quantized diffusers transformers

#374 Can run an int8 quantized model on CUDA?

#254 Packages created on the CI are missing cpp and cuda extension files