Repository Issue Activity (beta)

#329 Reduce training prompt costs with SuperCompress

#320 why dose my finetuned model repeat the given prompt before generating its response

#322 I have a question regarding the VRAM requirements for fine-tuning a 7B model.

#105 Alpaca problem solving team - QQ chat group

#133 ValueError: Your setup doesn't support bf16/gpu. You need torch>=1.10, using Ampere GPU with cuda>=11.0

#245 ERROR:torch.distributed.elastic.multiprocessing.api:failed (exitcode: -9) local_rank: 0

#162 error of multi-GPU: torch.distributed.elastic.multiprocessing.api:failed (exitcode: -9) local_rank: 0

#252 bf16 & tf32，can they be used together

#315 Tensors of the same index must be on the same device and the same dtype except `step` tensors that can be CPU and float32 notwithstanding

#321 Tensors of the same index must be on the same device and the same dtype except `step` tensors that can be CPU and float32 notwithstanding

#314 train.py fails with TypeError: Object of type Tensor is not JSON serializable

#313 Keyword arguments {'add_special_tokens': False} not recognized.

#319 ValueError: Trying to set a tensor of shape torch.Size([32769536]) in "weight" (which has shape torch.Size([32001, 4096])), this looks incorrect.

#317 SFT Mistral；

#155 [Large Data Training] It can train, but there seems to be a error

#71 Any plans for using GPT-4 for self-instruct? Or using larger llama models?

#296 How to finetune with a own private data and then build chatbot on that?

#175 Why resize embedding?

#304 weight_diff.py state_dict_recovered[key].add_(state_dict_raw[key]) RuntimeError: The size of tensor a (32001) must match the size of tensor b (32000) at non-singleton dimension 0

#159 How to run the fine-tuning using LLaMA?

#256 weight_diff AssertionError: Naive integrity check failed. This could imply that some of the checkpoint files are corrupted.

#170 Training LLaMA 7B took only 40 minutes when using 8 80GB A100 GPUs

#168 Fine-tuning finished in 1.5 hours

#262 Question on License : commercial use permitted, but at the same time, research-purpose only ?

#316 RuntimeError: The size of tensor a (65539072) must match the size of tensor b (262156288) at non-singleton dimension 0