Repository Issue Activity (beta)

#5 Train an Encoder on the BigCode Dataset

#88 Saved model checkpoint in different precision ?

#87 IndexError: too many indices for tensor of dimension 2

#86 Without "<suffix>" token

#80 Want explanation of the MQA related code

#78 LM Head FLOPs

#77 How to use my dataset to finetune

#8 Train Python model with FIM

#75 How to generate several result with num_beams

#6 List of things to be (potentially) ported from Megatron-DeepSpeed

#63 how to convert huggingface?

#69 OOM while merging starcoder model (after sft) from TP=4,PP=4 to TP=8,PP=1

#60 how to prepare the training data to train starcoder?

#59 ValueError: Invalid attention arguments: AttnType.self_attn, None

#61 RuntimeError: Error building extension 'scaled_upper_triang_masked_softmax_cuda'

#57 Incomplete humaneval evaluation code

#56 Incomplete humaneval evaluation code

#49 Script to train starcoder

#2 TF-Model Architecture

#3 TF-Tokenization

#4 TF-Multi Node Training Layout

#16 Experiment plan

#19 Support MQA in tools.checkpoint_loader/saver_megatron

#45 Support interleaved pipeline schedules in checkpoint merging tools

#31 Log GPU throughput