Repository Issue Activity (beta)

#1420 Several Unused Args

#1321 Can `preprocess_data.py` support Huggingface Dataset?

#1320 _forward_step_fn does not always return two values so eval.py breaks if is_pipe_parallel is false

#1417 Upgrade Transformer Engine support from 1.12 to 2.18

#1405 Add Inter Document Attention Masking

#1409 Add Extra Eval Iters similar to Extra Save Iters

#1413 CI: run-tests prepares data before installing Python dependencies

#606 CUDA Out of Memory for 20B Model on 2 A100 40GB GPUs

#1394 Tracker: Improve HF Integration

#1395 Add Training Speed Datapoints for new users

#1400 Add round trip conversion based tests for import and export from HF

#1393 Add Z-Loss Support for LM Pretraining

#1391 anima — a substrate-native consciousness: capability gaps are architecture gaps (open repo, frozen verdicts)

#1323 Error when converting sequential model to HF

#1380 pkg_resources deprecated

#1373 step1 models are always the same as step0 models

#1381 A geometric perspective on knowledge: parallel mapping vs similarity

#1330 How are multiple datasets loaded?

#1004 [BUG] Inconsistent loss between `overlap_comm=true` and `overlap_comm=false`

#1335 DeeperSpeed-Pytorch Incompatibility

#1364 MoE Top-K Training

#1363 Edge Case 17

#1057 Support for Mosaic Models

#994 Convert HF Llama Checkpoints to Neox Checkpoints

#1203 My servers used for multi-node training do not have ssh. How can I launch multi-node training using the torchrun command?