OpenMythos is an open-source, community-driven theoretical reconstruction of Anthropic's Claude Mythos model architecture, built from first principles using publicly available research literature and speculation. The project is written in Python and distributed via PyPI, with an active community presence on Discord. According to GitGenius tracking data, the repository has shown consistent engagement with a median issue and pull request response latency of 1.3 hours across 23 tracked items, though the mean response time extends to 58 hours, indicating variable response patterns. The primary maintainer kyegomez has driven 18 tracked events, with additional contributions from ghost (14 events) and DeepPytorch (5 events). The repository's contributor network overlaps with major projects including Microsoft's VSCode and TypeScript implementations, as well as the Rust language repository, suggesting cross-pollination with established systems programming and development tool communities.
The core architectural innovation that OpenMythos implements is the Recurrent-Depth Transformer (RDT), also called a Looped Transformer. Rather than stacking hundreds of unique transformer layers, this design recycles a subset of layers and executes them multiple times per forward pass using identical weights. This approach enables deeper reasoning without proportional parameter growth. The architecture divides computation into three functional blocks: a Prelude of standard transformer blocks, a Recurrent Block that loops up to a configurable maximum number of iterations, and a final Coda. The recurrent mechanism injects the original encoded input at every loop step through learned injection parameters, preventing hidden state drift across iterations.
The implementation provides seven pre-configured model variants ranging from 1 billion to 1 trillion parameters, with corresponding adjustments to embedding dimensions, expert counts, loop iterations, and context window sizes. The smallest variant (mythos_1b) uses 2048 dimensions with 64 experts and 16 loop iterations, while the largest (mythos_1t) scales to 16384 dimensions with 512 experts and 64 loop iterations. The attention mechanism is switchable between two implementations: Grouped Query Attention (GQA) with optional Flash Attention 2 support for improved memory efficiency, and Multi-Latent Attention (MLA) inspired by DeepSeek-V2, which caches compressed KV latents rather than full key-value pairs.
Training infrastructure is provided for the 3-billion parameter model on the FineWeb-Edu dataset, with support for both single-GPU and multi-GPU training via PyTorch Distributed Data Parallel. The training pipeline uses the OpenAI GPT-OSS-20B tokenizer, AdamW optimization, bfloat16 precision on modern hardware, and a learning schedule with 2000-step linear warmup followed by cosine decay, targeting approximately 30 billion tokens of training data.
The repository's theoretical foundation rests on the hypothesis that Mythos achieves its distinctive capabilities through three mechanisms. First, looped transformers demonstrate systematic generalization—the ability to compose knowledge in novel ways unseen during training—through a three-stage grokking process of memorization, in-distribution generalization, and out-of-distribution compositional reasoning. Second, the architecture enables depth extrapolation, allowing models trained on shallow reasoning chains to handle deeper chains at inference time by increasing loop iterations. Third, each loop iteration functions as an implicit chain-of-thought step operating in continuous latent space rather than discrete tokens, enabling simultaneous exploration of multiple reasoning directions within a single forward pass.
The repository addresses the critical stability challenge in training looped models through the Parcae architecture framework, which constrains injection parameters to guarantee spectral stability by construction. This involves parameterizing injection matrices as continuous negative diagonal matrices and discretizing them to ensure the spectral radius remains below one, preventing residual explosion and training divergence. Complete API documentation is provided in dedicated markdown files covering the OpenMythos class interface, configuration options, and recommended training datasets with token budget guidance per model scale.
