Transformer Debugger is a tool for investigating and understanding the internal behavior of small language models through interactive exploration and automated interpretability techniques.
The tool addresses the challenge of understanding why language models make specific predictions or attend to particular tokens. It combines automated interpretability methods with sparse autoencoders to identify which model components—neurons, attention heads, and autoencoder latents—drive particular behaviors. Users can intervene in the forward pass to observe how changes affect model outputs, and the tool automatically generates explanations of what causes identified components to activate. By tracing connections between components, it helps discover the circuits underlying model decisions.
The tool suits researchers and practitioners investigating model behavior in small language models, particularly those studying mechanistic interpretability. It is designed for rapid exploration without requiring custom code, making it accessible to those new to interpretability work. The release includes a React-based neuron viewer for interactive exploration, a backend activation server that performs inference and serves precomputed data, a simple inference library for GPT-2 models and their autoencoders, and collated activation datasets showing top-activating examples for model components.
Development activity shows consistent maintenance with validation processes in place for changes, including test execution, type checking, and functional verification of core features. The project maintains organized documentation covering terminology and setup procedures, with clear separation between frontend and backend components that allows independent development and deployment.
