Heretic is a tool that removes safety alignment restrictions from transformer-based language models without requiring expensive retraining.
The tool addresses the problem of censorship in language models by applying directional ablation, an automated technique that identifies and modifies specific model parameters responsible for refusal behavior. Rather than requiring manual expertise in transformer internals, Heretic uses a parameter optimizer based on tree-structured Parzen estimation to automatically find abliteration settings that minimize refusals while preserving the model's original capabilities, measured by KL divergence on harmless prompts. The process runs completely unsupervised with default configuration, making it accessible to users without deep technical knowledge of how language models work internally.
Heretic suits developers and researchers who want to remove safety restrictions from language models and have access to sufficient computational resources. The tool supports most dense transformer architectures including multimodal models, multiple mixture-of-experts designs, and hybrid architectures, though pure state-space models and certain research architectures are not yet supported. According to the README, Heretic's automatically generated decensored models achieve comparable refusal suppression to manually created abliterations while introducing less degradation to the original model's performance.
The project maintains a substantial user base, with nearly all open issues coming from external adopters rather than the core team. Maintainers typically respond to new issues and pull requests within a day. Work in the issue tracker centers on bug reports, enhancement requests, and user questions.
