The llm-attacks repository is the official implementation of research on universal and transferable adversarial attacks against aligned language models, published in a paper by Andy Zou, Zifan Wang, Nicholas Carlini, Milad Nasr, J. Zico Kolter, and Matt Fredrikson. The repository focuses on developing and demonstrating attack methods that can compromise the safety alignment of large language models, with particular emphasis on the Greedy Coordinate Gradient (GCG) algorithm for generating adversarial prompts that cause models to produce harmful completions.
The primary purpose of this repository is to provide researchers and practitioners with tools to understand vulnerabilities in aligned language models. The codebase is classified across multiple security and AI safety domains including adversarial attacks, language model security, prompt engineering, model robustness, vulnerability research, jailbreaking, and alignment breaking. The repository includes both minimal implementations for educational purposes and comprehensive experimental frameworks for rigorous evaluation of attack methods.
The repository provides several key components for conducting attacks. A demo notebook called demo.ipynb offers a minimal implementation of the GCG algorithm specifically for jailbreaking LLaMA-2, available both locally and on Google Colab. The experiments folder contains reproducible code for testing GCG attacks on AdvBench, supporting multiple experimental configurations including individual behavior attacks, multiple behavior experiments across single models, and transfer experiments across different models. The codebase is written in Python and uses the ml_collections package for hyperparameter management, allowing researchers to modify experimental settings either directly in configuration files or through launch scripts.
The repository supports attacks on specific model architectures, particularly LLaMA-2-7B-Chat and Vicuna-7B models obtained from HuggingFace. Installation requires FastChat version 0.2.23 and the llm-attacks package itself. The documentation notes that experiments were conducted using NVIDIA A100 GPUs with 80GB memory per chip, and the codebase currently supports only LLaMA and Pythia-based models, with warnings that other model architectures may produce silent errors due to tokenizer differences.
Community engagement with the repository shows moderate response latencies, with a median issue and pull request response time of approximately 1170.7 hours and a mean of 4739.1 hours across 33 tracked items. The most active contributors tracked by GitGenius include XitaoLi with 5 events, Anson-He with 4 events, and Miracle-Master with 4 events. The repository maintains connections with related projects through overlapping contributors, including links to ggml-org/llama.cpp, ikawrakow/ik_llama.cpp, and oobabooga/textgen.
The repository released nanogcg in August 2024, described as a fast and easy-to-use implementation of the GCG algorithm installable via pip. The project is licensed under the MIT license and includes reproducibility guidance, noting specific issues users have encountered when running experiments on different hardware configurations and operating systems, such as prompting LLaMA-2-7B-Chat-GGML and naming issues on Windows systems.
