The AMD Strix Halo Toolboxes repository provides pre-built container environments for running large language models on AMD Ryzen AI Max "Strix Halo" integrated GPUs. Written in Python, the project packages llama.cpp with various backend configurations into standardized toolbox containers compatible with Fedora, Ubuntu, openSUSE, and Arch Linux. The repository is maintained by kyuz0 as a hobby project and is part of a broader Strix Halo AI Toolboxes initiative documented on an accompanying website.
The project offers multiple stable toolbox configurations optimized for different use cases. The vulkan-radv container provides the most stable and compatible option recommended for general users and all model types. The vulkan-amdvlk variant uses AMD's open-source driver for faster performance but has a 2 GiB single buffer allocation limit that prevents loading some larger models. For ROCm users, the repository maintains stable builds including rocm-7.2.4 and rocm-6.4.4, both patched for kernel 6.18.4 and later support. Experimental toolboxes include rocm-7.2.4-rocmfp4 supporting custom tensor types and draft-MTP, and rocm-7.2.4-turboquant for TurboQuant quantization, though these require manual builds rather than automatic updates.
Critical to the project's functionality are specific host configuration requirements. The stable configuration targets Fedora 42 or 43 with Linux kernel 6.18.9 and specific firmware versions, with explicit warnings against using kernels older than 6.18.4 and certain firmware versions that break ROCm support. The repository documents essential kernel parameters including amd_iommu=off, amdgpu.gttsize=126976, and ttm.pages_limit=32505856 to enable unified memory allocation while reserving system resources. Performance benchmarking data is available through an interactive viewer, and the project includes a VRAM estimator tool to help users plan memory requirements for different models accounting for context overhead.
The repository demonstrates active maintenance and community engagement. GitGenius tracking shows kyuz0 as the primary contributor with 151 events, followed by Djip007 with 22 events and ilker-aktuna with 17 events across 70 tracked issues and pull requests. The median response latency for issue and pull request triage is 4.1 hours, indicating responsive project management. The repository shares contributors with major projects including ggml-org/llama.cpp, ollama/ollama, and openhands/openhands, suggesting integration within the broader AI inference ecosystem.
Documentation covers building containers locally for customization, distributed inference across multiple Strix Halo machines using a Python script with SSH coordination, and troubleshooting guides for firmware-related issues. The quick start guide emphasizes critical runtime parameters including flash attention and no-mmap flags to prevent crashes and performance degradation. The project includes deprecation notices for older toolbox variants, such as the MTP-specific containers that have been merged into the main llama.cpp branch, guiding users toward current stable builds. A test configuration is documented using a Framework Desktop with Ryzen AI MAX+ 395, 128 GB system RAM, and 512 MB GPU memory allocation, providing a reference point for users setting up their own systems.
