kyuz0/amd-strix-halo-toolboxes

AMD Strix Halo Llama.cpp Toolboxes is a container system that enables running large language models on AMD Ryzen AI Max "Strix Halo" integrated GPUs.

The project solves the problem of deploying LLMs efficiently on Strix Halo hardware by providing pre-built Toolbx containers with multiple backend options. Toolbx is Fedora's standard developer container system, now compatible with Ubuntu, openSUSE, and Arch. The toolboxes bundle llama.cpp with different GPU backends—Vulkan via Mesa RADV or AMDVLK, and ROCm—allowing users to choose based on stability and performance needs. The project includes host configuration guidance, performance benchmarks, memory planning tools, and distributed inference support.

Developers working with Strix Halo hardware who want to run LLMs should consider this tool if they are using compatible Linux distributions. The Vulkan RADV backend is recommended as the most stable and compatible option for most users and models. The AMDVLK backend offers the fastest performance but has a single buffer allocation limit that prevents loading some larger models. ROCm containers provide access to AMD's supported stack, with versions available for both the latest and previous stable releases. The project includes specific guidance on stable kernel versions and firmware to avoid, indicating that hardware compatibility requires careful configuration.

The project maintains a substantial base of real-world adopters, as evidenced by most open issues being raised by outside users rather than the core team. Maintainers respond to new issues and pull requests within hours, indicating active engagement with the community.