HAMi is a Kubernetes device plugin and scheduler extender that enables GPU virtualization and heterogeneous accelerator sharing on Kubernetes clusters.
AI infrastructure teams typically allocate entire GPUs to small workloads, leading to poor utilization and resource contention. HAMi solves this by inserting a virtualization layer between Kubernetes and accelerator hardware. It uses a mutating webhook to intercept pod specifications, a scheduler extender to place workloads with device-aware policies, and device plugins paired with in-container virtualization components to enforce memory and compute isolation. Workloads request fractional accelerator resources through standard Kubernetes resource requests, and the tool coordinates scheduling, allocation, and per-workload limits without requiring application code changes.
HAMi suits shared AI clusters running multi-tenant notebook, training, and inference workloads where GPU utilization is a concern. It is particularly valuable for teams operating heterogeneous accelerator environments spanning NVIDIA, Ascend, Cambricon, Hygon, Iluvatar, Kunlunxin, MetaX, Moore Threads, and other vendors, since it provides a unified scheduling and allocation workflow across different device types and operational models. The tool integrates with standard Kubernetes schedulers like kube-scheduler and Volcano, and includes production-ready features such as metrics, dashboards, a web UI, and Helm-based installation.
The project maintains active development across multiple areas. Device support is regularly expanded to cover new accelerator vendors and models. The codebase receives ongoing refinement in core scheduling logic and isolation mechanisms. Documentation is kept current across multiple languages. Community engagement appears consistent through issue triage and feature requests. The project operates as a CNCF Incubating initiative with established governance and landscape recognition.
