Foundry Local is Microsoft's end-to-end local AI solution designed to enable applications to run AI models entirely on users' devices without requiring cloud connectivity, API keys, or backend infrastructure. Written primarily in C++, the project provides native SDKs for C#, JavaScript, Python, and Rust, allowing developers to integrate on-device inference across multiple programming languages. The runtime package is exceptionally lightweight at approximately 20 MB, making it practical to bundle with applications and distribute to end users.
The core runtime handles model acquisition, hardware acceleration via ONNX Runtime, model management, and inference execution. Foundry Local automatically detects available hardware on the user's device and selects the optimal execution provider among NPU, GPU, or CPU options, ensuring models run efficiently regardless of the target hardware. The platform includes a curated catalog of high-quality models optimized specifically for on-device use, covering chat completions with models like GPT OSS, Qwen, DeepSeek, Mistral, and Phi, as well as audio transcription through Whisper models. Every model in the catalog undergoes extensive quantization and compression to balance quality and performance, and models are versioned to allow applications to pin to specific versions or receive automatic updates.
The project maintains an OpenAI-compatible API that supports both request and response formats from OpenAI, including the OpenAI Responses API format. This compatibility allows applications already using the OpenAI SDK to integrate Foundry Local with minimal code changes. An optional local web server provides an OpenAI-compatible endpoint for serving models to multiple processes, integrating with tools like LangChain, or enabling REST-based experimentation, though the primary design targets in-process SDK usage for embedded scenarios.
GitGenius activity data reveals active development and maintenance, with 323 tracked issues and pull requests showing a median response latency of 15.1 hours and mean latency of 192.0 hours. The most active labels tracked are ADO with 37 items, followed by papercut and bug categories each with 33 items. Primary contributors include samuel100 with 552 recorded events, natke with 270 events, and bmehta001 with 92 events. The repository shares overlapping contributors with microsoft/vscode, microsoft/typescript, and rust-lang/rust, indicating integration with Microsoft's broader development ecosystem.
The project provides comprehensive samples across all supported languages, with C# offering 12 samples, JavaScript 12 samples, Python 9 samples, and Rust 8 samples. These samples demonstrate native chat functionality, audio transcription, tool calling, model management, web server integration, and LangChain compatibility. A command-line interface in public preview allows developers to explore available models and experiment interactively, with installation available through GitHub releases and full documentation provided on Microsoft Learn.
Foundry Local explicitly positions itself as a client-side inference solution rather than a server framework, optimized for single-user scenarios on hardware-constrained devices where user data remains local, responses start immediately with zero network latency, and applications function entirely offline. The platform supports Windows, macOS with Apple silicon, and Linux, with the SDK licensed under MIT and the CLI under Microsoft Software License Terms.
