llmware is a unified framework designed for building enterprise retrieval-augmented generation (RAG) pipelines using small, specialized language models. The project emphasizes sustainable, accurate, and cost-effective AI applications that can run locally on devices with minimal compute footprint. Written in Python and supporting versions 3.10 through 3.14, the framework targets deployment across Windows, Mac, and Linux platforms, including edge devices and self-hosted environments.
The core of llmware consists of two main components. The first is a model catalog containing over 300 models in quantized, optimized formats. This includes 50+ llmware-finetuned specialized models from the SLIM, Bling, Dragon, and Industry-Bert families designed for enterprise process automation tasks. The catalog also integrates support for leading cloud models from OpenAI, Anthropic, and Google. The second component is a comprehensive RAG pipeline with integrated tools for the complete lifecycle of connecting knowledge sources to generative AI models, featuring extensive document parsing and ingestion capabilities alongside scalable knowledge base creation.
The framework provides unified access to multiple inference technologies optimized for different platforms. It supports GGUF, OpenVINO, ONNXRuntime, ONNXRuntime-QNN for Qualcomm processors, WindowsLocalFoundry, and PyTorch. The Model Depot on Hugging Face contains 95+ OpenVINO-based LLMs ready for deployment. For specialized tasks, llmware offers 20 OpenVINO-optimized encoding models through the OVEmbeddingModel class, supporting embeddings, rerankers, and classifiers. The framework includes 7 NPU-optimized models for running on Snapdragon processors in Windows Arm64 environments.
Key features include a unified model catalog for consistent access regardless of underlying implementation, a Library component for ingesting and indexing knowledge at scale with parsing and embedding capabilities, Query functionality supporting text, semantic, hybrid, metadata, and custom filters, and a Prompt with Sources feature combining knowledge retrieval with LLM inference. The framework supports multi-model agents using SLIM models on CPU for complex workflows and includes advanced document processing with OCR capabilities for embedded images, enhanced PDF, Word, PowerPoint, and Excel parsing with improved text-chunking controls and table extraction.
The repository demonstrates active development with 79 tracked issues and pull requests showing a median response latency of 52.6 hours. The most active contributor, doberst, has logged 90 events, with RS-labhub and NYDocutest also contributing significantly. The project maintains strong community engagement through Discord with live member counts displayed and YouTube tutorials. Common issue labels include good first issue (8 occurrences), enhancement (6), and help wanted (1), indicating ongoing development priorities.
The framework includes extensive example solutions covering OpenVINO encoders and rerankers, WindowsLocalFoundry integration, image generation with multimedia bots, audio processing with WhisperCPP for voice transcription, and natural language to SQL conversion with Slim-SQL. The project provides over 100 cut-and-paste recipes and a fast-start tutorial series for new users. Installation is straightforward via pip with options for core or full installations, the latter adding a wider set of related Python libraries. The repository connects to related projects including PyTorch, SurrealDB, and CrewAI through overlapping contributors, indicating integration within a broader ecosystem of AI development tools.
