OpenKB is an open-source command-line system that transforms raw documents into a structured, interlinked wiki-style knowledge base powered by large language models. Rather than relying on traditional retrieval-augmented generation that rediscovers knowledge on every query, OpenKB compiles documents once into a persistent wiki that accumulates and synthesizes knowledge over time. The system is built on PageIndex's vectorless, reasoning-based retrieval engine, enabling accurate handling of long documents without requiring a separate vector database.
The repository supports a broad range of input formats including PDF, Word documents, Markdown, PowerPoint, HTML, Excel, CSV, plain text, and URLs. A key architectural distinction is how OpenKB handles documents of different lengths. Short documents are converted to Markdown and read in full by the LLM, while long PDFs of twenty pages or more are processed through PageIndex into hierarchical tree indexes that the LLM reads instead of full text. This approach enables the system to scale to complex, lengthy documents while maintaining accuracy and context awareness. The system also provides native multi-modality support, retrieving and understanding figures, tables, and images alongside text.
When documents are added to the knowledge base, the LLM generates summary pages, reads existing concept and entity pages, creates or updates concepts with cross-document synthesis, and automatically extracts and maintains entity pages for people, organizations, places, and products. Wiki pages follow the Google Open Knowledge Format specification for knowledge sharing, and the entire wiki is stored as plain Markdown files with cross-links, making it compatible with Obsidian for graph visualization.
OpenKB operates across two functional layers. The wiki foundation layer handles compilation and maintenance through commands like init, add, list, status, watch, and lint. The generators layer transforms the compiled wiki into useful outputs including one-off queries with citations, interactive multi-turn chat sessions, interactive knowledge graphs with multiple visualization modes, redistributable agent skills through the Skill Factory, and HTML slide decks. Query results and chat sessions can be persisted for later resumption.
The system includes a bundled web UI called the Knowledge Workbench served at the root path of the REST API, allowing users to browse the knowledge base, upload and compile documents, and stream queries and chats directly in the browser. The web interface is built with Vite and can be run in development mode or built for production bundling.
Configuration is handled through LiteLLM, which provides support for multiple LLM providers including OpenAI, Claude, Gemini, and others. Users specify their model during initialization or in a configuration file using LiteLLM's provider/model format. The system can be installed from PyPI, GitHub, or from source for development purposes. Authentication for the web API is disabled by default for local-first usage but can be enabled by setting an API token environment variable.
The repository is classified under the domains of agents, AI, knowledge bases, large language models, and retrieval-augmented generation. The project is maintained by VectifyAI and has a homepage at pageindex.ai, with documentation available at docs.pageindex.ai.
