pdf-inspector is a fast Rust library developed by Firecrawl for classifying PDFs and extracting text without requiring OCR processing. The library intelligently detects whether a PDF is text-based, scanned, image-based, or mixed, enabling smart routing decisions in document processing pipelines. This classification happens rapidly, typically within 10-50 milliseconds by sampling content streams and returning confidence scores along with per-page OCR routing recommendations. The core motivation behind pdf-inspector is practical: approximately 54% of PDFs are already text-based and do not require expensive OCR services, so the library allows organizations to skip OCR for these documents and process them locally in under 200 milliseconds.
The library provides position-aware text extraction that captures font information and X/Y coordinates while automatically determining multi-column reading order. It converts extracted text to clean Markdown with support for headings (H1-H4 detected via font size ratios), bullet and numbered lists, code blocks identified through monospace font detection, tables detected through both rectangle-based methods from PDF drawing operations and heuristic text alignment analysis, and formatting like bold and italic text. The Markdown conversion also handles URLs, page breaks, and advanced features like hyphenation repair, drop caps, and dot leaders for table of contents styling.
pdf-inspector includes specialized support for complex PDF features such as CID fonts with ToUnicode CMap decoding for Type0 and Identity-H fonts across UTF-16BE, UTF-8, and Latin-1 encodings. It automatically detects encoding issues and flags broken font encodings so callers can fall back to OCR when necessary. The library implements multi-column layout detection for newspaper-style columns with sequential reading order and right-to-left text support. Table detection operates in dual mode, combining rectangle-based detection from PDF drawing operations with heuristic detection from text alignment, handling financial tables, footnotes, and continuation tables across pages.
The library is available across multiple platforms through language bindings for Python, Node.js, and browser WebAssembly, allowing the same Rust parser to run locally in browsers and Web Workers with embedded CMaps and no server round trip. The core implementation is lightweight, written in pure Rust with a single dependency on the lopdf library for PDF parsing, and contains no machine learning models or external service dependencies.
Benchmark results from July 31, 2026, on the opendataloader-bench corpus of 200 PDFs show pdf-inspector achieving an overall score of 0.875, with 0.915 on reading order, 0.814 on tables, and 0.788 on headings detection. It processed the complete corpus in 0.470 seconds, significantly faster than competing engines like liteparse (0.750s), opendataloader (2.569s), pymupdf4llm (17.117s), and markitdown (16.165s). The library is positioned as a strong local default for native-text PDFs including reports, research papers, financial documents, invoices, and legal PDFs that require clean structured Markdown without OCR latency or infrastructure overhead.
The architecture loads a PDF document once and shares it between detection and extraction stages, eliminating redundant parsing. Classification works by parsing the xref table and page tree without full object loading, then sampling pages according to configurable scan strategies (early exit, full scan, sampling, or specific pages) to detect text operators and image operators in content streams. This approach enables detection of 300+ page PDFs in milliseconds with per-page OCR routing capabilities rather than all-or-nothing classification.
