pdf-inspector: Classify the PDF Before Paying for OCR
Most document pipelines rely on a convenient assumption: every PDF goes to OCR. It’s the safe default — you don’t know what’s inside the file, so you send it to a service that can read anything, wait between two and ten seconds, and pay per page.
That assumption is false for most documents you actually process. Reports, invoices, papers, contracts, bank statements: they were born digital. The text is already inside the file, sitting in the content stream as plain operators. Running them through OCR is paying a vision model to read a photo of something you could have read directly.
pdf-inspector is Firecrawl’s answer to that: a Rust library that opens the PDF, decides in milliseconds whether it needs OCR, and — if it doesn’t — extracts the text and delivers clean Markdown to you locally. No model, no GPU, no API key. It’s been on npm since April and on crates.io since June, this week it crossed 8,100 stars on GitHub, and it’s the classification layer that runs underneath Firecrawl’s own commercial parsing endpoint.
How It Decides, Without Rendering Anything
This is the part worth understanding before installing it, because the mechanism explains both the speed and the limits.
pdf-inspector doesn’t rasterize pages or run a layout model. It parses the xref table and page tree without loading the entire document, traverses the content streams, and looks for two things: text operators (Tj, TJ) and image operators (Do). Pages with text operators have extractable text. Pages with only images don’t. That’s the entire classification, and it’s why a 300-page document resolves in milliseconds instead of seconds.
What it returns isn’t binary but four-way: TextBased, Scanned, ImageBased, or Mixed — plus a confidence score between 0 and 1, and a list called pagesNeedingOcr.
That last field is what changes your architecture. Classifying at the document level is all-or-nothing: one scanned appendix and the entire 150-page report goes to OCR. Routing at the page level means the 148 native pages are extracted locally in milliseconds and only the two scanned ones cost you anything.
Installation and First Run
The Node package comes with prebuilt binaries for Linux x64 and ARM64 (glibc and musl, so Alpine works), macOS ARM64, and Windows x64. npm downloads only the one that matches your platform — about 5–6 MB, with TypeScript definitions included and no Rust toolchain anywhere in the process.
npm install @firecrawl/pdf-inspector
# or
bun add @firecrawl/pdf-inspector
Python and Rust get the same core:
pip install pdf-inspector
cargo add pdf-inspector
The project’s documentation site also documents a direct invocation via npx to test it on a file without installing anything, if you want the quickest possible look before committing.
A minor gotcha: version numbers aren’t aligned across registries. npm is at 1.12.0, PyPI at 0.2.6, and crates.io at 0.1.7 — same project, three independent versioning streams. Pin by registry and don’t try to reason across them.
The Routing Branch
This is the form that matters in a real pipeline:
import { readFileSync } from 'fs';
import { classifyPdf, processPdf } from '@firecrawl/pdf-inspector';
const pdf = readFileSync('document.pdf');
const meta = classifyPdf(pdf);
console.log(meta.pdfType); // "TextBased" | "Scanned" | "Mixed" | "ImageBased"
console.log(meta.pageCount); // 42
console.log(meta.pagesNeedingOcr); // [5, 12, 15] (0-indexed)
console.log(meta.confidence); // 0.875
if (meta.pdfType === 'TextBased' && meta.confidence > 0.8) {
const { markdown } = processPdf(pdf); // local, no network
// done
} else {
// send to your OCR service only meta.pagesNeedingOcr
}
Two calls, and the expensive path goes from being the default to being conditional.
If you’re building a hybrid pipeline where a layout model detects regions on rendered pages, there’s extractTextInRegions(buffer, pageRegions): you pass bounding boxes in PDF points and get the text back by region, each with a needsOcr flag that fires on empty text, fonts with GID encoding, or garbage output. When it specifically suspects a corrupted text layer, ocrReason comes back as "suspected_garbled_text". That flag is the honest part of the design: the library tells you when it doesn’t trust its own extraction, instead of handing you garbage with confidence.
Tuning How Hard It Looks
The classification accepts a ScanStrategy, and choosing the right one is a real decision:
| Strategy | Behavior | When to use it |
|---|---|---|
EarlyExit (default) |
Stops at the first page without text | You’re routing TextBased documents to a fast path |
Full |
Scans all pages, no early exit | You need to distinguish Mixed from Scanned well |
Sample(n) |
Samples n pages — first, last, middle | Very large PDFs where speed matters more than precision |
Pages(vec) |
Only the pages you specify | You already know where to look |
The default is optimized for the routing case, not for classifying with precision. If you care about distinguishing Mixed from Scanned — and you should, if you’re routing at page level — use Full.
The Markdown Side
Classification is half the tool; the other half is the converter, and it’s more complete than the “no ML models” description suggests. It infers headings H1–H4 from tiers of font size relative to body text, detects bold and italic by font name, recognizes bulleted, numbered, and lettered lists, identifies code blocks by monospaced fonts (Courier, Consolas, Menlo, Fira Code, JetBrains Mono), reconstructs reading order in multi-column layouts, handles CID/Type0 fonts via ToUnicode CMaps, supports RTL, rejoins words split by hyphens across lines, merges drop caps, filters page numbers, and collapses ellipses in tables of contents.
Table detection runs two ways at once: rectangle-based, reading the actual drawing operations in the PDF, plus a heuristic pass on text alignment for tables drawn without lines. It handles tables that span pages and separates consolidated numerical values in financial tables.
From the Rust crate you also get two CLIs:
cargo install pdf-inspector
pdf2md document.pdf --compact # token-efficient output
pdf2md document.pdf --select-pages 1,3,5-10
pdf2md document.pdf --pages # inserts <!-- Page N --> markers
detect-pdf document.pdf --analyze --json
--compact is worth knowing about if you’re going to feed this to an LLM: it collapses dot leaders and similar padding from the source document, which is pure token waste downstream.
There’s also a WASM build for browsers (@firecrawl/pdf-inspector-wasm) that runs the same Rust parser inside a Web Worker, so a file a user drops on your frontend never leaves their machine.
Reading the Benchmark Honestly
Firecrawl published numbers against the opendataloader-bench corpus (200 PDFs), updated July 31st on an Apple M4 Pro, with OCR disabled, comparing only local engines without ML:
| Engine | Overall | Reading order | Tables (TEDS) | Headings | Speed (200 docs) |
|---|---|---|---|---|---|
| pdf-inspector | 0.875 | 0.915 | 0.814 | 0.788 | 0.470s |
| liteparse | 0.873 | 0.913 | 0.693 | 0.811 | 0.750s |
| opendataloader | 0.831 | 0.902 | 0.489 | 0.739 | 2.569s |
| pymupdf4llm | 0.735 | 0.886 | 0.401 | 0.424 | 17.117s |
| markitdown | 0.589 | 0.844 | 0.273 | 0.000 | 16.165s |
It’s a vendor benchmark — Firecrawl ran it, on their own fork of the harness — so read it as a claim, not a verdict. And read it carefully, because the headline doesn’t survive close reading.
That 0.875 versus liteparse’s 0.873 is a tie, not a win. Same with reading order. Where pdf-inspector really separates is in tables — 0.814 versus 0.693 is a real difference, and it’s consistent with the dual detection strategy — and in speed, where 0.470s versus 17.117s isn’t a difference but a different category of tool. On headings it loses to liteparse, something the current README doesn’t say out loud, though an earlier version did.
One number to completely ignore: Firecrawl’s launch post claims “0.002s per page”. Their own benchmark shows 0.470 seconds for 200 documents, or about 2 ms per document. Per page the number is different. Use the table, not the tweet.
And a note in fairness about the last row: MarkItDown, which we covered here in April, is a generalist that converts Word, PowerPoint, Excel, HTML, and audio in addition to PDFs. Measuring it on a corpus exclusively of PDFs against a dedicated parser isn’t a fair fight, and that 0.000 on headings says more about the test than the tool. Different jobs.
Where It Doesn’t Fit
It doesn’t do OCR. That’s not a limitation, it’s the premise: pdf-inspector tells you when you need OCR and gets out of the way. If your corpus is mostly scanned, this saves you a classification step and nothing else.
Headings are its weakest axis, and the reason is structural: it infers hierarchy from font size, so documents that mark titles with bold at the same size as body text will flatten out. If heading structure is what you use for chunking in RAG, test that specifically before committing.
And that “~54% of PDFs don’t need OCR” is a figure from Firecrawl, presented without a source. It’s credible in direction, but measure it against your own corpus instead of budgeting with it — a two-call script over a hundred of your real documents gives you your actual number in an afternoon.
Worth an Afternoon
The barrier here is low and the result is concrete: you install a package, run classifyPdf on a sample of your production documents, and discover what fraction of your OCR spend is buying text you already had. If the answer approaches half, the routing branch pays for itself the week you deploy it. And unlike almost everything that promises savings, this is MIT, runs local, and verifies with two function calls.
And you? What percentage of the PDFs you process end up in OCR unnecessarily — have you measured it yet, or are you still sending everything down the same path?