A document intelligence app turns the unstructured chaos of PDFs, scans, and images into clean structured data and answerable questions. It is one of the most commercially valuable AI applications — invoices, contracts, forms, and reports all need extracting — and one of the most deceptively hard, because the AI is the easy part and the messy real-world documents are not. Here is the pipeline.
Stage 1 — Ingest and OCR
Documents arrive as a mess: native PDFs, scanned images, phone photos, faxes. Your first job is to get clean text and layout out of them. For native digital PDFs, extract the text layer directly. For anything scanned or photographed, run OCR — open-source Tesseract for low cost and control, or managed services like AWS Textract and Google Document AI for higher accuracy and built-in table and form detection. This stage is where most real-world failures originate, so handle rotation, multi-column layouts, and tables deliberately rather than hoping the model copes downstream.
Stage 2 — Classify and extract with an LLM
With text in hand, the LLM does two jobs. Classification decides what kind of document this is — invoice, contract, receipt, ID — which determines what to extract. Extraction then pulls the fields you care about. The critical discipline here is structured output: define a JSON schema, instruct the model to return only valid JSON matching it (using function calling or a structured-output mode), and validate the result in code. Never scrape values out of free-form prose; reject or retry anything that does not parse. A typical prompt supplies the document text, the target schema, and the rule “return null for fields you cannot find rather than guessing.”
Stage 3 — Query with retrieval
Beyond fixed-field extraction, users often want to ask questions of a document — “what is the termination clause?” The right pattern is retrieval-augmented generation (RAG): split the document into chunks, embed each into a vector, and store the vectors. At query time, embed the question, retrieve the most relevant chunks, and feed them to the model alongside the question. This grounds the answer in the actual text and lets you cite the source passage, which sharply reduces hallucination compared with asking the model from memory.
Stage 4 — The review interface
No production document system runs fully unattended, because accuracy degrades on the long tail of bad scans and odd layouts. The fix is a confidence and review loop: have the pipeline flag low-confidence extractions and route them to a simple human review UI showing the document beside the extracted fields, so a person corrects and approves in seconds. These corrections become training data to improve the system over time. Designing for the human-in-the-loop from day one is what makes the app trustworthy enough to use on real invoices and contracts.
Putting it together
A robust document intelligence app is mostly plumbing — ingestion, validation, and review — wrapped around a thin layer of LLM calls. Spend your engineering effort there, keep every extraction structured and validated, and ground every answer in retrieved text. To understand why the model invents data when its input is poor, read how LLMs work, and for the summarisation-pipeline patterns that overlap heavily with this one, see how to build a personalised AI newsletter.