← All news and guides
AI adoption

Document Intelligence Pipelines: Extraction, Validation and Human Review

How to build a document intelligence pipeline: extraction, schema and business-rule checks, confidence routing, a review queue, and an accounts payable example.

Mono Team ·

An empty wooden sorting rack with rows of open pigeonholes in a quiet mailroom, late afternoon light through a tall window, shallow depth of field.

Short answer: a document intelligence pipeline is five steps, and the model is only one of them. Documents come in and are classified. A model extracts fields. Schema and business-rule checks validate them. A confidence score routes each document either straight through or to a person. Everything is logged and measured. Most failures come from the steps around the model: no validation, one threshold for every field, and no reviewer queue. Build those and a mid-range extraction service will do. Skip them and the best model will still post wrong numbers to your books.

This guide is for engineering and operations leads who want to turn invoices, forms, contracts or KYC files into structured data. It covers the pipeline stages, how to choose an extraction method, what to validate, where human review belongs and how to design it. It ends with accounts payable (AP) as a worked example, because AP shows clearly which errors matter and which do not. If you are still deciding whether to build this at all or buy a specialist product, start with our build vs buy framework for internal AI tools. For help with the business side of an AI rollout, see our AI adoption service.

What does a document intelligence pipeline do?

It turns unstructured documents (scans, PDFs, photos, emails) into structured records that another system can trust. The useful way to think about it is as stages, each with one job and one failure mode.

StageJobTypical failureControl
1. IntakeJob: Receive files, check type and size, store the originalTypical failure: Wrong file type, password-protected PDF, duplicate uploadControl: Reject or flag early. Hash the file to spot duplicates
2. ClassificationJob: Decide what the document is (invoice, credit note, statement)Typical failure: A credit note processed as an invoiceControl: Classify first and route by type. Send low-confidence types to review
3. ExtractionJob: Read text, tables and fieldsTypical failure: Misread digits, merged columns, missed linesControl: Return a confidence value per field and keep page coordinates
4. ValidationJob: Check the fields against a schema and business rulesTypical failure: Valid-looking but wrong valuesControl: Rules that cross-check fields and systems of record
5. Routing and reviewJob: Auto-accept, or send to a personTypical failure: Everything goes to review, or nothing doesControl: Thresholds per field, plus random sampling
6. Output and auditJob: Write to the downstream system with a full recordTypical failure: No way to explain a posted value laterControl: Log the source page, model version, rule results and reviewer

Keep the original file and the extracted values side by side, with a link between them. When an auditor or a supplier asks why a number was posted, that link is your answer.

How should you extract the data: OCR, a document service or an LLM?

A tall stack of plain cream paper sheets in a shallow wooden tray on a worn desk, only the edges visible, soft morning light from a side window, the background out of focus.

There are three common routes. Most real pipelines mix them.

ApproachGood forWatch out for
Classic OCR plus templates or regular expressionsGood for: A few fixed layouts from known sendersWatch out for: Breaks when a supplier changes its layout. Every new format needs work
Managed document services (for example AWS Textract, Azure Document Intelligence, Google Document AI)Good for: Forms, tables and standard documents such as invoices, with field-level confidenceWatch out for: Your documents are processed in the provider's cloud. Feature sets and regions change, so check current docs
A general LLM with a schemaGood for: Varied layouts, free text, contracts, messy scans that need judgementWatch out for: Slower and costlier per page. Schema-valid output can still be wrong

A few facts from the providers' own documentation help you plan. AWS Textract's AnalyzeDocument supports tables, forms, queries, signatures and layout feature types, and returns a confidence score on each block. Its synchronous limit is 10 MB and asynchronous PDF jobs go up to 500 MB (AWS docs, checked 20 November 2026). Azure Document Intelligence describes field confidence as an estimated probability between 0 and 1, and says to use it to decide whether to accept a prediction automatically or flag it for human review (Microsoft Learn, updated 19 October 2026).

Structured output features on LLM APIs help too. Claude's structured outputs use constrained decoding to guarantee schema-valid JSON, and OpenAI's Structured Outputs with `strict: true` constrain completed output to your JSON Schema. Both vendors say the limits plainly. Claude's feature does not support numerical or string-length constraints, among others. OpenAI says your app must still handle refusals and incomplete responses and validate business rules that the schema cannot express. In short, schema-valid is not the same as correct. A model can return a well-formed total that is wrong.

A practical pattern is to use a document service or OCR for the first pass, then call an LLM only for documents that fail validation or arrive in a layout you have not seen. That keeps cost and latency down and keeps the cheaper path as the default. Pick based on a labelled sample of your own documents, not on a demo.

What should you validate after extraction?

Validation is where a pipeline earns trust. Use two layers.

  • Schema checks. Use JSON Schema (the current version is 2020-12) to check that required fields exist, types are right, dates parse and enums match. This catches malformed output. It is cheap and should run on every document.
  • Business-rule checks. These test meaning. Do the line items add up to the subtotal? Does the subtotal plus tax equal the total? Is the supplier in your vendor master? Does the tax ID have a valid format? Does the purchase order exist, and is it still open? Is this invoice number already posted for the same supplier?

Write rules as code with clear names and a result per rule, so a reviewer sees "total does not equal sum of lines" and not just "failed". Store each rule result with the record.

Cross-field checks are strong because errors rarely fool all of them. If a model misreads one digit in a line amount, the sum check fails even when every field carries a high confidence score. Add checks against your systems of record too: the vendor master, open purchase orders and goods receipts. For any field that decides money or identity, prefer a check against your own data over trusting the document.

Where does human review belong in an AI workflow?

Two empty office chairs pulled up to a wide desk with two blank monitors switched off, a closed notebook between them, afternoon light across the floor, shallow depth of field.

Put people where an error is costly and a machine cannot check it. Do not put them everywhere. A review step that touches every document removes the benefit of automating. A pipeline with no review step posts errors silently. Use three triggers.

TriggerExampleWhy
Low confidence on a critical fieldExample: Total, tax amount, bank account or PO number below its thresholdWhy: The model is unsure about a field that moves money
A failed business ruleExample: Totals do not add up, duplicate invoice number, unknown supplierWhy: The document and your records disagree
Random sampling of accepted documentsExample: A small, fixed share of auto-accepted documents checked each weekWhy: Confidence scores can be wrong. Sampling finds errors the scores miss

Set thresholds per field, not per document. A supplier name can tolerate more doubt than a bank account number. Azure's guidance for custom models is to aim for an accuracy score of at least 80 per cent, and close to 100 per cent for financial or medical records. Treat that as a reminder that the bar depends on the cost of an error. Tune each threshold on a labelled sample of your own documents and revisit it when volumes or suppliers change. Confidence values from different services are not comparable, so recalibrate if you switch.

How do you design the review queue?

Build the queue as a normal product feature. Do not rely on a vendor's built-in human review. Amazon Augmented AI (A2I), Textract's review feature, is no longer open to new customers (AWS docs, checked 20 November 2026), and Google's Document AI Human-in-the-Loop has been deprecated since 16 January 2024 (provider docs, checked 20 November 2026). Their trigger ideas are still sound: key confidence ranges, missing keys and random sampling. The tooling is yours to own.

A usable review queue has these parts:

  • Source and result side by side. Show the page image with the extracted field highlighted at its coordinates. Reviewers should never hunt for a value.
  • Only the doubtful fields. Show the fields that triggered review at the top, with the reason ("low confidence" or "rule failed: totals").
  • One-click accept, edit or reject. Corrections should be fast, with a required reason for rejects.
  • Priority and ageing. Sort by due date or payment terms. Alert when items wait too long.
  • Roles and permissions. Reviewers see only what their job needs. Changes to bank details need a second approver.
  • An audit log. Record who changed what, from which value to which, and when. Keep the model version and rule results with it.
  • A feedback path. Store corrections as labelled data. Use them to tune thresholds and test new models, and to spot a supplier whose layout keeps failing.

Design for reviewer fatigue. If people see mostly correct items, they stop looking. Keep the share of items that reach review small enough that each one is worth reading, and use sampling and seeded known-bad items to check that reviewers are still paying attention.

Worked example: how should accounts payable automation handle accuracy?

In AP, an extraction error becomes a payment. But not every field matters equally, and a good design spends its accuracy budget where the money is. Rank fields by the cost of an error and by whether a later control would catch it.

FieldCost if wrongCaught later?Handling
Total and tax amountsCost if wrong: Overpayment or underpaymentCaught later?: Only by the sum check and the PO matchHandling: Strict threshold. Always run sum checks. Review on any doubt
Supplier identity and tax IDCost if wrong: Payment to the wrong party, wrong tax treatmentCaught later?: Vendor master matchHandling: Match to the vendor master. Unknown supplier goes to review
Bank account detailsCost if wrong: Payment to a fraudsterCaught later?: Often not caughtHandling: Never accept changed bank details from an invoice alone. Verify through a known contact before updating the vendor record
PO number and quantitiesCost if wrong: Wrong match, wrong approval routeCaught later?: Three-way matchHandling: Match against open POs and goods receipts. Mismatch goes to review
Invoice number and dateCost if wrong: Duplicate payment, wrong periodCaught later?: Duplicate checkHandling: Check against posted invoices for the same supplier
Line descriptionsCost if wrong: Poor coding suggestionsCaught later?: Approver sees themHandling: Allow lower confidence. Wrong text rarely moves money
Supplier address, remarks, payment terms textCost if wrong: LowCaught later?: Often irrelevant to postingHandling: Extract if useful. Do not block on it

Three controls do most of the work. First, a match to your own records (purchase order, goods receipt, vendor master) catches errors that no confidence score can. Second, a duplicate check on supplier plus invoice number, and on the file hash, stops the same invoice being paid twice. Third, a bank-detail change rule: treat any change as a separate controlled process, never as a side effect of reading an invoice.

In India, also check the format of the supplier's GSTIN and, where the invoice is an e-invoice, the Invoice Reference Number (IRN). The GST e-invoice system (IRP) returns a unique IRN, a digitally signed e-invoice and a QR code for B2B invoices, so those fields give you something to validate and not just read. Which invoices must carry an IRN, and how input tax credit rules apply, are questions for your tax adviser.

Does your document pipeline handle personal data lawfully?

Invoices, KYC files, CVs and forms often hold personal data: names, phone numbers, bank accounts, Aadhaar or PAN details. Once the pipeline is live, it is one more place your data goes. The checks are the same as for any AI feature. Map every store (the raw files, extracted fields, review images, logs and any prompts sent to a model). Decide how long each is kept and how it is erased. Check where each vendor processes the data and what its contract says about training on your inputs. Our guide to DPDP compliant AI architecture walks through this store by store, and DPDP erasure automation covers deletion and retention.

If documents are sensitive, or a customer's contract limits where data can go, you can run extraction on infrastructure you control. Open-weight models and self-hosted OCR can run in your own cloud account or servers. That is the idea behind our Own Your AI service. Treat it as an option to compare on cost, accuracy and operations, and not as a default. Whether a given use is lawful is a question for counsel.

How do you measure whether the pipeline works?

Measure at the field level, on a labelled sample of your own documents. A single accuracy number hides the fields that matter.

  • Field accuracy for each critical field, against a hand-checked sample.
  • Straight-through rate: the share of documents that need no human touch. Report it next to error rate, because one can be bought at the cost of the other.
  • Escaped errors: wrong values found after acceptance, from sampling or downstream corrections. This is the number that tells you whether thresholds are too loose.
  • Review load and time per item, to size the team and spot bad document types.
  • Drift: accuracy by supplier or form type over time. A layout change shows up here first.

Re-run your labelled sample whenever you change the model, the prompt, the OCR service or a threshold. Treat those changes like code releases.

What should you build first?

  • Pick one document type with enough volume and a clear cost of error. Invoices are a common start.
  • Collect a labelled sample of your own documents, including the ugly ones. Define the fields and the correct value for each.
  • Write the schema and the business rules before you choose a model.
  • Try one document service and one LLM route against the sample. Compare field accuracy, cost and time per page.
  • Build the review queue and audit log, then turn on automatic acceptance only for fields and suppliers that have earned it.
  • Add sampling from day one, and review the numbers weekly for the first months.

If your team wants help choosing use cases and sequencing the work, our AI adoption service covers it, including document automation.

Sources: AWS Textract API reference, AnalyzeDocument, and Amazon Augmented AI task type for Textract (docs.aws.amazon.com), checked 20 November 2026. Google Cloud, Document AI Human-in-the-Loop (docs.cloud.google.com/document-ai/docs/hitl). Microsoft Learn, Document Intelligence accuracy and confidence scores, updated 19 October 2026. JSON Schema specification, version 2020-12 (json-schema.org). Anthropic, Structured outputs (platform.claude.com). OpenAI, Structured Outputs guide (developers.openai.com). GST e-Invoice System (einvoice1.gst.gov.in). Last reviewed 20 November 2026.

FAQ

What is a document intelligence pipeline?

It is a set of steps that turns scans, PDFs and forms into structured data a system can trust. The steps are intake, classification, extraction, validation, routing to automatic acceptance or human review, and an audit log. The extraction model is one step. Validation rules and a review queue decide whether the output is safe to use.

Is schema-valid JSON from an LLM the same as correct data?

No. Structured output features can guarantee that a response matches your JSON Schema, but they do not check meaning. A total can be well-formed and wrong. OpenAI's own guide says apps must still validate business rules the schema does not express. Add checks that totals add up, IDs have the right format and values match your records.

Where should a human review extracted documents?

Review documents where a critical field has low confidence, where a business rule fails, and on a random sample of documents the system accepted. Set thresholds per field, so a doubtful bank account goes to review while a doubtful address does not. Sampling matters because confidence scores can be wrong.

Should I use Amazon A2I or Google Document AI human review for a new build?

No. AWS docs say Amazon A2I is no longer open to new customers, and Google lists Document AI Human-in-the-Loop as deprecated since 16 January 2024 (both checked 20 November 2026). Build your own queue with a confidence threshold, a reviewer screen and an audit log. Their trigger ideas still work.

Which accounts payable fields need the highest accuracy?

Amounts, tax, supplier identity, bank details, PO number and invoice number, because errors there become wrong or duplicate payments. Line descriptions and address text can tolerate more doubt. Back the critical fields with checks against your purchase orders, vendor master and posted invoices, and never accept changed bank details from an invoice alone.

Does a document pipeline need to run on private infrastructure?

Not always. Managed document services and LLM APIs can be used with the right contracts, retention settings and data map. Private or self-hosted extraction is an option when documents are sensitive or a contract limits where data can go. Compare accuracy, cost and operating effort on your own sample, and take legal questions to counsel.

Turn document backlogs into checked, structured data.