Skip to content

Parse PDFs and scanned pages

A PDF may look like a normal document to you, but the file does not always store paragraphs in reading order. It can store individual characters at positions on a page.

That is why copying text from a two-column PDF sometimes mixes both columns together.

First identify the kind of PDF

I begin by checking whether the PDF contains selectable text.

A text PDF stores characters that a parser can extract. A scanned PDF contains page images. Scanned pages need OCR, which means optical character recognition: software looks at the image and turns visible letters into machine-readable text.

Some PDFs combine both. A report may contain normal text pages and scanned appendices.

Try direct extraction

For a straightforward text PDF, pypdf gives you a useful baseline:

uv add pypdf

from pathlib import Path
from pypdf import PdfReader

path = Path("report.pdf")
reader = PdfReader(path)

pages = []
for page_number, page in enumerate(reader.pages, start=1):
    pages.append({
        "page": page_number,
        "text": page.extract_text() or "",
    })

Keep page numbers with the text. They are the most useful Citation location a PDF gives you.

Read the output from pages with headings, lists, tables, and multiple columns. Direct extraction is enough only if the reading order remains usable.

Use layout-aware parsing when needed

Libraries such as Docling and Unstructured try to recognize page layout, headings, tables, and images. They add larger dependencies and may use local machine-learning models.

I choose them when direct extraction repeatedly scrambles the Sources I need. I compare the output on a small set of representative PDFs before choosing a library for the whole collection.

For scans, use an OCR engine and keep its confidence score when available. Low-confidence text should be easy to inspect or exclude.

Do not hide extraction failures

A parser should record:

  • Source path and content hash;
  • parser name and version;
  • page number;
  • whether OCR was used;
  • extracted text;
  • warnings or confidence information.

Reject encrypted files you cannot open, record empty pages, and cap page count and file size before parsing untrusted uploads.

Evaluate the parsed text

Search for known phrases, compare extracted tables with the visible PDF, and include parser-sensitive examples in the Evaluation Set. A retrieval failure caused by scrambled columns will not be fixed by changing the embedding model.

I keep the original PDF available for Citation links and debugging. The extracted text is a searchable representation, not a replacement for the Source.