Convert PDF to Markdown for LLMs without losing tables

Markdown sits in a useful middle ground. It is plain text, so it costs few tokens and every model reads it without special handling. It still carries structure: a line that starts with "##" is a heading, a line that starts with "-" is a list item, and a pipe table is a table with rows and columns.

That structure matters more than it looks. When a model answers "what was the Q3 revenue for the EMEA segment", it has to find the right table, the right row and the right column. In flat text the row label and the number may sit forty tokens apart with other numbers in between. In a Markdown table they sit on the same line, separated by a pipe.

Headings matter for retrieval too. If you split a document into chunks for search, a chunk that begins with "## 4.2 Termination for convenience" is far easier to match and to cite than a chunk that begins in the middle of a sentence.

• Headings become "#", "##" and "###" lines that mirror the document outline.
• Paragraphs stay whole instead of breaking at every visual line end.
• Lists keep their bullets and numbering.
• Tables become pipe tables with one header row and one line per data row.
• Page furniture such as running headers, footers and page numbers is removed.

A PDF page is a set of drawing instructions. The content stream says "place these glyphs at x=72, y=708 with this font", then "move to x=310 and draw these glyphs". There is no paragraph object, no table object and often no reliable word spacing. Everything a human sees as structure is implied by geometry.

Some PDFs carry a tagged structure tree that marks headings, paragraphs and tables. Many do not, and the tags that exist are often wrong because the authoring tool guessed. Scanned pages have no text at all, only an image, so they need OCR before any of this applies.

Here is the simplest possible extraction with an open source library. It works on a clean single column page and fails in instructive ways on anything else.

# pip install pypdf
from pypdf import PdfReader

reader = PdfReader("annual-report.pdf")
for number, page in enumerate(reader.pages, start=1):
    text = page.extract_text() or ""
    print(f"--- page {number} ---")
    print(text[:800])

On a two column research page, the output often alternates between the left and right column line by line, because the extractor emits text in the order it appears in the content stream or sorted by vertical position. On a financial table, it prints "Revenue 1,204 1,377 1,512 Cost of sales 640 702 760" as one line, and the information about which number belongs to which year is gone.

Reading order is the sequence a person follows: down the left column, then down the right column, then the full width footnote. Sorting text by y coordinate gives the wrong answer for every multi column page, because line 1 of the left column and line 1 of the right column share the same y.

A better approach groups characters into lines, lines into blocks, and then orders blocks. You can get closer with layout aware open source tools that expose character positions.

# pip install pdfplumber
import pdfplumber

with pdfplumber.open("two-column.pdf") as pdf:
    page = pdf.pages[0]
    mid = page.width / 2
    left = page.crop((0, 0, mid, page.height))
    right = page.crop((mid, 0, page.width, page.height))
    text = (left.extract_text() or "") + "\n\n" + (right.extract_text() or "")
    print(text)

This fixes one page layout and breaks the next one. A full width title above two columns gets cut in half. A three column newsletter needs different crop boxes. A figure caption that spans both columns ends up split. Hard coding geometry does not scale past a handful of templates.

A layout model looks at the whole page as an image plus its text layer, finds blocks, classifies them as heading, paragraph, table, list or figure, and predicts the order between blocks. That is what PdfParse does on every page, and the order is returned explicitly so you can check it.

Running headers, footers and page numbers

A 200 page report repeats "Company Name, Annual Report 2025" at the top of every page and "Page 37 of 200" at the bottom. Left in place, these lines land in the middle of sentences that cross page breaks, and they pollute every chunk with the same tokens.

A reliable heuristic finds text blocks that appear at nearly the same position on most pages with nearly the same content, allowing a changing number. Removing them before joining pages restores sentences that cross page boundaries.

import re
from collections import Counter

def normalize(line):
    return re.sub(r"\d+", "#", line.strip().lower())

def strip_repeating_lines(pages, min_share=0.6):
    """pages is a list of lists of lines, one list per page."""
    edges = Counter()
    for lines in pages:
        for line in lines[:2] + lines[-2:]:
            edges[normalize(line)] += 1
    limit = max(2, int(len(pages) * min_share))
    repeated = {key for key, count in edges.items() if count >= limit}
    return [[l for l in lines if normalize(l) not in repeated] for lines in pages]

This handles the common case. It misses headers that alternate between odd and even pages and can remove a legitimate line that happens to repeat, so treat it as a filter you test, not a guarantee.

Headings without tags

Without a structure tree, heading level has to be inferred from font size, weight, numbering ("3.2.1") and spacing. Font size alone is unreliable because a document may use the same size for a section title and a bold table caption. A practical signal is to rank distinct font sizes used for short lines, map the largest to "#", the next to "##", and so on, then confirm with numbering patterns.

Tables are where most conversions fail

A table in a PDF is usually text placed on a grid, sometimes with ruling lines and sometimes with nothing but whitespace. To rebuild it you need column boundaries, row boundaries, and the header row. Merged cells make it harder: a header such as "2025" spanning "Q1" to "Q4" belongs to four columns, and a row label spanning three rows belongs to all three.

Markdown pipe tables cannot express merged cells. The honest conversion repeats the merged value into each covered cell, or flattens a two level header into "2025 Q1", "2025 Q2". Both keep every number next to its correct label, which is what the model needs. For a deeper walk through of grid detection and merged cells, see our guide on how to extract tables from pdf python and the pdf table extraction page.

Footnotes, figures and captions

Footnotes are small text at the bottom of a page that belongs to a sentence higher up. Left in reading order, they interrupt a paragraph that continues on the next page. A cleaner conversion moves each footnote to the end of its section or keeps it as a separate block with a reference marker, so the main text reads continuously.

Figures carry no text of their own, but their captions do, and a caption such as "Figure 3. Revenue by segment" is often the best summary of what the figure shows. Keep the caption as its own paragraph next to the place where the figure sits, and drop text that is only part of a chart, such as axis tick labels, which add numbers without meaning.

Scanned pages and mixed documents

Many real PDFs mix born digital pages with scanned ones: a signed contract with a scanned signature page, or a report with a scanned appendix. A text extractor returns nothing for the scanned pages and you silently lose content. Check every page for a text layer, and send pages without one through OCR. Our ocr api python guide covers rotation, low DPI scans and confidence scores in detail.

from pypdf import PdfReader

def pages_without_text(path, min_chars=20):
    reader = PdfReader(path)
    empty = []
    for number, page in enumerate(reader.pages, start=1):
        if len((page.extract_text() or "").strip()) < min_chars:
            empty.append(number)
    return empty

print(pages_without_text("contract.pdf"))

PdfParse runs layout detection, reading order, header and footer removal, table reconstruction and OCR for scanned pages in one request. You ask for Markdown, and you can ask for JSON blocks and CSV tables in the same call. Install the Python SDK and set your key as an environment variable.

pip install pdfparse-ai
export PDFPARSE_API_KEY="pp_live_your_key"

import os
from pdfparse_ai import PdfParse

client = PdfParse(api_key=os.environ["PDFPARSE_API_KEY"])
result = client.parse(
    "annual-report.pdf",
    outputs=["markdown", "json"],
    ocr="auto",
)

with open("annual-report.md", "w", encoding="utf-8") as f:
    f.write(result["markdown"])

print("pages billed:", result["usage"]["pages"])

The same request with curl, for quick testing or for languages without an SDK:

curl https://api.pdfparse.ai/v1/parse \
  -H "Authorization: Bearer $PDFPARSE_API_KEY" \
  -F "[email protected]" \
  -F 'outputs=["markdown","json"]' \
  -F "ocr=auto"

Small files return the result directly. Larger files return a job id, and you read the status from GET /v1/jobs/{id} or receive a parse.completed webhook. The full request reference is in the document parsing api documentation, and the pdf to markdown page shows the output on real layouts.

What the Markdown looks like

For a page with a heading, a paragraph and a table, the output reads like a document a person wrote by hand.

## 4. Segment results

Revenue grew in every segment. EMEA remained the largest contributor.

| Segment | 2024 Q4 | 2025 Q1 | 2025 Q2 |
|---|---|---|---|
| EMEA | 1,204 | 1,377 | 1,512 |
| Americas | 980 | 1,021 | 1,104 |
| APAC | 512 | 560 | 601 |

Each block also appears in the JSON output with its type, text, bounding box and confidence, so you can trace any Markdown line back to its position on the page.

Conversion errors are quiet. A model will answer confidently from a table whose columns shifted by one. A few cheap checks catch most problems before they reach a prompt.

• Count tables per page in the JSON and compare with what you expect for known templates.
• Check that every Markdown table row has the same number of cells as its header.
• Parse numeric columns and verify totals where the document prints a total row.
• Flag blocks with low confidence and route those pages to review.
• Compare the page count in "usage" with the page count of the file.

def check_markdown_tables(markdown):
    problems = []
    lines = markdown.splitlines()
    i = 0
    while i < len(lines):
        if lines[i].startswith("|") and i + 1 < len(lines) and set(lines[i + 1]) <= set("|-: "):
            width = lines[i].count("|")
            j = i + 2
            while j < len(lines) and lines[j].startswith("|"):
                if lines[j].count("|") != width:
                    problems.append((j + 1, "cell count differs from header"))
                j += 1
            i = j
        else:
            i += 1
    return problems

def low_confidence_blocks(result, threshold=0.8):
    flagged = []
    for page in result["pages"]:
        for block in page["blocks"]:
            if block.get("confidence", 1.0) < threshold:
                flagged.append((page["number"], block["type"], block["text"][:60]))
    return flagged

Pick the threshold from your own documents. Look at twenty flagged blocks and twenty unflagged ones, and move the threshold until the flagged set contains the blocks you would want a person to check.

If the Markdown goes into a retrieval index, split by section, not by a fixed character count. A chunk should end at a heading boundary, keep each table whole, and carry the heading path so the model knows where it came from. PdfParse returns these chunks directly when you pass a chunking option, with page range, section path and bounding box on every chunk.

result = client.parse(
    "annual-report.pdf",
    outputs=["markdown"],
    chunking={"strategy": "section", "max_tokens": 800},
)

for chunk in result["chunks"][:3]:
    path = " > ".join(chunk["section_path"])
    print(f"[{path}] pages {chunk['page_start']}-{chunk['page_end']}")
    print(chunk["text"][:200])

Storing "section_path" and the page range next to each vector lets your application cite "Annual report, section 4 Segment results, page 37" instead of an anonymous text fragment. RAG chunking with metadata is part of the Growth plan and above, see pdf extraction api pricing.