Working with long documents and big inputs

Drop a 60-page contract into ChatGPT and ask "extract every clause," and you will get a confident, fluent answer that quietly skips a third of the document. The model did not lie. It just hit a wall you could not see. This lesson is about seeing that wall, then engineering around it.

The two limits you are actually fighting

There are two separate constraints, and people conflate them constantly.

The first is the context windowcontext windowThe context window is the maximum amount of text (measured in tokens) a language model can process at once, including both the input prompt and the generated output.View full definition →: how much text the model can hold in working memory at once. Modern ChatGPT models have large windows, but "large" is not "infinite," and the window holds your file *plus* your instructions *plus* the entire prior conversation *plus* the model's own output. A 60-page contract can easily eat a big chunk of that on its own.

The second is the attention budget: even when text technically fits, models attend unevenly across a long input. Content in the middle of a huge block gets weaker treatment than content at the start or end. This is why a contract that "fits" still produces a summary that misses clause 34 buried on page 41.

So the failure mode is rarely a hard error. It is silent omission. **Your job is to design prompts and pipelines that make omission *visible* and *cheap to catch***.

How ChatGPT ingests a file (and why it matters)

When you upload a PDF in ChatGPT, one of two things happens depending on the tool in play.

If Advanced Data Analysis (the Code Interpreter sandbox) is active, ChatGPT can run Python to open the file, parse it, and extract text page by page. It is not "reading" the whole thing into the model at once. It is writing code that processes the document programmatically, then reasoning over the results.

If you are in a plain chat without the sandbox, the file gets read into context more directly, and you are far more exposed to both limits above.

This distinction is the single most important lever you have. For a long, structured document, you want the model writing code over the file, not stuffing the file into its head. See OpenAI's overview of file handling and tools for current behavior.

Strategy 1: Force a structured extraction, not a summary

"Summarize this contract" is the wrong ask. Summaries compress, and compression is where clauses die.

Instead, define a schemaschemaA schema is the formal blueprint that defines how data is structured, named, typed, and related within a database, file, or message.View full definition →: the exact fields you want for every item, repeated mechanically. This converts a fuzzy reading task into a fill-in-the-blanks task, which models do far more reliably.

Here is the prompt pattern for the contract:

Go through the attached contract section by section. For every numbered clause and sub-clause, output a row with: clause_id, heading, verbatim_text (first 200 chars), plain_english, obligations_on_us, obligations_on_them, risk_flag (low/med/high). Do not summarize across clauses. If a clause has sub-parts, list each sub-part as its own row. End with a count of total clauses found.

Two things make this work. The per-clause schema prevents the model from collapsing detail, and the final count gives you a cheap integrity check. If page-1 numbering ends at 9.4 but the model reports 22 clauses, you have a discrepancy to chase.

Strategy 2: Chunk on purpose, not by accident

For a 60-page contract, do not paste the whole thing and hope. Chunk it into logical units (by Article, by Section), and process each chunk with the *same* schema. Chunking here means splitting the input into pieces small enough that the model attends to all of each piece.

The key rule: chunk on semantic boundaries, not arbitrary character counts. Splitting mid-clause creates orphaned text that confuses extraction. Split at "Article 5," "Article 6," and so on.

In ChatGPT, you can do this conversationally inside one Project (so the schema and instructions persist):

1. "Here is the contract. First, list every Article heading and its page range. Do not extract yet."
2. Verify that list against the table of contents yourself.
3. "Now extract clauses for Article 1 through 3 only, using the schema."
4. Repeat per Article batch, then ask for a merged table.

Doing it in a Project matters: your schema, custom instructions, and the uploaded file stay scoped to that workspace, so every batch inherits the same rules without re-pasting.

Strategy 3: Let Code Interpreter do the heavy lifting

For a 60-page document, the most reliable path inside ChatGPT is to make Advanced Data Analysis split the file deterministically, then extract per chunk. You can literally ask:

Using Python, open the PDF, split the text by the regex for clause numbers (e.g. lines starting with \d+\. or \d+\.\d+), and show me how many clauses you detected and their IDs. Don't extract content yet.

Now the *code* found the clauses, not the model's attention. You get a deterministic count to anchor everything else. Then you extract clause text in batches, again via code, so nothing silently vanishes.

When ChatGPT is not enough: the APIAPIApplication Programming Interface: a standardised interface that lets applications communicate and exchange data without knowing each other's internal workings.View full definition → and structured outputs

The chat UI is great for one-off contracts. For dozens of contracts, or for output you can trust without eyeballing every row, move to the OpenAI API, where you can enforce the schema programmatically.

The relevant feature is Structured Outputs: you hand the model a JSON Schema, and the response is *guaranteed* to match that shape. No "the model forgot a field" surprises. Combine it with chunking and you have a real extraction pipelinepipelineAll active sales opportunities across the stages of the sales process, together with their combined potential value and probability of closing.View full definition →.

Here is a compact example using the Responses API. You loop over pre-split chunks and force each result into your clause schema:

from openai import OpenAI

client = OpenAI()

clause_schema = {
    "type": "object",
    "properties": {
        "clauses": {
            "type": "array",
            "items": {
                "type": "object",
                "properties": {
                    "clause_id": {"type": "string"},
                    "heading": {"type": "string"},
                    "plain_english": {"type": "string"},
                    "risk_flag": {"type": "string", "enum": ["low", "med", "high"]},
                },
                "required": ["clause_id", "heading", "plain_english", "risk_flag"],
                "additionalProperties": False,
            },
        }
    },
    "required": ["clauses"],
    "additionalProperties": False,
}

def extract(chunk_text: str) -> dict:
    resp = client.responses.create(
        model="gpt-4.1",
        input=[
            {"role": "system", "content": "Extract every clause. Do not summarize across clauses."},
            {"role": "user", "content": chunk_text},
        ],
        text={"format": {"type": "json_schema", "name": "clauses", "schema": clause_schema, "strict": True}},
    )
    return resp.output_text

for i, chunk in enumerate(article_chunks):
    print(f"Article {i + 1}:", extract(chunk))

Because strict is True, every chunk returns valid JSON in exactly your shape. You merge the arrays, dedupe by clause_id, and you have a complete, machine-checkable extraction. The official reference is Structured Outputs in the OpenAI docs.

Note what we did *not* do: we did not send 60 pages in one call. We sent semantic chunks, each well within comfortable attention range, and the schema guaranteed completeness per chunk.

Verification: the step everyone skips

A long-document pipeline is only as good as its checks. Three cheap ones catch most failures.

Count reconciliation. Code Interpreter (or your regex) found N clause IDs. Your extracted table has M rows. If N does not equal M, list the missing IDs. This alone catches the most common silent-omission bug.

Verbatim spot-check. Ask: "For clauses 12.3, 31.1, and 48.2, paste the exact source text next to your plain-English version." If the verbatim text does not exist in the document, the model invented it, and you have caught a hallucinationhallucinationA hallucination is when an AI model generates output that is fluent and confident but factually wrong, fabricated, or unsupported by its source data.View full definition → before it reached your summary.

Adversarial re-read. "Re-scan the document for any indemnification, termination, or auto-renewal language NOT already in the table." Targeted re-reads find what a single pass missed, because you are now directing attention to specific risk categories.

Choosing your approach

A quick decision guide for long inputs:

• One document, exploratory: ChatGPT chat with Advanced Data Analysis, plus the structured-extraction prompt. Fast, conversational, good enough.
• One document, high stakes: Same, but add all three verification steps and demand verbatim quotes for any high-risk clause.
• Many documents, repeatable: API with chunking and Structured Outputs. You get auditable JSON and can diff contracts against each other.
• An ongoing intake (e.g. every new contract that lands in a folder): wrap the API pipeline and trigger it. Connectors and the broader agent tooling can route documents in, but the extraction core stays the same: chunk, schema, verify.

The temptation with bigger context windows is to throw the whole document in and trust the model. Resist it. A 60-page contract is not a context-window problem you solve by buying more window. It is an attention and verification problem you solve with structure.

Key Takeaways

• Schema beats summary. Define exact per-clause fields and demand a final count. This turns omission from invisible into measurable.
• Chunk on semantic boundaries (Articles, Sections), never arbitrary character counts, and process each chunk with the identical schema.
• Make code find the structure first. Have Advanced Data Analysis split the PDF and count clauses deterministically before any extraction, so you have a ground-truth number to check against.
• For scale and trust, move to the API with Structured Outputs (strict: true) so every chunk returns guaranteed-valid JSON you can merge and audit.
• Always run the count reconciliation and a verbatim spot-check. A long-document pipeline without verification is just a confident guess.