The context window: the model's working memory

Paste a 40-page PDF into Claude and ask for a summary. It works instantly. Paste a 900-page novel and you hit a wall: "This message exceeds the maximum length." Same tool, same prompt, totally different outcome. The difference is the context windowcontext windowThe context window is the maximum amount of text (measured in tokens) a language model can process at once, including both the input prompt and the generated output.View full definition →, and once you understand it, half the weird behavior of AI chatbots suddenly makes sense.

The one thing to understand first

An LLMLLMA Large Language Model is an AI system trained on vast text data to predict and generate language, enabling tasks like writing, summarizing, and answering questions.View full definition → (large language model, the technology behind ChatGPT, Claude, and Gemini) has no memory between conversations. When you start a new chat, the model knows nothing about your last one. It doesn't "remember" you the way a coworker would.

Everything the model "knows" during a chat lives in one place: the context window.

Think of the context window as the model's working memory, like the desk space in front of you. Whatever fits on the desk, the model can see and use. Anything that falls off the edge is gone.

This includes:

• Your messages
• The model's replies
• Any files or text you pasted in
• Hidden system instructions the app adds behind the scenes

All of it sits on the same desk. And the desk has a fixed size.

TokensTokensA token is the basic unit of text that language models process, often a word fragment, whole word, or punctuation mark rather than a single character.View full definition →: the budget you're spending

The size of the context window is measured in tokens, not words.

A token is a chunk of text. Roughly, 1 token is about 4 characters, or three-quarters of a word. So 100 tokens is around 75 words.

Quick examples:

• "cat" = 1 token
• "unbelievable" = often 3 or 4 tokens (broken into pieces)
• A typical page of text = about 500 to 700 tokens

You can see this yourself with OpenAI's free Tokenizer tool. Paste any text and watch it split into colored tokens. It's the fastest way to make this concept click.

Why does this matter? Because the context window is a budget, and tokens are the currency.

In early 2026, typical limits look like this:

• ChatGPT (GPT-5 class models): around 128,000 to 256,000 tokens depending on the tier
• Claude (Sonnet and Opus 4.x): 200,000 tokens, with some enterprise options far higher
• Gemini 2.x: up to 1,000,000 tokens or more in some versions

A million tokens sounds infinite. It isn't. Let's do the math.

Why the 40-page PDF works (and the novel doesn't)

A 40-page PDF is roughly 20,000 to 28,000 tokens. Drop that into a 200,000-token window and you've used maybe 12 percent of the desk. Plenty of room left for your questions and the model's answers.

A 900-page novel is roughly 400,000 to 500,000 tokens. That overflows even a 200,000-token window. The app rejects it because it physically cannot fit on the desk.

This is the practical rule:

If your content fits in the window, the model can reason over all of it at once. If it doesn't, you have to break it up.

When something is too big, your options are:

1. Split it into chunks and process them one at a time.
2. Summarize as you go, feeding summaries forward instead of full text.
3. Use a model with a bigger window (this is exactly why Gemini's million-token window is useful for huge documents).

Why long chats start to "forget" the beginning

Here's the behavior that confuses people most.

You're 50 messages deep into a long brainstorming chat. You ask the model to recall a detail you mentioned at the very start. It gets it wrong, or invents something.

The model isn't being lazy. The beginning of your conversation fell off the desk.

Remember: every message, yours and the model's, takes up token budget. A long conversation keeps adding tokens. When the total exceeds the window, the oldest content gets pushed out to make room for the newest.

So in a very long chat, the model may literally no longer see what you said an hour ago. It's not in the working memory anymore.

A simple mental model:

[ CONTEXT WINDOW = 200,000 tokens ]

Earliest messages  -->  pushed out when full
   ...
Recent messages    -->  always visible
Your latest prompt -->  always visible

This is also why the model can contradict itself in long sessions. The instruction you gave at message 3 ("always reply in British English") may be gone by message 80.

"But ChatGPT remembers my name across chats?"

Good catch. Modern apps add features on top of the raw context window:

• Memory (ChatGPT, Gemini): the app saves a few facts about you and quietly re-inserts them into new chats.
• Projects (Claude, ChatGPT): a shared space where instructions and files persist across conversations.

These aren't the model remembering. They're the app re-feeding information into the context window each time. The window is still the only thing the model actually sees.

How to work with the window, not against it

Once you treat tokens as a budget, you start making better choices. Here's the practical mindset.

1. Start fresh for new topics

Don't run your whole week through one endless chat. A bloated chat wastes budget on irrelevant history and increases the chance of confusion. New task, new chat.

2. Front-load what matters

Put your most important instructions and reference material near your actual question, not buried 40 messages up. If a rule matters, restate it.

3. Paste the relevant part, not everything

You don't need to paste a full 200-page contract to ask about the termination clause. Paste the clause and a bit of surrounding context. Less noise, sharper answers, lower cost.

4. Summarize to "save" a long chat

When a long conversation gets sluggish or forgetful, ask:

"Summarize everything we've decided so far as a bullet list I can paste into a new chat."

Then start a clean chat with that summary on top. You've compressed hours of work into a few hundred tokens.

5. Use Projects for ongoing work

If you keep returning to the same material (a book draft, a product spec, a client account), put it in a Project so it's always loaded without re-pasting.

A concrete example: counting your budget

If you build with the APIAPIApplication Programming Interface: a standardised interface that lets applications communicate and exchange data without knowing each other's internal workings.View full definition → (the developer interface to these models), token usage is reported to you directly. Even non-developers benefit from seeing it once.

# pip install anthropic
import anthropic

client = anthropic.Anthropic()  # uses your API key

response = client.messages.create(
    model="claude-sonnet-4-5",
    max_tokens=300,
    messages=[
        {"role": "user", "content": "Summarize the causes of WWI in 5 bullets."}
    ],
)

print(response.content[0].text)

# The model tells you exactly how much budget you spent:
print("Input tokens:", response.usage.input_tokens)
print("Output tokens:", response.usage.output_tokens)

input_tokens is what you sent (your prompt plus any pasted files). output_tokens is what the model wrote back. Both count against limits and, on paid plans, against cost. Watching these numbers makes the whole concept tangible.

Common misconceptions, cleared up

"A bigger context window means a smarter model." No. It means a bigger desk, not a sharper mind. A model with a huge window can still misread what's on the desk.

"The model reads my whole document carefully every time." Not exactly. With very large inputs, models can overlook details buried in the middle, a known pattern sometimes called "lost in the middle." Important instructions belong near the start or end of your input.

"Once I tell it something, it knows it forever." Only within that chat, and only while it stays in the window. Across chats, only Memory or Projects carry it forward.

Key Takeaways

• The context window is the model's working memory. Everything the model can "see" (your messages, its replies, pasted files) lives there, and it has a fixed size measured in tokens.
• Tokens are a budget: roughly 75 words per 100 tokens. A 40-page PDF fits easily in a 200,000-token window; a full novel does not.
• Long chats forget the beginning because old messages get pushed out when the window fills. Restate important instructions, and start fresh chats for new topics.
• Summarize to preserve work: ask the model to bullet-point your decisions, then paste that into a clean chat to reclaim budget.
• Memory and Projects aren't real memory. They re-feed facts into the window each session, so use them for material you reuse, and paste only the relevant parts for everything else.