Context Window
Also: Context Length, Token Window, Maximum Context Length
The context window is the maximum amount of text (measured in tokens) a language model can process at once, including both the input prompt and the generated output.
What It Is
The context window is the fixed limit on how much text a large language model (LLM) can consider at a single time. It is measured in tokens, not words or characters. A token is a chunk of text, roughly 3 to 4 characters in English, so 1,000 tokens equals about 750 words.
The context window covers everything in a single interaction:
- The system prompt (instructions defining the model's behavior)
- The user input (your question or task)
- Any retrieved documents or pasted content
- The conversation history in a chat
- The model's own response as it is generated
If the combined total exceeds the window, the oldest or least relevant content must be dropped or truncated.
Why it matters
The context window defines the practical limits of what a model can reason about. A model cannot use information it never received, and it cannot remember anything outside the current window unless that information is fed back in.
Key implications:
- Larger windows allow processing long documents, full codebases, or extended chats, but cost more and can slow responses.
- Smaller windows force you to summarize, chunk, or selectively retrieve content.
- Models may show degraded accuracy in the middle of very long inputs, an effect often called "lost in the middle."
How it is used in practice
Professionals manage the context window through several techniques:
- Chunking: splitting long documents into smaller pieces.
- Retrieval Augmented Generation (RAG): pulling only the most relevant passages into the window.
- Summarization: condensing earlier chat turns to free up space.
- Prompt budgeting: reserving enough tokens for the output.
Concrete Example
Suppose a model has an 8,000 token context window. You paste a 6,000 token contract and ask a question using 200 tokens. That leaves roughly 1,800 tokens for the system prompt and the answer. If you paste a 9,000 token contract instead, it will not fit, so you must summarize it or retrieve only the relevant clauses. Planning around this budget is essential for reliable, cost effective AI applications.