On this page
Read
The useful mental model
A context window is a fixed workbench for one model call. Instructions, conversation history, retrieved evidence, tool results, and the answer all compete for room, so builders must choose what deserves to stay on the bench.
In plain English
How much the AI can keep in mind at one time.
Where builders use it
- 01Long document summarization
- 02Conversation memory
- 03Complex planning prompts
Why it matters
Instructions, conversation history, tool results, retrieved evidence, and the model's output share finite space. Exceeding that space can fail or truncate a request, while filling it with irrelevant material can make useful context harder to use.
Why now
Long-context models can process much larger working sets, but agent loops, tool output, and multimodal inputs can consume that capacity quickly. Context management is therefore part of application design, not only prompt writing.
A common misconception
“The context window is the model's permanent memory.”
It is the working information available to one request. Long-term memory requires an external store plus a method for selecting what returns to context.
“A larger context window means every included detail will be used equally well.”
Capacity is not guaranteed recall or reasoning quality. Relevant, well-structured context can work better than filling the window with everything available.
See
See the mental model
The visual is generated from the same structured knowledge as the written explanation.
A fixed context budget is divided between pinned instructions, relevant retrieved evidence, compacted prior work, and reserved output space for the next workbook section.
Watch
Watch the explainer
A short visual explanation built from the same reviewed knowledge object.
Context Window: choose what belongs in this call
See how an AI assistant curates a long coding session into one useful model call by pinning requirements, retrieving current evidence, compacting older work, and reserving room for the answer.
Read transcript
A coding assistant is turning a long session into a workbook. The session holds the brief, decisions, failed tries, screenshots and tool output. A context window is the limit on what the model can read in one call. Instructions, history, evidence, tool results and the answer all share it. If everything is sent, old noise crowds out what matters now. So the assistant pins the brief and the active rules. It fetches only the passages this step needs. It squeezes old work into a short checked summary, and saves space for the output. A bigger window holds more, but it is not memory. Ask what the model needs for this step, not how much fits.
Builder example
Keep a coding-session workbook inside a useful context
An AI assistant using MCP tools is turning a long landing-page coding session into a reproducible build workbook. The session contains the original brief, decisions, failed attempts, screenshots, and repeated tool output.
Failure
Sending the entire raw session for every workbook section consumes the shared token budget, leaves less room for the answer, and makes the current constraints harder to find among stale details.
Intervention
Keep the build brief and active constraints pinned, retrieve only the coding-session passages needed for the current section, reserve output space, and compact older working notes into a checked summary.
Observable result
Each workbook section is generated from the relevant evidence and current constraints while the request remains within the selected model's token budget.
Sources
Go deeper
- OpenAI API - Conversation state and context windows
- OpenAI API - Key concepts
- Anthropic - Effective context engineering for AI agents
Last researched 11 October 2026