What Is a Context Window, and Why Does Size Matter?

The model's working memory

A context window is the maximum amount of text — measured in tokens — that a model can consider at once, including your prompt, any conversation history, and the response it is generating. Anything beyond that limit simply is not seen by the model.

Tokens, not words

Context windows are measured in tokens, and a token is often shorter than a whole word — roughly four characters of English text on average. A window of 8,000 tokens holds somewhere around 6,000 words, which is why long documents can quietly exceed a limit that sounds generous at first.

words_estimate = tokens * 0.75   # rough rule of thumb for English text

What happens when you go over the limit

Once a conversation or document exceeds the context window, something has to be dropped — usually the oldest messages. This is why a long chat session can suddenly seem to "forget" something you mentioned earlier: it simply fell outside the window.

Bigger is not automatically better

A larger context window lets you paste in more reference material, but models do not always use that space evenly. Research on long-context models has repeatedly found a "lost in the middle" effect, where information placed in the middle of a very long prompt gets less attention than information near the beginning or end.

Working within the limit

For tasks with more content than fits in one window, the common approach is retrieval: instead of pasting an entire document, you search it for the most relevant sections and include only those in the prompt. This keeps the input focused and avoids diluting the model's attention across irrelevant text.

Key takeaways

The context window is the model's entire field of view for a single request — nothing outside it exists as far as the model is concerned. Bigger windows help, but relevance and placement of information inside that window still matter more than raw size alone.