GPT-4 Turbo made context a product budget

GPT-4 Turbo made context feel like a budget.

A larger window changes the question from “can this fit?” to “should this be included?” Context competes with latency, cost, privacy, and the user’s attention.

A request assembler can make the trade explicit:

context_budget:
  essential:
    - current user request
    - facts needed to answer safely
  helpful:
    - recent turns that prevent re-asking
    - a few examples
  expensive:
    - long history
    - every retrieved chunk
    - stale notes

A large window is not a reason to send everything. Retrieval may become less desperate, but stuffing every result into the prompt can make the answer slower, harder to debug, and more likely to trust irrelevant context.

Conversation history should not behave like a memory leak. Example blocks need a reason to remain. A growing prompt needs a policy for what enters, what expires, and what gets excluded.

The useful habit is simple: reserve space for the current request, keep a bounded recent window, and add retrieval only when the task needs it. More tokens create room for judgment. They do not remove the need for it.