Long context changed retrieval expectations

Long context changed my default unit for retrieval.

The old instinct was to make every chunk smaller. Smaller chunks were easier to index, easier to rank, and easier to fit into a prompt. They were also easy to misunderstand once the surrounding argument had been removed.

With a larger window, a retrieved result can be a section, a policy, or a complete page. That gives the model enough context to interpret the evidence instead of receiving a handful of sentences whose references point somewhere outside the prompt.

The selection loop can stay simple:

def select_context(query, candidates):
    scored = rank_by_similarity(query, candidates)
    ordered = sort_by_source_and_time(scored)
    return dedupe_by_section(ordered)[:8]

The difficult decisions are elsewhere:

• Is this source unit coherent?
• Does the ordering preserve the argument?
• Does the citation still identify the evidence?
• Was the source eligible for this answer?

Long context does not remove retrieval judgment. It changes the tradeoff. There is more room to keep evidence together, so retrieval can optimize for meaning and traceability instead of only minimizing token count.