Vector stores need product discipline

A vector store can remember nearest neighbors. The product still has to define memory.

Chunking decides what can be retrieved. Prose, tables, code, policies, slide decks, and chat transcripts need different boundaries. A bad chunk can omit the exception, import, header, or speaker note that gives the result meaning.

Keep source shape in metadata:

{
  "chunk_id": "policy-refunds-v3:section-4:chunk-2",
  "document_id": "policy-refunds-v3",
  "document_version": "2024-07-18",
  "section": "regional exceptions",
  "chunk_kind": "policy_section",
  "text": "Customers in region CA may request manual review..."
}

Metadata is control, not decoration. Include workspace, document version, owner, permissions, language, content type, timestamps, retention, embedding version, and chunker version.

Filter before context assembly:

const results = await vectorStore.search({
  queryEmbedding,
  topK: 12,
  filter: {
    workspaceId,
    allowedDocumentIds,
    contentType: ["policy", "runbook"],
    archived: false,
  },
})

If the user cannot open the source document, the model should not retrieve its chunk.

Changing embedding models is a migration. Dimensions, similarity, multilingual behavior, and code retrieval can change. Stage a sample, compare results, build a shadow index, run evals, and keep rollback available.

Retrieval evals need required and forbidden results:

{
  "query": "Can a Canadian customer get a refund after 30 days?",
  "must_retrieve": ["policy-refunds-v3:section-4"],
  "must_not_retrieve": ["policy-refunds-v2:section-2"],
  "allowed_answer": "manual_review_required"
}

Memory also needs deletion. Source documents, chunks, embeddings, cached summaries, and derived features all need a removal and rebuild path. Prove deletion through scoped retrieval tests rather than a successful delete request alone.

Vector stores become useful when they are treated as memory systems with rules, not magical piles of semantic proximity.