Embeddings are product plumbing

An embedding is a useful hint about similarity, not an answer.

It turns text, images, audio, or code into a vector that preserves some relationships and discards others. That loss is what makes search work. It is also why a refund policy and a cancellation policy can be close while answering different questions.

Vector distance is a first pass. The retrieval system still needs metadata filters, reranking, citations, and a way to ask a clarifying question when the nearest result is not good enough.

Chunking decides what can be found. A chunk that is too small loses context. One that is too large mixes unrelated ideas. Splitting a code block, table, or policy section at the wrong point creates results that are semantically nearby and practically useless.

The record should keep source context visible:

type EmbeddedChunk = {
  id: string
  sourceId: string
  sourceType: "doc" | "ticket" | "code" | "trace"
  title?: string
  sectionPath?: string[]
  text: string
  embeddingModel: string
  embeddingVersion: string
  metadata: {
    owner?: string
    language?: string
    createdAt?: string
    access?: "public" | "team" | "private"
  }
}

Metadata is what lets the system say “nearby, but not eligible.” A stale migration note may be close to current billing documentation. Similarity does not grant permission or freshness.

Memory needs an even clearer boundary. Saving everything and embedding it creates a system that retrieves old preferences, abandoned plans, temporary instructions, and one-off exceptions. I would separate memory by lifecycle:

preference: durable and user editable
project note: scoped to one project
recent context: short-lived
blocked fact: explicitly excluded from future use

In an agent system, stale memory becomes authority. The fix is lifecycle design, not a larger vector database.

Evaluate the misses rather than the demo:

{
  "query": "how do I rotate the webhook secret?",
  "expected": ["docs/security/webhooks.md#secret-rotation"],
  "acceptable": ["runbooks/incident-response.md#webhook-leak"],
  "bad": ["docs/api/webhooks.md#create-webhook"]
}

Recall@K and nDCG are useful, but opening the wrong neighbors matters too. A relevant document at rank 12 suggests reranking. A stale document at rank 1 points to a content or freshness problem. If the right document never appears, the issue may be chunking or the embedding model.

The map metaphor is fine as long as it stays a map. Distance is not relevance, similarity is not permission, and a retrieved chunk is not an answer.

Embeddings are plumbing. Chunking, metadata, reranking, citations, deletion, and a test set built from misses are what make the plumbing useful.