Multimodal embeddings made search feel wider

Text search assumes the useful evidence can become words first. That breaks when someone remembers a slide layout, chart shape, screenshot, video moment, or PDF figure.

Multimodal embeddings let those artifacts participate in retrieval. The main design choice is the retrieval unit:

• PDF section, page, or figure crop
• sampled video window or scene
• full image or object crop
• table snippet rather than the entire document

Segmentation decides whether the index can find the evidence at all.

Scores need calibration too. Text-to-text, image-to-text, and video-to-text similarities may have different distributions. A score of 0.78 is not automatically comparable across lanes.

A mixed retrieval path can stay explicit:

query -> encode
      -> retrieve text
      -> retrieve image
      -> retrieve video
      -> apply permission filters
      -> rerank mixed candidates

Reranking is where the product can distinguish the best screenshot match from the best page match instead of pretending all vectors share one scale.

Hard negatives make the index useful: a red error banner versus a red success banner, a slide with the right title but wrong chart, two similar video clips with different actions, or a PDF page where the phrase appears only in a footer.

Multimodal embeddings match how people remember work. They do not remove the usual retrieval responsibilities: choose the segment, preserve metadata, filter permissions early, calibrate scores, and test the close failures. The map gets wider only when the product keeps it readable.