Explainer

What an embedding can tell you—and what it cannot

Why a numerical representation helps search find related text, and why a high similarity score is not a fact-check.

By Saud AlotaibiPublished 3 October 20263 min read
The takeaway

Use similarity to find candidates; use evidence and task-specific tests to decide whether those candidates are useful.

A customer searches for “package has not arrived.” Your help page is titled “Track a delayed delivery.” These phrases share an idea even though several words differ. Embeddings are one way to make that relationship useful to a computer.

An embedding represents an input as a vector of numbers. A search system can compare a question vector with document vectors to rank candidate passages. This does not turn the numbers into a truth score. It creates a way to search by learned relationships rather than exact wording alone.

Think about the task, not the coordinates

It is natural to ask what each number means. In many learned representations, individual coordinates do not have simple labels such as “delivery” or “anger.” The representation works through relationships across the vector.

For a practical search project, the more useful question is whether the representation ranks the right page highly for real queries. A visually appealing plot of vectors cannot replace that check.

Imagine three help pages: tracking a delayed delivery, changing a delivery address, and requesting a refund. A query about a late package might reasonably retrieve the first and third pages. The best ordering depends on the user's intent and what each page actually says.

A nearby result can still be wrong

Two sentences can discuss the same subject while disagreeing. “The plan includes international calls” and “The plan excludes international calls” are related in topic but opposite in meaning. A similarity system may retrieve both.

That is not necessarily a retrieval failure. It may be useful to retrieve conflicting evidence. The failure occurs if the next stage treats closeness as agreement and presents the wrong statement as an answer.

The same distinction matters for dates and identifiers. A policy about a 2024 plan can be close to a question about a 2026 plan. Exact fields such as product ID, region, and version may need filters or keyword checks in addition to a vector ranking.

Build a small relevance worksheet

Collect questions that visitors would actually ask. For each question, list the page that should answer it, pages that are related but insufficient, and pages that should not appear.

For the late-package query, your worksheet might say:

  • Useful: the tracking guide and the policy for delayed shipments.
  • Related but insufficient: instructions for changing an address.
  • Wrong context: a guide for returning a delivered item.

These labels give you something concrete to compare when changing an embedding model or retrieval setting. Keep ambiguous questions in the set too; an unclear query is part of the real product experience.

Avoid universal score cutoffs

A score that works for one collection may not transfer to another model or dataset. Start by inspecting ranking quality on your worksheet. If you need a cutoff, choose it against labeled examples from your own task and record the tradeoff between missing relevant pages and showing weak ones.

For a bilingual collection, test both languages and mixed-language queries. A model's availability does not prove that it handles your audience's wording well.

Embeddings are useful because they can surface relationships that exact word matching misses. Keep them in that role. Find candidates, preserve their context, and let evidence determine what the final answer can responsibly say.

Sources & further reading