Finding an answer without matching the exact words
Someone searches a collection of documents for “how long can I keep the equipment?” The relevant policy might say “loan duration” rather than repeat the question’s words. Semantic search can help connect those differently worded expressions.
An embedding model converts the query and passages into numerical vectors. A search system compares the vectors using a chosen similarity or distance measure. Sentence-BERT is one influential approach to producing sentence representations for such comparisons, and the Sentence Transformers documentation describes how this supports semantic retrieval.
What the numbers represent
A vector is an ordered list of numbers. Its dimensions do not generally function like a simple spreadsheet with one human-readable concept per column. The representation is learned from a training objective, and its usefulness depends on how well that objective and training data support the task.
The idea of nearby points is a helpful analogy, but it has limits. “Nearby” means similar under a particular mathematical measure in the model’s representation. It does not mean that two texts have identical meanings, agree with each other, or contain equally reliable information.
For example, passages saying that a policy permits an action and that it prohibits the action may share many concepts and vocabulary. A retrieval system still needs to distinguish the relationship expressed, not merely the topic.
Similarity is not a truth score
Imagine a document collection containing an old policy and its replacement. Both may be highly similar to the same query. Embedding similarity alone does not establish which is current. Metadata, dates, access rules, and the system’s retrieval logic matter.
Likewise, a confidently written false claim can be semantically close to a question. A similarity score does not check the claim against reality. This is a general distinction between finding related content and evaluating evidence.
A score of 0.85 also should not automatically be read as an 85% probability that the passage answers the question. The meaning depends on the model, normalization, metric, and task. Thresholds often require evaluation on representative examples rather than interpretation as universal percentages.
Why exact words can still matter
Keyword search can be particularly useful for names, codes, quoted phrases, or precise identifiers. Semantic search can help with paraphrases and conceptual overlap. A hybrid system combines signals from both approaches rather than assuming one must replace the other.
Consider a query for a specific policy number. Retrieving a broadly related policy may be less useful than matching the identifier exactly. For a question phrased informally, semantic similarity may help find terminology the user did not know. The appropriate balance depends on the collection and typical questions.
Documents are often split into passages
A long document may be divided into chunks before embedding. This makes retrieval more focused, but the boundaries can affect meaning. A passage might contain a rule while the next passage contains its exception. Retrieving only the first can create an incomplete account even if the similarity calculation worked as intended.
Larger chunks preserve more context but can mix several topics. Smaller chunks can target a question more closely but omit necessary surrounding information. There is no universally best size independent of the documents and task.
The search system may retrieve several passages, expand them with surrounding context, or rerank them using another model. Those design choices explain why two products using embeddings can behave differently.
Retrieval and generation have different failure points
In retrieval-augmented generation, a system can supply retrieved passages to a language model. If retrieval misses the relevant passage, the model may not have the evidence it needs. If retrieval succeeds, the model can still misread, omit, or overstate the passage.
A linked citation is useful only when it supports the associated claim. The fact that a system found a semantically related source does not establish that the final sentence faithfully represents it. Evaluating the retrieval stage separately from the answer stage helps identify where an error arose.
What good performance would mean
A useful test asks whether the system retrieves material that answers representative questions, handles exact identifiers, distinguishes outdated information, and preserves relevant exceptions. The evaluation needs examples from the intended collection and users, not only a general benchmark.
Embeddings are a way to make certain comparisons computationally practical. Their value comes from how they support the larger search task. Understanding that role makes semantic search less mysterious without treating a vector or similarity score as a complete understanding of the source.
