Embedding Models for RAG: How to Evaluate Retrieval Quality

A retrieval system fails before generation when it selects the wrong evidence. The cause may be the embedding model, but it may also be document segmentation, stale content, access filters or a query that requires an exact identifier.

Choose embeddings through a controlled experiment on your own retrieval task rather than a table of unrelated leaderboard scores.

Establish a simple baseline

Collect representative questions and label the documents or passages that support an answer. Include exact product codes, paraphrased questions, multiple languages where relevant, and questions the collection cannot answer.

Run a lexical baseline alongside dense retrieval. An exact error code can be easier to match lexically, while a paraphrase may benefit from semantic similarity. Keep the same corpus and permission rules for each comparison.

Measure recall at a chosen result count: how much of the relevant evidence appears among the returned candidates? Also inspect rank position and irrelevant results. Do not hide a missing critical document inside a single average score.

Compare documented configurations

OpenAI documents text-embedding-3-small and text-embedding-3-large, with default dimensions of 1,536 and 3,072 and a dimensions parameter. Smaller vectors can change storage and retrieval costs; the effect on your task needs measurement.

The BGE-M3 model card describes dense, sparse and multi-vector retrieval capabilities. Supporting several modes does not mean every vector database can serve each mode without additional implementation.

Keep model identifiers, dimensions, chunking and normalization settings with the index version. Query and document embeddings must be compatible. A migration should not silently mix vectors from unrelated models.

Isolate the effect of reranking

A reranker can reorder a candidate set, but it cannot recover evidence that the first stage never retrieved. Compare retrieval alone with retrieval plus reranking using the same candidates.

For example, if the correct passage is absent from the first 30 results, changing the final ordering of those 30 cannot solve the problem. Investigate candidate generation, segmentation or query interpretation first.

Reranking is a tradeoff, not a universal requirement. Measure its quality improvement alongside added latency and cost.

Preserve the evidence boundary

Apply authorization before exposing retrieved content to a model or user. Test with two users who have different document access. A semantically relevant private document is still an invalid result for an unauthorized request.

Finally, evaluate whether generated answers cite the retrieved evidence faithfully and decline unsupported questions. Retrieval quality and answer quality are connected, but neither a larger vector nor a reranker guarantees that an answer is correct.

Sardar Mukhshaf

Full-Stack & DevOps Engineer

I build scalable web applications, secure backend systems, and production-ready cloud infrastructure. My engineering focus spans interface design, API architecture, deployment pipelines, and observability. Follow for more deep dives into system architecture, performance engineering, and emerging technologies.

ShareTwitterLinkedInCopy Link