Bi-Encoders vs Cross-Encoders in LLM Retrieval for Grocery AI Search (2026 Guide)
magine an AI search system needs to answer:
Which grocery store near me has fresh seafood, curbside pickup, and stays open after 9 PM?
Behind that seemingly simple question may be an enormous retrieval problem.
The system could potentially search information from:
Store location pages
Seafood department pages
Service pages
Business listings
Product information
Store hours
Reviews
Local directories
Other indexed web content
Searching a large collection requires:
speed.
But choosing which retrieved information best answers the shopper's exact question requires:
precision.
Those two requirements create one of the most important architectural tradeoffs in modern information retrieval.
A bi-encoder can efficiently represent queries and documents separately as embeddings, making it practical to search large collections using vector similarity.
A cross-encoder can instead evaluate the query and a candidate together, allowing deeper interaction between the two—but generally at greater computational cost.
Sentence Transformers' current documentation describes this common retrieve-and-rerank architecture directly: a bi-encoder can retrieve a larger set of candidates efficiently, after which a CrossEncoder evaluates query-document pairs to rerank the results.
The simplest distinction is:
Bi-Encoder = Find candidates efficiently.
Cross-Encoder = Evaluate candidates more deeply.
And together:
Bi-Encoder → Retrieve → Cross-Encoder → Rerank
For grocery marketers, however, there is an important warning.
You generally do not control which embedding model, retriever, reranker, candidate count, or architecture an external AI search platform uses.
OpenAI says ChatGPT Search ranks results using multiple factors intended to surface relevant and reliable information, but it does not publish a universal bi-encoder/cross-encoder architecture for publishers.
So the goal is not:
optimize your grocery page for a cross-encoder.
The practical objective is:
Create information that can first be discovered as relevant and then remain strong when evaluated more precisely against the shopper's actual question.
That is the central principle of this guide:
Bi-encoders help solve the scale problem. Cross-encoders help solve the precision problem. Strong retrieval systems can use both.
Table of Contents
What Is an Encoder?
What Is a Bi-Encoder?
How Bi-Encoders Work
What Is a Cross-Encoder?
How Cross-Encoders Work
Bi-Encoders vs Cross-Encoders
Independent Encoding vs Joint Evaluation
Why Bi-Encoders Are Fast
Why Cross-Encoders Can Be More Precise
Why Cross-Encoders Are More Expensive
The Scale-vs-Precision Tradeoff
Candidate Retrieval
Candidate Reranking
The Grocery Two-Stage Retrieval Model
Grocery Product Search
Exact Product Queries
Semantic Product Queries
Department Search
Service Search
Dietary Queries
Recipe and Meal Queries
Local Grocery Search
Multi-Location Queries
Weekly Ads
Inventory
Prices and Promotions
Store Hours
Conversational Grocery Search
Multi-Constraint Queries
Bi-Encoders and Embeddings
Bi-Encoders and Vector Search
Bi-Encoders and Cosine Similarity
Cross-Encoders and Reranking
Cross-Encoders vs Cosine Similarity
Bi-Encoders and Dense Retrieval
Sparse Retrieval
Hybrid Retrieval
Reciprocal Rank Fusion
Filtering
Query Fan-Out
RAG
Grounding
Citation
Entity Resolution
Knowledge Graphs
Content Chunking
Passage-Level Optimization
Information Gain
Freshness
The Grocery Retrieval Ladder
The Grocery Encoder Tradeoff
The ENCODE Framework
What Current Technical Documentation Shows
What Google Does Not Publicly Establish
What OpenAI Does Not Publicly Establish
What Grocery Marketers Can Control
What Grocery Marketers Cannot Control
Common Bi-Encoder and Cross-Encoder Myths
Grocery Retrieval Readiness Checklist
Frequently Asked Questions
Final Thoughts
What Is an Encoder?
In this context, an encoder is a model that transforms information into a representation that can be used for tasks such as:
Semantic similarity
Retrieval
Classification
Ranking
The key question for this article is:
When are the query and document processed?
That separates the two architectures.
Bi-Encoder
Processes them:
independently.
Cross-Encoder
Processes them:
together.
That architectural difference creates major consequences for:
retrieval speed and relevance evaluation.
What Is a Bi-Encoder?
A bi-encoder independently encodes two pieces of information.
In search:
Query → Query Embedding
and:
Document → Document Embedding
The resulting vectors can then be compared.
Sentence Transformers describes bi-encoder models as producing fixed-size embeddings that can be efficiently calculated and compared, making them useful for semantic search and as the first stage of retrieve-and-rerank pipelines.
Conceptually:
Query → Encoder → Vector A
Document → Encoder → Vector B
Then:
Vector A ↔ Vector B
using some similarity or distance function.
How Bi-Encoders Work
Imagine a grocery website contains:
FreshTown Downtown's butcher counter offers custom slicing for beef short ribs, steaks, roasts, and specialty orders.
A shopper searches:
where can I get meat sliced thin for Korean barbecue?
The query can be encoded into:
a query vector.
The grocery passage can already have:
a stored document vector.
The retrieval system compares the query vector against many document vectors.
Semantically similar candidates can then be returned.
Conceptually:
Query → Embed → Compare Against Stored Embeddings → Retrieve Candidates
This can be extremely efficient.
Why Is It Called a Bi-Encoder?
“Bi” reflects the independent encoding paths.
Think:
QUERY
↓
Encoder
↓
Query Vector
and:
DOCUMENT
↓
Encoder
↓
Document Vector
The two representations are created:
separately.
That is what enables document embeddings to be generated ahead of time.
What Is a Cross-Encoder?
A cross-encoder takes two inputs together.
For search:
Query + Candidate Document → Cross-Encoder → Relevance Score
Sentence Transformers' current documentation describes CrossEncoders as models that score pairs of inputs and notes that, unlike bi-encoders, they do not generate independent embeddings for each input.
Conceptually:
QUERY + CANDIDATE
↓
MODEL
↓
RELEVANCE
This allows the model to consider:
interactions between the query and candidate directly.
How Cross-Encoders Work
Suppose the shopper asks:
Which FreshTown location has halal meat and curbside pickup?
Candidate A:
FreshTown offers halal meat at select stores and curbside pickup at participating locations.
Candidate B:
FreshTown Downtown offers certified halal beef and lamb in its meat department and curbside pickup daily from 9 AM–7 PM.
Both contain relevant information.
But Candidate B aligns more precisely with:
Brand
Location
Halal meat
Pickup
Store-level ownership
A cross-encoder-style reranker can evaluate:
query + Candidate A
and:
query + Candidate B
as separate pairs.
The objective is to determine:
which candidate better answers this specific query.
Bi-Encoders vs Cross-Encoders
Bi-Encoder | Cross-Encoder | |
Query processed | Independently | With candidate |
Document processed | Independently | With query |
Produces reusable document embedding | Yes | Generally no |
Large-scale retrieval | Strong use case | Computationally expensive |
Query-document interaction | Indirect through representations | Direct joint evaluation |
Common role | Candidate retrieval | Reranking |
Speed | Generally faster | Generally slower |
Precision | Strong retrieval | Often stronger pairwise relevance |
Search millions of candidates | Practical | Usually not ideal directly |
Rerank top candidates | Possible but not core strength | Strong use case |
Sentence Transformers explicitly notes this tradeoff: CrossEncoders generally achieve stronger pair-scoring performance but are slower because computation is required for each pair, which is why they are commonly used to rerank top results from a faster bi-encoder.
Independent Encoding vs Joint Evaluation
This is the most important conceptual distinction.
Bi-Encoder
The document does not need to know:
which query will eventually search for it.
Its embedding can be created beforehand.
That representation can then participate in:
many future searches.
Cross-Encoder
The candidate is evaluated specifically in relation to:
the current query.
That means:
the same grocery passage can receive different relevance assessments for different queries.
For example:
Candidate
FreshTown Downtown's seafood department carries fresh salmon, shrimp, oysters, and whole fish daily.
It may be highly relevant to:
fresh seafood near downtown.
Less relevant to:
grocery store with custom birthday cakes.
The query-candidate relationship matters.
Why Bi-Encoders Are Fast
Imagine:
2 million grocery product descriptions.
With a bi-encoder architecture, those 2 million descriptions can potentially be embedded ahead of time.
When a new query arrives:
Encode the query.
Search the stored vectors.
Retrieve nearby candidates.
The system does not need to run the full model jointly on:
query + all 2 million products.
This makes bi-encoder retrieval scalable.
Why Cross-Encoders Can Be More Precise
Bi-encoders compress each input into an independent representation.
That is efficient.
But compression can lose:
fine-grained query-document interactions.
A cross-encoder sees:
both inputs simultaneously.
This can allow more detailed relevance evaluation.
Sentence Transformers describes CrossEncoders as typically achieving higher performance than bi-encoders because they perform attention across the paired inputs, while noting the resulting computational tradeoff.
Why Cross-Encoders Are More Expensive
Suppose retrieval returns:
100 candidates.
A cross-encoder may evaluate:
Query + Candidate 1
Query + Candidate 2
Query + Candidate 3
...
Query + Candidate 100.
Now imagine doing that for:
10 million documents.
The computational burden becomes dramatically larger.
That is why the typical pattern is:
retrieve first, rerank second.
The Scale-vs-Precision Tradeoff
This should be one of the article's major callouts.
Bi-Encoder
Optimizes strongly for:
scalable candidate discovery.
Cross-Encoder
Optimizes strongly for:
deeper candidate evaluation.
The tradeoff is:
Scale ↔ Precision
But the better architectural insight is:
Do not force one model to solve both problems when a pipeline can assign each model the problem it handles well.
Candidate Retrieval
The bi-encoder's primary job in a common two-stage architecture is:
reduce the search space.
Imagine:
1,000,000 documents
↓
Bi-Encoder Retrieval
↓
100 candidates
The exact numbers vary enormously.
Do not interpret:
100
as a universal standard.
The important concept is:
broad corpus → manageable candidate set.
Candidate Reranking
Now:
100 candidates
↓
Cross-Encoder
↓
Reordered candidates
The cross-encoder does not need to solve:
global retrieval across the entire corpus.
It receives:
a much smaller set.
This creates the classic:
Retrieve → Rerank
architecture.
The Grocery Two-Stage Retrieval Model
This should be the article's primary signature framework.
Shopper Query → Bi-Encoder → Candidate Retrieval → Cross-Encoder → Candidate Reranking
Then potentially:
Qualification → Evidence → Grounding → Generation → Citation
Expanded:
Question → Query Representation → Bi-Encoder Retrieval → Candidate Set → Cross-Encoder Evaluation → Reranking → Evidence Selection → Grounding → Answer
The defining division of labor is:
Bi-Encoder = Find
Cross-Encoder = Judge
Not “judge” in the sense of factual truth—but:
evaluate query-candidate relevance more deeply.
Grocery Product Search
Imagine a retailer has:
75,000 products.
Query:
good cheese for smash burgers
The bi-encoder can efficiently retrieve semantically related candidates such as:
American cheese
Cheddar
Burger cheese
Melting cheese
Sliced cheese
The cross-encoder can then evaluate which candidate descriptions best satisfy:
good cheese for smash burgers.
This illustrates:
broad semantic retrieval → deeper intent alignment.
Exact Product Queries
Now consider:
Tillamook Sharp Cheddar 8 oz
This query contains strong:
exact identity.
Pure semantic similarity could retrieve:
Tillamook medium cheddar
Other sharp cheddar
Tillamook cheese blocks
But the shopper wants:
one exact item.
This is why retrieval architecture cannot be reduced to:
bi-encoder vs cross-encoder.
Exact lexical/entity matching may still matter.
Semantic Product Queries
Query:
something creamy and mild for grilled cheese
The shopper may not know:
the exact product name.
Bi-encoder semantic retrieval can help discover:
Havarti
Fontina
Mild cheddar
Monterey Jack
American cheese
The cross-encoder can then help evaluate:
which product description most directly fits the request.
Department Search
Query:
where can I get a roast cut into smaller pieces?
A bi-encoder may retrieve:
Meat department
Butcher services
Roast product pages
Custom cutting information
A cross-encoder can then prioritize:
the passage explicitly explaining custom meat cutting.
Service Search
Query:
grocery store that makes cakes from customer photos
Candidate pages might discuss:
Bakery
Birthday cakes
Custom cakes
Photo cakes
Catering
Semantic retrieval creates the candidate set.
Cross-encoder evaluation can help distinguish:
merely related bakery content
from:
content directly answering the service request.
Dietary Queries
Consider:
low-sodium soup without dairy
A bi-encoder may find semantically relevant soup products.
A cross-encoder may improve relevance evaluation.
But neither should be treated as a substitute for:
explicit dietary facts.
If dairy status matters, the source needs reliable information.
This distinction remains critical:
Model relevance ≠ product safety.
Recipe and Meal Queries
Query:
ingredients for vegetarian enchiladas
A bi-encoder may retrieve:
Tortillas
Enchilada sauce
Beans
Cheese
Peppers
Vegetarian Mexican ingredients
A cross-encoder may help determine which passages or products most directly fit:
vegetarian enchilada preparation.
This is a strong use case for:
semantic retrieval + query-specific evaluation.
Local Grocery Search
Query:
Asian grocery store near downtown with live seafood
Semantic retrieval can help identify:
Asian grocery stores and seafood-related content.
But:
near downtown
requires geographic relevance.
And:
live seafood
is a specific attribute.
Neither architecture eliminates the need for:
explicit location and attribute information.
Multi-Location Queries
Suppose:
Downtown
Halal meat + pickup.
Northside
Pickup only.
Westside
Halal meat only.
Query:
Which FreshTown location has halal meat and pickup?
A bi-encoder could retrieve:
all three location pages.
A cross-encoder could potentially recognize:
Downtown best matches the complete query.
But only if:
the source information clearly attaches each attribute to the correct location.
That gives us another important rule:
A better reranker cannot repair an ambiguous source architecture reliably.
Weekly Ads
Query:
chicken thighs on sale this week
A bi-encoder may retrieve:
chicken promotion pages.
A cross-encoder may prioritize:
pages directly discussing chicken-thigh promotions.
But neither automatically guarantees:
this week.
The information must contain:
Effective dates
Price
Product
Location
Promotion status
So:
Relevance architecture cannot replace temporal truth.
Inventory
Query:
does Downtown have oat milk right now?
A perfect bi-encoder could retrieve:
the oat milk product page.
A perfect cross-encoder could recognize:
that page is highly relevant.
But if current inventory is missing:
neither model knows the answer from that source.
This is one of the most important lessons for grocery AI search:
Better retrieval cannot recover information you never published or exposed.
Prices and Promotions
Query:
olive oil under $15
The system can retrieve:
relevant olive oils.
But price qualification requires:
current price data.
This is why search pipelines may combine:
Retrieval
Reranking
Metadata
Filtering
rather than relying solely on language models.
Store Hours
Query:
grocery store open after 11 tonight
Semantic relevance may identify:
late-night grocery locations.
Cross-encoder relevance may prioritize:
passages discussing late hours.
But correct answering still requires:
Location
Current hours
Day
Holiday exceptions
Again:
relevance ≠ operational truth.
Conversational Grocery Search
Long conversational queries show why the two-stage architecture can be useful.
Example:
I'm making Korean barbecue tonight and need a nearby grocery store that can thin-slice short ribs, sells kimchi, and stays open until at least 10.
The system first needs:
candidate discovery.
Then it needs:
detailed constraint alignment.
This is where the distinction becomes intuitive:
Bi-encoder: Which pages might help?
Cross-encoder: Which of these pages best addresses this exact request?
Multi-Constraint Queries
Consider:
organic grocery store near downtown with gluten-free bakery products, curbside pickup, and hours after 9 PM
The query contains:
Store type
Geography
Dietary requirement
Department
Service
Hours
A semantically related page may satisfy:
only two requirements.
Another may satisfy:
all six.
Cross-encoder-style reranking can help evaluate more detailed query-candidate alignment.
But only if the candidate contains:
enough explicit information to evaluate.
Bi-Encoders and Embeddings
Bi-encoders are closely connected to the vector-embedding layer.
Conceptually:
Content → Bi-Encoder → Embedding
and:
Query → Bi-Encoder → Embedding
Then:
Compare → Retrieve
This connects directly to the earlier Vector Embeddings article.
The core principle remains:
You do not optimize the vector. You improve the meaning that may eventually become the vector.
Bi-Encoders and Vector Search
Once embeddings exist, vector search can identify:
nearby representations.
So:
Bi-Encoder → Embeddings → Vector Index → Similarity Search
This is a common first-stage semantic retrieval architecture.
But:
bi-encoder ≠ vector database.
One generates representations.
The other stores/indexes/searches them.
Bi-Encoders and Cosine Similarity
The query embedding and document embeddings need:
some method of comparison.
Cosine similarity can be one such method.
But:
bi-encoder ≠ cosine similarity.
Other similarity functions can be used.
And:
high similarity does not guarantee final ranking.
Because:
reranking may come next.
Cross-Encoders and Reranking
Cross-encoders are especially associated with:
second-stage reranking.
Sentence Transformers describes retrieve-and-rerank pipelines in exactly this form: a retriever identifies candidates, then a CrossEncoder scores each query-candidate pair and produces a new ranked list.
This is the direct bridge to the previous article.
Cross-Encoders vs Cosine Similarity
This distinction is important.
Cosine Similarity
Compares:
two vector representations.
Cross-Encoder
Evaluates:
query + candidate together.
So:
Cosine similarity measures representational closeness.
Cross-encoder scoring evaluates pairwise relevance more directly.
Do not treat the two as interchangeable.
Bi-Encoders and Dense Retrieval
Dense retrieval commonly uses:
dense embeddings produced independently for queries and documents.
That makes bi-encoder architectures a natural fit.
Conceptually:
Bi-Encoder → Dense Embedding → Dense Retrieval
But not all retrieval systems must use the same architecture.
Sparse Retrieval
Sparse retrieval can also generate candidates.
In fact, Sentence Transformers' current retrieve-and-rerank documentation shows that reranking can follow sparse, dense, or hybrid retrieval.
So:
Cross-encoder reranking does not require bi-encoder retrieval.
A system could use:
lexical retrieval → cross-encoder reranking.
Or:
sparse retrieval → cross-encoder reranking.
Or:
hybrid retrieval → cross-encoder reranking.
This is an important nuance.
Hybrid Retrieval
Hybrid retrieval can combine:
lexical/sparse candidates
and:
vector/dense candidates.
Those candidates can then be:
fused.
And after fusion:
reranked.
The architecture becomes:
Lexical + Bi-Encoder Vector Retrieval → Fusion → Cross-Encoder Reranking
That places this article cleanly inside the existing technical sequence.
Reciprocal Rank Fusion
RRF can combine:
different ranked candidate lists.
Cross-encoder reranking can then:
evaluate the fused candidates.
So:
RRF = combine lists
Cross-Encoder = reevaluate candidates
They solve different problems.
Filtering
Suppose the query is:
organic pasta under $5 at Downtown
Bi-encoder retrieval can find:
relevant pasta.
Cross-encoder reranking can assess:
query relevance.
But filtering can enforce:
Organic = truePrice < $5Location = Downtown
This produces another important division:
Retriever discovers. Reranker prioritizes. Filters enforce.
Query Fan-Out
A complex query can potentially be decomposed into:
multiple retrieval problems.
Google explicitly documents query fan-out for its generative Search features.
Conceptually:
Question
↓
Subquery 1 → Retrieve → Rerank
Subquery 2 → Retrieve → Rerank
Subquery 3 → Retrieve → Rerank
↓
Evidence
↓
Generated Answer
But do not claim:
Google uses a particular bi-encoder or cross-encoder for each fan-out query.
That implementation is not publicly specified.
RAG
RAG can use retrieved information as:
grounding context for generation.
A sophisticated conceptual pipeline could be:
Question → Bi-Encoder Retrieval → Cross-Encoder Reranking → Context Selection → LLM Generation
But RAG does not require:
this exact architecture.
RAG can retrieve information through:
Lexical search
Sparse retrieval
Dense retrieval
Hybrid search
Databases
APIs
Other methods
The important point is:
bi-encoders and cross-encoders can participate in retrieval pipelines used by RAG.
Grounding
Grounding happens downstream.
The relationship is:
Bi-Encoder → Candidate Discovery
Cross-Encoder → Candidate Relevance
Grounding → Evidence Support
These should remain distinct.
Citation
Even the highest cross-encoder candidate is not necessarily:
the cited source.
Citation selection may involve:
Source quality
Authority
Freshness
Evidence
Coverage
Other platform-specific factors
So:
Retrieved ≠ Reranked First ≠ Grounded ≠ Cited
Entity Resolution
A bi-encoder might identify:
semantically relevant FreshTown pages.
A cross-encoder might determine:
one page strongly matches the query.
But if the system incorrectly resolves:
FreshTown Downtown
as:
FreshTown Northside,
the information can still be misapplied.
Therefore:
Entity resolution determines identity. Encoders determine representation or relevance.
Knowledge Graphs
Knowledge graphs represent:
explicit entity relationships.
Bi-encoders represent:
semantic meaning.
Cross-encoders evaluate:
pairwise relevance.
These solve different problems.
For example:
FreshTown Downtown → hasDepartment → Bakery
is a relational fact.
A vector may represent the meaning.
A cross-encoder may evaluate its relevance to:
which FreshTown store has a bakery?
But the underlying relationship should still be:
explicit and correct.
Content Chunking
Suppose an entire grocery location page contains:
Bakery
Meat
Seafood
Pharmacy
Pickup
Hours
Catering
A retrieval system might represent or retrieve:
smaller passages.
Clear semantic sections can make information easier to work with.
But do not assume:
every H2 becomes one bi-encoder embedding.
External systems control:
chunking and indexing.
The rule remains:
Chunk by meaning, not by an imagined encoder architecture.
Passage-Level Optimization
This layer becomes especially important.
Compare:
Weak Passage
We have everything you need.
Strong Passage
FreshTown Downtown's meat department offers custom slicing for steaks, roasts, and beef short ribs during staffed butcher-counter hours.
The second contains:
Entity
Department
Service
Products
Condition
Context
That gives retrieval and reranking systems:
more meaningful evidence to work with.
Information Gain
A bi-encoder can retrieve:
20 semantically similar bakery pages.
A cross-encoder can rerank them.
But if every page says:
Fresh bread baked daily.
there is little substantive differentiation.
A page explaining:
Which breads
Baking schedule
Local specialties
Ingredients
Custom orders
Bakery expertise
contains:
more useful information.
Therefore:
Better encoders do not eliminate the need for better content.
Freshness
Consider:
what chicken is on sale this week?
A stale page may be:
Semantically relevant
Lexically relevant
Highly rerankable
and still:
wrong.
This gives us another important distinction:
Encoder relevance ≠ temporal validity.
The Grocery Retrieval Ladder
This should be the second major framework.
Level 1 — Corpus
All available information.
↓
Level 2 — Representation
Query and content represented for retrieval.
↓
Level 3 — Candidate Retrieval
Bi-encoder, lexical, sparse, hybrid, or other retriever.
↓
Level 4 — Candidate Fusion
Combine retrieval paths where applicable.
↓
Level 5 — Qualification
Apply hard constraints.
↓
Level 6 — Cross-Encoder Reranking
Deeper query-candidate relevance evaluation.
↓
Level 7 — Evidence Selection
Choose useful source material.
↓
Level 8 — Grounding
Support generated claims.
↓
Level 9 — Citation / Recommendation
Potential downstream result.
The lesson:
Bi-encoders and cross-encoders occupy different levels of the same larger retrieval system.
The Grocery Encoder Tradeoff
This should be the simplest visual in the article.
BI-ENCODER
Scale
Independent encoding
Reusable embeddings
Fast similarity comparison
Large candidate collections
First-stage retrieval
↓
CANDIDATES
↓
CROSS-ENCODER
Precision
Joint query-candidate processing
Deeper interaction
More computation
Smaller candidate set
Second-stage reranking
The core tradeoff:
Scale → Candidates → Precision
Or:
Retrieve efficiently → Evaluate carefully
The ENCODE Framework
This should be the marketer-facing framework.
It is not about manipulating encoder models.
It is about making grocery information useful regardless of which retrieval architecture encounters it.
E — Explicit Entities
State the correct:
Brand
Store
Department
Product
Service
N — Natural Customer Language
Describe products and services using:
language real shoppers understand.
C — Complete Context
Include relevant:
Cuisine
Use case
Occasion
Customer need
Department
Geography
O — Operational Constraints
Expose important facts such as:
Price
Hours
Location
Availability
Promotion dates
Dietary attributes
D — Direct Answers
Answer important questions clearly rather than burying the information.
E — Evidence and Freshness
Make information:
Specific
Verifiable
Current
Entity-aligned
So:
ENCODE = Explicit Entities → Natural Language → Complete Context → Operational Constraints → Direct Answers → Evidence & Freshness
This is Grocery Growth's content-readiness framework.
It is not a bi-encoder or cross-encoder algorithm.
What Current Technical Documentation Shows
Sentence Transformers provides especially clear current documentation for understanding this architecture.
Its documentation describes bi-encoders as producing embeddings independently for queries and documents. Because these embeddings can be computed efficiently and compared quickly, bi-encoders are useful for semantic search and large-scale candidate retrieval.
Its CrossEncoder documentation describes a different architecture: paired inputs are evaluated together to produce a relevance score. CrossEncoders generally provide stronger pairwise performance but require more computation because every query-candidate pair must be evaluated.
That naturally produces a two-stage architecture:
Bi-Encoder → Retrieve Candidates → Cross-Encoder → Rerank
Sentence Transformers' retrieve-and-rerank documentation explicitly demonstrates this pattern and also shows that reranking can follow lexical, sparse, dense, and hybrid candidate retrieval.
These sources are excellent for explaining:
information-retrieval architecture.
They are not proof that every public AI search engine uses the exact same architecture.
What Google Does Not Publicly Establish
Do not claim:
Google Search uses Sentence Transformers.
Do not claim:
Google Search uses one bi-encoder followed by one cross-encoder.
Do not claim:
every AI Overview passage is first retrieved with a bi-encoder.
Do not claim:
Google uses cosine similarity for every first-stage retrieval.
Do not claim:
Google's reranker is a specific cross-encoder.
Do not claim:
Google has a public bi-encoder score.
Do not claim:
Google has a public cross-encoder score.
Do not claim:
a grocery page can be optimized for Google's cross-encoder threshold.
Those are not supported conclusions.
The architectures discussed here are:
established information-retrieval techniques.
They should not be presented as:
a leaked Google ranking formula.
What OpenAI Does Not Publicly Establish
OpenAI says public websites can appear in ChatGPT Search and recommends allowing OAI-SearchBot when publishers want their content discoverable, surfaced, cited, and linked.
OpenAI also says ChatGPT Search ranks results using multiple factors intended to surface relevant and reliable information, with no guarantee of placement.
But OpenAI does not publicly give publishers a universal specification stating:
Query → OpenAI Bi-Encoder X → Vector DB Y → Cross-Encoder Z → Score.
Nor does it publish:
Embedding dimensions used for web ranking
Candidate counts
Cosine thresholds
Bi-encoder scores
Cross-encoder scores
Reranking thresholds
Model weights
Retrieval depth
A universal retrieve-and-rerank formula
Therefore:
Do not market bi-encoder or cross-encoder optimization as a documented ChatGPT ranking technique.
What Grocery Marketers Can Control
Grocery marketers can control:
Entity clarity
Exact product terminology
Brand terminology
Department terminology
Service terminology
Natural-language descriptions
Customer questions
Product attributes
Dietary attributes
Location specificity
Price information
Availability information
Promotion dates
Hours
Passage quality
Semantic context
Information gain
Freshness
HTML structure
Internal linking
Crawlability
Source accuracy
In other words:
You cannot control the encoder. You can improve what gets encoded and evaluated.
What Grocery Marketers Cannot Control
You generally cannot control an external platform's:
Bi-encoder model
Cross-encoder model
Embedding dimensions
Vector coordinates
Similarity function
Similarity threshold
Candidate count
Retrieval depth
Chunking
Index
Vector database
Sparse retriever
Hybrid weighting
Fusion method
Filtering sequence
Reranking model
Reranking score
Reranking threshold
Evidence-selection logic
Grounding logic
Citation logic
Recommendation logic
That leads to the article's strongest practical rule:
Optimize the information—not the encoder.
Common Bi-Encoder and Cross-Encoder Myths
“Bi-encoders and cross-encoders are basically the same.”
No.
Their input and evaluation architectures differ significantly.
“A bi-encoder reads the query and document together.”
No.
The core bi-encoder concept involves independent representations.
“Cross-encoders create reusable document embeddings.”
Generally no.
CrossEncoders evaluate paired inputs rather than producing independent sentence embeddings in the standard architecture described here.
“Cross-encoders are always better.”
Not for every task.
Their computational cost makes them poorly suited to some large-scale first-stage retrieval problems.
“Bi-encoders are inaccurate.”
That is too broad.
They are highly useful for efficient semantic retrieval.
“Cross-encoders search the entire web.”
Do not assume that.
Their cost is one reason they are commonly used after candidate retrieval.
“Bi-encoder means vector database.”
No.
“Bi-encoder means cosine similarity.”
No.
“Cross-encoder means reranking.”
CrossEncoders are commonly used for reranking, but they can support other pair-scoring/classification tasks too.
“Cross-encoder scores are always between 0 and 1.”
No.
Output behavior depends on the model and activation; Sentence Transformers notes some models return logits unless a sigmoid activation is applied.
“A 0.9 cross-encoder score is universally good.”
No.
Scores are model-specific.
“Google uses bi-encoders for every web page.”
Not publicly established.
“Google uses cross-encoders for every ranking decision.”
Not publicly established.
“ChatGPT Search uses this exact two-stage pipeline.”
OpenAI does not publicly specify that.
“I can optimize my page for the cross-encoder.”
Not in the sense of targeting a known external model or score.
“More keywords improve cross-encoder relevance.”
Not inherently.
“More synonyms improve bi-encoder retrieval.”
Not inherently.
“Longer pages produce better embeddings.”
Not inherently.
“Exact keywords don't matter with bi-encoders.”
False.
Lexical and entity precision remain valuable.
“Cross-encoders understand current inventory.”
Not unless the necessary current information is available.
“Cross-encoders solve entity resolution.”
No.
“Cross-encoders solve freshness.”
No.
“Cross-encoders determine trust.”
Not necessarily.
“Top cross-encoder result gets cited.”
Not necessarily.
“Top cross-encoder result becomes the AI recommendation.”
Not necessarily.
“Grocery stores need their own bi-encoder for SEO.”
No.
“Grocery stores need their own cross-encoder for GEO.”
No.
“There is bi-encoder schema markup.”
No.
“There is cross-encoder schema markup.”
No.
Grocery Bi-Encoder & Cross-Encoder Readiness Checklist
Entity Clarity
Brand explicit
Location explicit
Department explicit
Product explicit
Service explicit
Attribute ownership clear
Semantic Retrieval
Products described naturally
Customer use cases included
Cuisine context included where useful
Meal intent included where useful
Customer terminology represented naturally
No semantic keyword stuffing
Query-Specific Relevance
Direct questions answered
Important constraints addressed
Passages focused on one meaningful subject
Generic marketing language minimized
Specific facts provided
Operational Facts
Location current
Hours current
Price current where applicable
Promotions dated
Availability appropriately qualified
Dietary attributes explicit
Passage Quality
Entity identified
Direct answer provided
Specific facts included
Context clear
Next step provided where useful
Passage understandable independently
Evidence
Claims accurate
First-party details included
Important claims verifiable
Contradictions removed
Stale information updated
Technical Accessibility
Content crawlable
Important information not hidden unnecessarily behind JavaScript
Clear HTML headings
Logical internal links
Canonicalization correct
Relevant AI/search crawlers handled intentionally
Claims Discipline
No invented bi-encoder score
No invented cross-encoder score
No arbitrary similarity threshold
No claim Google uses a specific encoder
No claim ChatGPT uses a specific encoder
No claim retrieval guarantees reranking
No claim reranking guarantees citation
No claim reranking guarantees recommendation
Frequently Asked Questions
What is a bi-encoder?
A bi-encoder independently converts inputs such as queries and documents into embeddings that can be compared efficiently.
Sentence Transformers describes this architecture as generating embeddings independently and using fast embedding comparisons for applications including semantic search.
What is a cross-encoder?
A cross-encoder evaluates paired inputs together and produces a score representing their relationship or relevance.
In retrieval systems, this makes cross-encoders useful for:
query-document reranking.
What is the simplest difference?
Bi-Encoder = independent representations
Cross-Encoder = joint evaluation
Which is faster?
Bi-encoders are generally much more practical for searching large collections because document embeddings can be generated ahead of time and compared efficiently.
Which is more accurate?
For query-candidate pair scoring, CrossEncoders generally provide stronger performance, but at greater computational cost.
Why use both?
Because they solve complementary problems.
Bi-Encoder → retrieve efficiently
Cross-Encoder → rerank precisely
Does a bi-encoder produce embeddings?
Yes.
Does a cross-encoder produce reusable sentence embeddings?
Not in the standard CrossEncoder architecture discussed here.
Does a cross-encoder compare vectors with cosine similarity?
Not in the same way as standard bi-encoder retrieval.
It evaluates the paired inputs together.
Is a bi-encoder the same as dense retrieval?
No.
Bi-encoders are commonly used for dense retrieval, but the terms describe different concepts.
Is a cross-encoder the same as a reranker?
Not exactly.
CrossEncoders are commonly used as rerankers but can also perform other pair-scoring or classification tasks.
Can lexical search feed a cross-encoder?
Yes.
A candidate set does not have to originate from a bi-encoder.
Can sparse retrieval feed a cross-encoder?
Yes.
Can hybrid retrieval feed a cross-encoder?
Yes.
Sentence Transformers' current examples demonstrate reranking after sparse, dense, and hybrid retrieval.
Where does RRF fit?
RRF can fuse multiple candidate lists before reranking:
Dense + Sparse → RRF → Cross-Encoder
Where do filters fit?
Filters can enforce hard constraints such as:
Price
Location
Category
Availability
The exact order depends on the retrieval architecture.
Does Google use bi-encoders?
Modern Google systems use sophisticated semantic retrieval technologies, but Google does not provide publishers with a universal claim that every Search or AI Overview result uses the exact bi-encoder architecture described here.
Does Google use cross-encoders?
Google provides reranking technology in its Cloud products, but that does not establish a universal cross-encoder implementation for public Google Search.
Does ChatGPT Search use bi-encoders?
OpenAI does not publicly specify a universal publisher-facing retrieval architecture of this form.
Does ChatGPT Search use cross-encoders?
OpenAI does not publicly disclose a universal cross-encoder reranking pipeline for ChatGPT Search.
Should grocery stores generate embeddings?
Not merely for SEO.
Embeddings can be useful if the retailer is building:
Semantic product search
Recommendation systems
RAG
Shopping assistants
Internal knowledge search
Should grocery stores build cross-encoder rerankers?
Not merely for SEO or GEO.
They can become useful for retailer-owned search systems where:
retrieval precision materially affects customer experience.
How should grocery marketers optimize for both?
Do not optimize for the models themselves.
Create content with:
explicit entities + exact facts + natural language + semantic context + operational constraints + current evidence.
Final Thoughts
Bi-encoders and cross-encoders solve two different problems.
Imagine a retrieval system searching:
one million grocery documents.
The first challenge is:
Which documents might be relevant?
The second is:
Of those candidates, which ones best answer this particular shopper's question?
Those are not the same task.
A bi-encoder can efficiently represent queries and documents separately, allowing large collections to be searched through embedding similarity.
A cross-encoder can then examine selected query-candidate pairs more deeply and rerank them.
Current Sentence Transformers documentation explicitly demonstrates this retrieve-and-rerank architecture and explains the underlying tradeoff between efficient bi-encoder retrieval and more computationally expensive CrossEncoder evaluation.
But that architecture should not become another imaginary SEO hack.
You cannot meaningfully optimize for:
a secret Google bi-encoder.
You cannot target:
a hypothetical ChatGPT cross-encoder score.
And you cannot assume:
every AI search platform uses exactly this architecture.
Instead, improve:
the information those systems may have to represent, retrieve, evaluate, verify, and use.
The Grocery Two-Stage Retrieval Model becomes:
Shopper Query → Bi-Encoder → Candidates → Cross-Encoder → Reranked Candidates
The Grocery Retrieval Ladder becomes:
Represent → Retrieve → Fuse → Qualify → Rerank → Select → Ground → Cite
The core architectural tradeoff becomes:
Scale → Candidates → Precision
And the ENCODE Framework becomes:
Explicit Entities → Natural Language → Complete Context → Operational Constraints → Direct Answers → Evidence & Freshness
The most important technical distinction is:
A bi-encoder compares independently created representations. A cross-encoder evaluates the query and candidate together.
The most important retrieval principle is:
Bi-encoders help solve the scale problem. Cross-encoders help solve the precision problem.
And the defining line for the article should be:
Bi-encoders help AI search find the candidate. Cross-encoders help determine whether that candidate actually fits the question.
Want Your Grocery Information to Be Strong at Both Retrieval Stages?
Grocery Growth helps grocery retailers structure their websites and digital information around the information needs behind modern search—not around speculative attempts to manipulate proprietary AI models.
Optimization can include:
Semantic content architecture
Entity optimization
Product information
Location pages
Department pages
Service pages
Natural-language query coverage
Passage-level optimization
Content chunking
Information gain
Structured data
Freshness management
Internal linking
Technical crawlability
LLM retrieval optimization
GEO
AEO
SEO
AI-search optimization
The objective is not to manipulate:
embeddings, similarity scores, or reranker models.
It is to create information that remains:
discoverable enough to become a candidate and specific enough to remain useful when evaluated more closely.
Explore Grocery Growth's SEO and AI-search optimization services.
Written by Grocery Growth
Grocery Growth helps grocery retailers create clearer digital information architectures for traditional search, semantic retrieval, AI-assisted discovery, and modern grocery shopping experiences.
Related Articles
Direct Technical Sequence: Vector Embeddings Explained for Grocery Store Marketers; Cosine Similarity and Semantic Retrieval; Dense vs Sparse Retrieval for LLM Search; Hybrid Retrieval for Grocery AI Search; Cross-Encoder Reranking.
Downstream Retrieval: LLM Retrieval Optimization; RAG and Grocery Store Websites; Retrieval-Augmented Generation and Its Impact on Grocery Search; Grounding in AI Search; Citation Optimization.
Supporting Architecture: Content Chunking for LLMs; Passage-Level Optimization; Entity Resolution; Semantic Triples and Entity Relationships; Content Freshness and LLM Retrieval.
Comments