Can We Stop AI From Adding Made-Up Words to Searches?
Search systems ask language models to add related words to short queries, and some are invented. This preprint cuts made-up terms by 85% without hurting search.
In one paragraph
Search systems often ask a language model to add related terms to a short query, and some added terms are hallucinated. This preprint introduces SAFE-QE, a filter that needs no training that keeps terms supported by retrieved documents and drops ones the model produces regardless of the query. Across six collections and three retrievers, it injected 4.30 unsupported terms per query against 27.83 for Query2doc and 57.47 for HyDE, with ranking quality statistically indistinguishable. It is slower per query.
How does SAFE-QE filter hallucinated search terms?
Query expansion adds related terms to a short query so it matches the words documents use. Language models now write many of these terms, but they also invent entities and off-topic words that can hurt strong search systems.
SAFE-QE samples candidate expansions from an open language model. For each term it checks support in the top retrieved documents, and it penalises terms that keep appearing when the query is randomly cut up and recombined but that no retrieved document supports, a sign of a model habit rather than the question. It then picks a small set covering as many retrieved documents as possible and drops terms that rarely reappear.
The authors also define the Hallucinated Expansion Term Rate (HETR), the share of added terms no relevant document supports, and tested six collections with three retrievers.
Did filtering hallucinated terms hurt search quality?
SAFE-QE injected 4.30 unsupported terms per query, against 27.83 for Query2doc and 57.47 for HyDE, reductions of 85% and 93%. Against the corpus-steered methods CSQE and MILL, the reductions were 45% and 43%.
Its HETR was 0.465, against 0.656 for Query2doc and 0.663 for HyDE, and it added only 10.3 terms per query.
Ranking quality (nDCG@10) was statistically indistinguishable from HyDE, CSQE and MILL and within 0.008 of Query2doc. Even with a 0.5B generator, SAFE-QE was more faithful than HyDE with a 9B one, so the benefit comes from the filter rather than from a large model. The advantage was largest with the dense retriever, where unconstrained expansions are least grounded.
Why do hallucinated search terms matter?
Invented terms add noise near the top of search results and can mislead strong retrievers. A method that removes them without lowering ranking quality offers faithfulness at no measurable cost in effectiveness.
Because SAFE-QE needs no training and works with a small generator, it could wrap around existing expansion pipelines. The new HETR measure, reported alongside the number of added terms, also gives builders a way to track how much unsupported text reaches the retriever.
A fair reading is that the method solves the problem it targets, invented terms, while leaving search quality the same, and its extra query time is the main trade-off to weigh.
What are the limits of the SAFE-QE study?
Read before you quote it
This is a preprint that has not been peer reviewed, and its limits are clear:
- Slower. About 10.97 s and 2.00 generation calls per query, against 4.06 s for Query2doc.
- Threshold tuning. With SPLADE its hallucination rate rose; the grounding threshold needs per-retriever calibration.
- No ablation yet. Which components matter has not been tested.
- Strong retrievers. On SPLADE, every expansion method ranked worse than no expansion.
The study does not show that removing unsupported terms improves rankings; the ranking result is a tie. It also does not show performance in a live search engine with real users.
- It does not show that removing unsupported terms improves search rankings; the ranking result is a tie.
- It does not show performance in a live search engine with real users.
- SAFE-QE costs more at query time: 10.97 s and 2.00 generation calls per query, against 4.06 s for Query2doc.
- Its grounding threshold is fixed and should be calibrated per retriever; with SPLADE its hallucination rate rose.
- A component ablation, testing which parts of the method matter, has not been done.
- On the strong SPLADE retriever, every expansion method, including SAFE-QE, ranked worse than no expansion.
Takeaways
- saysSAFE-QE added about four unsupported words per query, eighty-five percent fewer than Query2doc and ninety-three percent fewer than HyDE.
- saysSearch quality was statistically the same as the other methods, so the cleaner queries came at no measurable cost in rankings.
- saysIt's slower per query, and some of its settings still need tuning for each search engine.
- explainsQuery expansion adds related words to a short search query so it matches the vocabulary documents use.
- interpretsA fair reading is that invented search terms can be filtered out without losing ranking quality, but not in time.
Ask the paper
In PaperSays you can ask this paper anything, by text or by saying “Hey Aiden”. Three questions listeners asked:
What is query expansion with language models?
A language model adds related terms or a pseudo-document to a short query so search finds more relevant documents. Hear the full answer in the app →
How does SAFE-QE reduce hallucinated search terms?
It keeps terms supported by retrieved documents and drops ones the model produces regardless of the query. Ask in the app →
Does removing hallucinated terms hurt search results?
In this study, ranking quality was statistically unchanged while unsupported terms fell sharply. Ask in the app →
Briefing reviewed by an editor before publication. The paper belongs to its authors; this page summarises it and links to the original. Paper licence: CC BY 4.0. Corrections: support@papersays.com.