SILENT DECAY

Delete, reinsert, lose 14 answers: a Milvus user's silent recall drop

A Milvus user deleted and reinserted the same data. Hybrid grouped search went from 58 of 74 right answers to 44 of 74, with no error. Forced compaction brought the missing answers back. The fix shipped in 2.6.23.

2026-10-15

A Milvus user ran a fixed set of 74 test queries against their own data. Then they deleted everything and reinserted the exact same rows. The same 74 queries came back with 14 fewer right answers. No error, no warning, no missing vectors.

What the report shows

The setup, from milvus-io/milvus issue 52605 (filed Aug 18, 2026, against v2.6.22 standalone): 181 question-and-answer entities, 74 fixed queries, limit 5, hybrid dense-plus-sparse grouped search.

  • Fresh insert: 58 of 74 right answers.
  • After delete and reinsert, no manual compaction: 44 of 74.
  • After forcing compaction: 58 of 74 again.

The reporter checked the data itself. Row counts matched. The question-and-answer entries were identical. Quote: “No missing or orphan vectors were found.” The data was fine. The answers were wrong.

What the fix author says

The fix, PR 52663 (merged Aug 25, 2026, shipped in v2.6.23), matches the report to an iterator silently dropping candidates during grouped search. The reporter’s data and the fix author’s diagnosis agree on the shape of the failure. We did not run Milvus ourselves, so the match is the fix author’s reading, not ours.

One open question from the reporter, still unanswered as of Oct 15: does a flush also resolve this, or only compaction? The reporter’s Aug 26 question to the fix author and a reviewer has no reply in the issue.

Why this matters for pipelines

This is the failure mode that keeps data engineers up at night: a refresh that looks clean and quietly degrades answer quality. The row counts match. The vectors are all there. The only thing that changed is which answers come back, and nothing tells you.

The defense is boring and it works: a fixed set of test queries, run before and after every refresh, with the results compared. If 58 becomes 44, you know before your users do.

Try it on your own data

The issue lists the steps: fixed query set, grouped search on, delete, reinsert, flush, no manual compaction, same queries again, same queries without grouped search, then force compaction. The reporter’s dataset is not in the issue, so the exact numbers cannot be reproduced. But the shape of the check transfers directly: run your golden queries, refresh, run them again, compare.

Our reading of the check is in slide 6 of the carousel. It is our reading and untested.

Related: the Observability Pack ($45). Structured logging for every LLM call, with quality drift detection. Link in bio.

This post started as an Instagram post →

Discussion

Talk it through

Argue with us on Instagram.