A superseded document beats a correct one, and similarity is why

There is an argument going around that the retired policy is not retrieved by accident, that it wins, and that this changes which layer you fix. If it holds, "the wrong document came back" is not a ranking defect: the ranker works correctly and its correct behaviour is the problem. We think it is right, and it has been measured in peer-reviewed venues before either the comment or this article, though an earlier version of this piece said otherwise twice.

Why the dead document would win on merit

Consider a company that replaced its Parental Leave Policy with a Family Leave Standard. The old document carries the question's exact noun in its title and its exact vocabulary in the headings; the replacement calls the same thing "caregiver entitlement" and covers four leave types, diluting the question's vocabulary across a longer document. It is the correct answer, and nothing about similarity says so. That is reasoning about rankers, not a measurement of one.

A practitioner put the general form on r/AI_Agents in August: a dead policy is maximally similar to a question about that policy, and keeps its vocabulary because the replacement got rewritten while everyone learned to ask from the old one. It "does not merely get retrieved, it outranks the live version" (u/nejcar20, https://old.reddit.com/r/AI_Agents/comments/1vfa9iq/i_dont_think_rag_is_the_default_answer_for/, 4 August 2026, read 14 August). That comment sat at 2 points on a thread scoring about 92: the thread was popular, the diagnosis was not.

If he is right, ranking is anti-correlated with correctness on exactly the queries that matter most, and the parts you rewrite because they matter are the parts at risk.

This was measured before any of us wrote about it

An earlier version of this article said the mechanism was unmeasured. That was wrong. The correction we published next said it had been measured once, in a June 2026 preprint, and admitted our search had covered arXiv and not the ACL Anthology. The strongest prior work is in the ACL Anthology. That correction named the room it had not looked in, and published anyway.

The ranking claim was peer reviewed in the ACL main track in July 2025. HoH built a QA benchmark from Wikipedia snapshots that keeps each article's historical version alongside the current one. With no temporal signal the outdated document is retrieved more often at every cut-off, hit rate at 5 of 0.8837 against 0.8707, and the paper states that outdated information "actually shows slightly higher retrieval rates than relevant information" (Ouyang et al., https://aclanthology.org/2025.acl-long.301/, Table 2).

The mechanism claim is older, and also peer reviewed. NevIR ranks two documents differing only by a negation and finds that "most current information retrieval models do not consider negation, performing similarly or worse than randomly ranking" (Weller et al., EACL 2024, https://aclanthology.org/2024.eacl-long.139/). A retriever that cannot separate a document from its own negation will not separate a policy from its replacement.

Version-aware retrieval was measured in October 2025, with existing approaches at "only 58-64% accuracy on version-sensitive questions" (https://arxiv.org/abs/2510.08109). The architecture this article arrives at was refereed at SIGIR in July 2026, where each stored record carries "a temporal validity interval, and a lifecycle state" and invalid ones are filtered "prior to ranking" (NuggetIndex, https://arxiv.org/abs/2604.27306).

The June preprint stays for what it actually measured, which is not ranking. On 98 labelled pairs with one open-weights embedder, cosine separates duplicates from everything else at AUROC 0.5926, and its 15 to 40 percent stale-fact rate is the forced-answer regime across four author-built benchmarks of 20 to 30 scenarios each, all code and configuration, on a 7B model (https://arxiv.org/abs/2606.26511). Its limitations section adds the part that bites us: on a messier natural-language set, extraction fell to about 44 percent and the authors quarantined that benchmark, which is the case our worked example describes.

We read HoH and the June preprint in full, and NuggetIndex's SIGIR reference is the authors' declaration on arXiv, not our reading of the proceedings. This version searched the ACL Anthology, which the last correction had not. It did not search SIGIR, CIKM, WSDM or ECIR, which on past form is where the next omission will be.

Freshness is not validity

The obvious patch is a recency signal, and the same commenter names the error precisely. "Freshness is not validity." A 2019 returns policy nobody needed to change is still correct. It is not useless: HoH's time decay cut outdated retrieval at 5 nearly in half, at the cost of a fifth of relevant recall.

The date is not even a neutral input. Prepending artificial publication dates to TREC Deep Learning passages reversed up to 25 percent of pairwise preferences between passages of identical judged relevance (Fang et al., https://arxiv.org/abs/2509.11353, preprint). A model reading dates exhibits a bias you can trigger by editing a header, not a validity filter. Metadata filtering, the sophisticated version, needs someone to have marked the document dead, and if your organisation reliably did that you would not have this bug.

The filing rule, and why it works

What fixed it for that commenter was deleting: one document owns each answer, and the superseded one goes. He calls it "a filing rule, not a retrieval feature", and reports that it fixed what better chunking had not. It has a respectable published version: an EMNLP 2023 main-conference paper predicts how long a fact will stay true and discards the volatile ones (Zhang and Choi, https://aclanthology.org/2023.emnlp-main.879/). If you have a policy library and a wrong answer in production this quarter, go and delete.

He raised the caveat himself, asking whether the housekeeping stayed done and observing that every version of it rots about six months after the project ends, because the library still has no owner. That is a maintenance obligation with a decay curve, not a fix.

Deleting is needed at all, on his account, because a ranking has no way to express that a document is dead. That summary is ours rather than his phrasing. He reached for it because his storage layer could not express retirement, and every 2026 system that can express it went the other way: Engram invalidates, "never deleting, so every fact keeps provenance and a supersession chain"; TOKI keeps an annotation "that preserves the losing fact in an audit row"; NuggetIndex filters deprecated records instead of removing them. Five independent groups in one year, plus the June preprint, and not one deletes.

What the missing primitive is, and how old it is

An interval. Not a date: one with a start, an end, and the ability to stay open.

We did not work that out, and neither did the 2026 papers. A DELETE against a SQL:2011 system-versioned table "does not actually delete the qualifying rows"; it closes the row's system-time period, and the row stays readable in the past (Kulkarni and Michels, both editors of the standard, SIGMOD Record 41(3), 2012). Retire rather than delete, standardised fifteen years ago on a taxonomy published at SIGMOD in 1985. The ontology world arrived separately: owl:DeprecatedClass has been a W3C Recommendation since 10 February 2004, so that a deprecated term "should not be used in new documents" while old data keeps working.

So the claim is not that we found a missing primitive. It is that a forty-one-year-old primitive never reached document retrieval, for a structural reason: a SQL query has a WHERE clause to hang the validity predicate on, and a vector index has nowhere to put it, because similarity search is not a query language.

In our claim store the unit is a claim, not a document, and validity is a half-open interval running from the instant the fact was observed to the instant it stopped holding, an open end meaning still valid. The receipt we would point a sceptic at is a migration: when the end-of-validity timestamp was added it had to backfill closed intervals on rows superseded before the column existed, because otherwise the valid-time read would return both the dead and the current belief. That is this article's bug inside the storage layer.

What would falsify this

Two papers point at us. One asks how much of structured memory's benefit comes from the structure rather than from competent retrieval over raw history (https://arxiv.org/abs/2608.12888); our claim is narrower than "structure wins", since the raw corpus lacks the fact that would settle this failure. The other controls for what nobody controls for. Across 2,907 questions from issue histories and encyclopedic edits, a fine-grained revision ledger appears to beat a flat baseline by +0.182; under a render-matched control "almost all the gain comes from easier presentation", the residual is "indistinguishable from zero", and coarse invalidation then beats the fine ledger by 0.084 (https://arxiv.org/abs/2607.16019, preprint). So the failure is real, and the granularity we chose may exceed what our queries need. That challenges the size of the mechanism rather than its existence, and we would rather name it than have a reader find it.

One last detail from the June preprint, the one we would have been happiest to leave out. Its remedy is not deleting. It is a deterministic supersession rule over a fact's subject, relation and object, retiring the stale value into a bi-temporal ledger with no similarity threshold and no model call. Which is where every other group ended up, and where the database standard has been since 2011. Arriving somewhere correct and well trodden is worth saying plainly; claiming we got there first would not be. The delete-everything advice, still right for a travel policy, is dangerous where the retired version governs, and which domains those are is the next article.