Provenance is the only thing that makes an AI answer arguable

Provenance is the only thing that makes an AI answer arguable

On 3 August 2026, JFrog published an examination of 55 security advisories filed from a single GitHub account. Fifty-four of them were completely fabricated (https://research.jfrog.com/post/sqlite-critical-cves-or-llm-slops/, which drew 727 points and 375 comments on Hacker News). The advisories cited functions that did not exist in the affected version. One pointed at lines 3555 to 3575 of a file that is 2,706 lines long. The proof-of-concepts crashed nothing.

The interesting failure is not that a model produced them. It is what happened next: downstream triage systems consumed them as ground truth, because a CVE identifier has the shape of a citation. It looks like the answer already has provenance attached. It has an identifier, which is a different thing, and nothing in the pipeline was built to notice the difference.

The problem with an answer you cannot take apart

Here is the property that matters, and it is not accuracy.

If an assistant tells you that the buying committee at an account has changed, you have exactly two moves available: believe it, or do not. You cannot disagree with part of it. You cannot say "the source is right but you read it backwards", or "that was true in March and you are quoting March", or "that came from a form fill and I would not weight a form fill that heavily". Those are all specific objections, and none of them is expressible, because the answer arrived as a single opaque assertion.

An unarguable answer is not a confident answer. It is a smaller one. It carries less information than the same sentence with three fields attached, because with those fields the recipient can engage with it, and engagement is how a wrong answer gets caught by the one person in the building who happens to know better.

This is also why detection is the wrong tool and disclosure is the right one. On 13 July 2026, an Ask HN post requesting a provenance flag on submissions took 1,102 points and 460 comments (https://news.ycombinator.com/item?id=48886741). Read what was actually requested, verbatim: "This doesn't have to act as a regular flag, i.e., it won't de-rank the article; it could just show up as an indicator, allowing others (like myself) who don't like reading AI-generated text, to skip it." No penalty. No classifier. Just a disclosed origin, so the reader can decide.

Which is fortunate, because detection has stopped being credible in either direction. On 7 August 2026 YouTube's AI-slop detector flagged Kurzgesagt, and the r/technology thread about it ran to 24,786 upvotes and 1,363 comments. That figure comes to us from a parallel research stream rather than a first-hand read, so treat it as relayed. The direction is not in doubt: a false positive against a well-known human studio does as much damage to the idea of detection as a false negative does.

What we store, and why each field is a place to disagree

Our AI's memory is a claim store, and the schema is the argument. Six columns on every claim are NOT NULL: where the evidence sits, what produced the claim, which version of that thing produced it, how sure it was, when the fact was true, and when we came to believe it. There is no supported path that records a belief while leaving any one of those six blank.

The evidence field is a pointer, never a copy of the text. It carries the source table, the source row id, a character span, a hash of the quoted span, and a content hash of the raw source artifact. The reason for that last field is stated in the interface and is worth quoting because it is the part people skip: character offsets are a pointer, and "without this link there is nothing to point AT: the mailbox row can be edited, re-synced or deleted, and the offsets then dangle."

Now count the places a person can push back. They can dispute the source, because it is named and addressable. They can dispute the extractor, because its id and version are on the row, and version 3 of a prompt may handle negation worse than version 2 did. They can dispute the confidence, because it is a number with a recorded method rather than a tone of voice. They can dispute the time, on either axis, because both are stored. That is four independent handles on one sentence. Strip the fields and there is one handle, and it is a light switch.

Enforcement is in three layers rather than one, deliberately. A shape guard rejects a claim missing its evidence pointer before it reaches the database. The Postgres adapter refuses to fill a gap with a placeholder, and the comment explaining that refusal carries the sharpest statement of the principle we have: an earlier version substituted a placeholder for a missing method, which read as honest and was fail-open, minting rows that no calibration grouping can ever attribute and that no test could fail on. Underneath both, the NOT NULL constraints catch anything that gets past.

The part a provenance company has to admit

Showing the evidence is not the same as the evidence being read, and we have a measurement of that.

On 5 August 2026 the authors of an agent-permissions browser game published results from 409,000 approve/deny decisions at 66.3 percent mean accuracy (https://scalex.dev/blog/ai-agent-permissions-stats/). Their caveat travels with the number and we are quoting it in full: "Big caveat: this was a short game, where approximately 34% of the commands players saw were threats. In our day-to-day work these threats (fortunately, so far) appear rarely. Players were under artificial time pressure." Vendor-published, artificial base rate, do not headline the accuracy figure.

The finding that survives all of that: the single most-missed command was approved 64.7 percent of the time while the history log directly above the approval prompt displayed the payload piping output to a remote curl. The evidence was rendered, on screen, adjacent to the decision. Roughly two thirds of people approved anyway.

So the honest version of our own pitch is narrower than the marketing version. Provenance is necessary and it is not sufficient. Recording where an answer came from does not make anyone read it. What it does is make the disagreement possible at all, and make it cheap for the person who does look, and make the record survive long enough for the question to be asked later, by someone who was not in the room when the answer was generated. That is a smaller promise and it is one we can keep.

Two things we have not built

The function that closes the loop fetches the exact stored bytes a claim was extracted from and re-verifies the quote hash against the sliced span. It is not a shipped behaviour yet. We checked with a positive control, so the empty result is a real absence rather than a broken search. Which makes the precise statement this: every claim carries a verifiable quote hash, and that is true of the data today. Users cannot click through to a re-verified source span, and that is not true today. We are writing the first sentence and not the second.

Second, the extraction pipeline that would fill this store at volume is not generally available yet. The store is shipped and contract-tested against real Postgres semantics in CI. The extractor that pours facts into it is waiting on a human-labelled evaluation set. Everything above is a schema property you can verify by reading a migration file, not a claim about production traffic.

Why we are so careful about this

Because the alternative is visible in our own research. Preparing this series, we tried to source the statistics everyone in this market repeats, and five widely quoted figures turned out to have broken sourcing. One citation chain terminates at a domain that no longer resolves. A whole family of AI SDR performance numbers is vendor-published and mutually inconsistent, so the figures contradict each other while all being cited as though they were the same finding. We are not reproducing any of them here, which is a small loss of persuasive material and the only defensible choice.

Every one of those numbers is unarguable in exactly the sense described above. You cannot check them, so you can only adopt or ignore them, and most people adopt. That is what an answer without provenance does at scale, and it is the same failure as fifty-four fabricated advisories: not a lie that got past a check, but an assertion that arrived in a shape no check could grip.