We made our own marketing claims fail the build

We made our own marketing claims fail the build

This article is about what happened next, and specifically about the part that did not work first time. It is not a description of a solved problem. Two of the holes described below are still open today.

A note on what is deliberately missing. The exact strings are not reproduced here. Partly because an invented statistic quoted in an article is still an invented statistic sitting on a page, and partly for a more practical reason: the read-time guard described below would refuse to publish them. An article about a content rule that cannot pass the content rule is not a paradox worth engineering around.

The first version of the guard was wrong in an instructive way

The obvious fix is a pattern that catches a number next to a claim. Something like a digit adjacent to the word "teams". That version was written, and its own positive control failed it.

The reason is in the comment that now sits above the rule set. The shipped source was not a sentence. It was an object: one field holding the English claim, and a separate field holding the figure. The two were assembled into a sentence at render time, in a component, and were never adjacent anywhere a text search could see them. A number-adjacent pattern could not have fired on that source under any circumstances. It would have reported the site clean while the defect sat in the file it was reading.

So the rule changed shape. It matches the English claim and ignores the number entirely. There are five patterns, labelled adoption count, trust claim, usage rate, aggregate results claim, and meetings-booked multiplier, and every one of them is a phrase rather than a figure.

That decision got more important, not less, when the copy moved into a content management system. A claim and its figure become two fields on a document, and a validator that only ever sees one field at a time can never see them next to each other.

The disclosure has to be in the same sentence as the claim

The rules accept exactly four phrases as a disclosure: "not actual customers", "not real customers", "not measured results", and "illustrative". The list is short on purpose, and the comment above it explains what it rejected. "Example teams" was on the list and was taken off after a positive control: "Example teams running SAGARIS on the platform today" reads as a claim, not as a disclaimer, and accepting it would have reopened the original hole at full width.

The other constraint is placement. A disclosure only counts when it sits in the same piece of text as the claim, not somewhere nearby on the page. That is not a stylistic preference. It is the original defect stated precisely: the disclaimers existed, they were just never where the reader was.

A guard that survives a CMS migration, and one that would not have

The original guard walked the marketing tree and read component source. It worked. It also had an expiry date nobody had written down, because the moment the copy moved into a database, that guard would have been reading empty shells.

It would not have gone red. It would have gone green, over files that no longer contained the words, while the real words sat somewhere it was not looking. That is worse than having no guard, because everyone downstream reads a green check as coverage.

So the rules were separated from the thing that feeds them before the CMS work started, rather than after. The rules now take one piece of visitor-visible text and answer one question. What feeds them is somebody else's problem: today a source walker, on publish a CMS hook, and at render a check on whatever is about to reach a visitor. All three share one definition of dishonest, and there is a shared corpus that any new consumer has to pass. Reimplementing the patterns inside the CMS hook instead of importing them is the specific failure that split exists to prevent, because two copies of a rule set drift, and the copy that drifts is always the one nobody is testing.

What it still does not catch

This is the part that matters, and it is the reason this article is not a victory lap.

It does not catch a bare percentage. There is no pattern for a number standing on its own under a results-shaped label. One page carries exactly that: a counting figure under a reply-rate heading, whose "Illustrative example" wording is a content field an editor can blank. Blank it and the guard stays silent, because no pattern matches a percentage with no English claim attached.

It does not catch a comparative cost claim about other vendors. The pricing page states a monthly figure for the competing tool stack. Its qualifier, "illustrative market list prices, not live quotes", exists only inside source-code comments. Searching all nine files in the pricing directory for a visible string containing "illustrative" returns two files and both hits are comment lines; "not a quote" and "estimate" return nothing at all. The positive control on the same loop, the seat price, matches six files, so the search fires. This is the same failure shape as the original incident, on a different page and a different class of claim.

It does not catch claims about other people at all. The rules police what we say about our own customers, usage and results. They say nothing about what we say about anybody else. A separate register now covers negative claims naming a competitor, and its first version is worth reporting honestly: measured across the marketing tree, over 11,798 string literals in 304 files, it matched exactly one literal. It recognised the homepage's sentence and nothing else, while eight comparison pages made the identical assertion in a different voice. Widening the trigger took the population from 1 to 7. A broader pattern was tried, matched 12, and was rejected because it started treating "We", "The", "That" and "Third-party" as vendor names.

And five bylines on the blog are placeholder names. They came from a design file, they are not people, and a fetch of the live blog on 14 August 2026 returns all five of them, with 78 occurrences of the company name on the same page proving the fetch worked. Nothing in this system was ever looking at an author field.

The shape underneath all of it

Every item above is one defect wearing different clothes. A control states what it covers in one place, does its work in another, and nothing binds the two. The statement then drifts free of reality and nothing ever fails.

The cure is not more attention. It is to compute the covered set and the exempt set rather than declare them, and to assert that the population minus the covered set equals the exemption keys exactly, in both directions, with a written reason on every exemption. A new claim that escapes the control then changes a computed number and fails the build. An exemption without a reason is how things hide.

A build gate cannot make a claim true. It cannot know whether a competitor ships a feature, and it cannot verify a percentage. What it can do is make every claim enumerable, so that the set of things a reader might have to trust is a list somebody can read, rather than prose distributed across a site. That is a smaller promise than "our marketing is accurate", and it is the only one available that can be checked.