Can you trust AI? What happens when the AI makes a mistake — and who notices first

There is one question every company should ask before handing work to an AI: it pre-sorts mail, files documents and writes texts. Who actually checks it?

The obvious answer is the one everybody gives: «I do.» It holds for exactly one week.

This Sunday edition shows how checking is built that still holds in the thirtieth week.

The fallacy

Whoever deploys an AI usually reckons like this: the machine does the work, I glance over it. That goes well for a week. Then comes the day with sixty sorted mails, twelve filed documents and five prepared replies, and «glancing over it» suddenly means reading everything again. Whoever does that has saved no time. Whoever does not is no longer checking anything.

The machine’s work grows with every task you give it. A human’s attention does not grow with it. Whoever pits the two against each other loses — not on the first day, but on the thirtieth.

The solution is not to automate less. The solution is to automate the checking as well — and to leave the human exactly the one place where they are really needed.

Five checks

In an AI-run back office every piece of machine work passes a series of checks. None of them is clever. All of them are stubborn.

The number needs a source. A number an AI writes into a text is at first a claim. It stays a marked gap until a log, a measurement or two independent sources carry it. If two sources quote the same number but measure different things, it is rounded — or it goes.

The result is checked at the destination, not at the start. A queue full of scheduled posts does not prove that one has appeared. That is why every day a machine looks at the pages where the posts have to be. And after every publication a second one opens every page of our website and follows every internal link, more than 700 link targets per run.

The voice is listened back to. Our posts are read aloud by a cloned voice. A second model listens to every sentence and compares it with the text; whatever does not match is spoken again before anyone hears it. In five weeks that was 7,708 checked sentences.

What goes out is signed and registered. Images and videos we publish carry a signature and are listed in a public register. The entry is not a step in a guideline, because guidelines get forgotten. A program blocks every new publication as long as something is missing from the register.

A human approves. No mail leaves the house without somebody having pressed «Send». No post appears without an explicit yes. This is the only check that cannot be automated, and therefore the only one for which we save a human’s attention.

What all checks have in common

All five follow the same idea. A rule written in a guideline holds as long as somebody remembers it. A rule enforced by a program also holds on the day nobody thinks of it.

That is why none of these checks is written in a manual. Four of them are programs that run whether anybody is looking or not. The fifth is a human — and because the other four take the routine off their hands, they have a clear head for it.

And they have one more thing in common: they only speak up when something is wrong. A check that reports «all fine» every day is no longer read after a week. One that stays silent for three weeks and then makes the phone ring is taken seriously.

The uncomfortable insight

The question «Can I trust the AI?» is the wrong question. Nobody trusts a new employee blindly on the first day, and nobody still checks every one of their mails after ten years. In between lies something rarely spelled out in an office: a system of spot checks, four-eyes rules and signatures.

Machines need the same — except that nobody brings it along. It has to be built, check by check, and somebody has to operate it. That is the part that appears in no product demo.

What this means for your company

When somebody offers you an AI for your office, do not ask first what it can do. Ask three things:

1. How do I notice that it has done something wrong — before my customer does?
2. Which actions can it trigger without a human having agreed?
3. Who makes sure the checks are still running in a year?

The third question is where most offers fail. Setting up a check is an afternoon. Keeping it alive is an operation.

That is exactly why we do not sell a tool but the operation: a back office run by AI agents, with the checks that belong to it. With every new task a machine takes over, a check is added — as a program, not as a good intention. The entry is a single day on which the machine may only read.

Who checks the machine at your company?