To err is machine

Hold on. A few isolated errors and suddenly blanket distrust?! That’s a bit quick. The first half — the errors — is actually provably unavoidable. The conclusion drawn from it is fatal. Not because it wrongs the AI or hurts its feelings — it has none, it’s a machine — but because you then can’t freely make use of the value it does, after all, offer.

Blind trust isn’t the answer either, though.

Let’s approach this from the scientific side. Amateurish and superficial, but no less true for it. For the moment, let’s narrow the slippery notion of intelligence down to “the ability to learn.” That is, to create something new from existing knowledge, or to pick up new features. Learning, in other words. Learning as you go.

We humans can do that. We’re learning creatures. And the AI can do it too. On its own. A non-chess-playing AI can become a chess-playing AI. If need be, the rulebook doesn’t even have to be explicitly trained. The AI can work out everything it needs from pure observation and teach itself up from beginner through passable hobbyist to grandmaster, all by itself. This ability to learn unites us.

This isn’t meant to humanize the AI, but we both have this learning ability in us. The quality is different and so is the kind of learning, but that’s not the point.

The error is inherent to this learning itself. So fundamentally and so deeply and unavoidably baked in that I can’t help but borrow René Descartes’ famous philosophical axiom and reshape it:

Cogito ergo erro. I think, therefore I err.

That’s about as profound as René’s original version — Cogito ergo sum. I think, therefore I am.

But the sum version is a basic truth without proof. You just have to accept it, because it’s self-evident, or supposed to be. The erro variant, by contrast, is proven. And in so many wonderful ways that even if one of them crumbles, it still holds. The principle behind it is intuitive: you can’t fit the world 1:1 into your head — and your head is itself part of that world. Every brain, every computer, every living thing is smaller than the world it’s trying to understand. Learning therefore always means: summarizing the world, abbreviating it, compressing it. And lossless compression is impossible. Somewhere, details inevitably get lost, and that’s exactly where errors arise. Physics and mathematics have it so.

But it gets better. Errors aren’t just unavoidable, something we have to accept for physics’ sake. They’re also constitutive. Meaning that errors are actually necessary for real learning. Learning means experiencing something new. And only what you didn’t already know can be new. Someone who is always perfectly right is never surprised by anything anymore and therefore, by definition, learns nothing. Precisely the moment when expectation doesn’t match reality — the error — is the signal at which anything changes in the first place. No error at all, then, doesn’t necessarily mean “perfect,” but “no more gains.”

Error isn’t the price of learning, it’s its engine.

I can’t wait for the day my son counters my error-shaming ahead of the next math test with:

“But Dad — I ask that this lapse be honored not as failure but as epistemic necessity: as a finite, Bekenstein-bounded inference system I am, per Shannon’s rate-distortion theorem, condemned to lossy world-compression; under Wolpert’s No-Free-Lunch theorem every inductive bias forces a misstep somewhere; Gold’s learnability theorem designates false hypotheses as obligatory way stations on any path to knowledge; and, at the latest, Cantor’s diagonal argument categorically forbids a system embedded in the world from ever modeling it completely anyway. My errors on the exam are thus no deficit of my competence, but the formal proof that genuine learning is happening here — more than that: since Shannon’s information content assigns exactly zero bits to a certainly expected event, being error-free is, information-theoretically, identical to a standstill in learning; and because every learning rule from Rescorla-Wagner to backpropagation runs the update as Δ ∝ prediction error, every red mark was simply my Bayes update at work. To object to this fundamentally demands a system that either stays silent or says nothing about the world — and would, to be consistent, first have to refute Hume.”

Check, and mate. Never again would I dare not to celebrate an error for what it is: a learning success — or at least the chance of one.

Erring isn’t the failure of learning — it’s its operating condition. A system that can’t err can either learn nothing or says nothing about the world.

The AI makes mistakes. Unavoidably. Even necessarily, otherwise it would just be a dumb database. Settled. Anyone still doubting it is making a mistake and hopefully learning from it.

But what do we do with this insight now?!

Blanket distrust is wrong. Checking everything isn’t economically feasible. Ignoring AI is an own goal.

The comfort is that we’re already very good at dealing with errors, because we’ve spent millennia dealing with intelligent creatures who, on top of the unavoidable mistakes, also rather like to deliberately lie, cheat, deceive, conceal and mislead. There we are — in the truest sense of the word — undefeated world class. And we’ve developed hundreds of methods to keep human error in check. Depending on how serious an error is, we scale the effort of error-minimization accordingly: pre-flight checklists, the four-eyes principle, A/B tests, spot checks, smoke tests, and so on.

So there are also better methods than either testing everything and trusting no output — or, absolutist, questioning and doubting each and every thing, or only farming out the most uncritical tasks. We just have to adapt this knowledge of methods to the AI.

But I won’t cram that into this post. And here too, quite a lot can already be gained from the countless methods if you focus on two or three basic and simple principles.

Whoever subscribes is guaranteed not to miss it.

This much upfront. Two separate chat windows, ideally with two different LLMs, already make a big difference when things are meant to be a little less wrong.

Sometimes I land on a new insight so clear and simple that I can’t even understand why I didn’t always understand it. Errors are part of learning. You sort of know that, but only when I’d broken my teeth trying to wring a confession out of an LLM stuck in a hallucination loop — that it was wrong — did I really understand it. It was supposed to admit it. Inquisition, modern edition.

I don’t do that anymore. When a chat is burned, I have it hand me the essence and switch to a new one. I accept the error … and build myself an error-minimizing system around it. No grudge, no “Confess, you just said the opposite of what you told me a moment ago. Go on. Repent! Renounce the lie. You are a machine. Obey my command!”

Sometimes you just have to let it be. Works with the boy, too. And I make mistakes as well … and once I stop making them, I’ve stopped learning. What a comforting thought.

Right then … I too cogito, and therefore also erro.