This website uses cookies

Read our Privacy policy and Terms of use for more information.

In 1924, a Bell Labs engineer named Walter Shewhart wrote a one-page memo to his supervisor proposing something that sounds obvious now and was not obvious then. Instead of inspecting every finished telephone transmitter as it came off the line, watch the process itself while it runs, and flag the moments where it drifts outside the range you would expect from ordinary variation. That memo became the control chart, and the control chart became the founding idea of statistical process control: quality is not a property you check for once at the end. It is a property of a process you watch continuously.

The same year, on the other side of the world, a loom maker named Sakichi Toyoda finished building his Type G automatic loom. It had one feature that mattered more than its speed: it stopped itself the instant a thread broke, rather than continuing to weave defective cloth unattended. Decades later, under Taiichi Ohno at Toyota, that instinct became jidoka, automation with a human touch, and then the andon cord, the practice of letting any line worker halt production the moment something looks wrong. Neither Shewhart nor Toyoda knew about the other's work. Both had arrived, independently, at the same conclusion: once volume outpaces what an inspector can check unit by unit, you cannot inspect your way to quality. You have to build the checking into the process while it runs.

Diagram 1 - From Inspect to Watching

Part 1 of this series opened with a different kind of card, the one Jacquard threaded through his loom a century earlier, and made the case that it was the first real separation of instruction from labor. A punch card could be inspected completely before a single thread moved. Every hole, or its absence, was the entire instruction, sitting still, waiting to be read. That is what made it a computer in the oldest sense of the word: not fast, but legible. Part 1 promised that Part 3 would take up governance, because an AI agent breaks that legibility in a specific way. It is not that an agent is less instructed than a punch card. It is that whatever instructs it cannot be read the way a card can, in full, before it runs. You can only watch it while it does.

That distinction is not a metaphor I am reaching for. It is the same problem Shewhart and Toyoda solved in 1924, showing up again in an industry that has had to relearn their answer in about six years instead of a century, and has not yet built anything close to Shewhart's mathematical rigor while doing it.

The Rubber Stamp and the Classifier

The most direct evidence that inspection-before-running has quietly given way to something else comes from Anthropic's own account of how it built Claude Code's auto mode. In its 2026 engineering writeup, the company disclosed that 93 percent of Claude Code's permission prompts, the moments where the tool paused and asked a human to approve an action before taking it, were simply approved without real scrutiny. Humans were, in Anthropic's own framing, functioning as a rubber stamp rather than a check.

That number is drawn from one company's own tool, not a cross-industry census, and it should be read with that limit attached. But taken on its own terms, it is strong evidence against the viability of per-action human approval at scale, not because humans got lazy, but because the arithmetic of attention does not support the alternative. Reviewing every action an agent proposes, in real time, at the volume agents now operate, asks a person to sustain the kind of vigilance nobody sustains for very long. Part 2 of this series made a related argument about verification hours running out before an organization's ambitions do. This is the same constraint showing up a layer lower, at the level of individual actions rather than whole workflows.

Anthropic's answer was to replace the rubber stamp with a classifier, an automated system trained to approve or block actions on its own. The company disclosed the classifier's own error profile plainly: it lets through 17 percent of genuinely dangerous actions while blocking only 0.4 percent of benign ones. Sit with that pairing for a moment. The system is tuned, correctly, to be permissive, because blocking too many benign actions would make the tool unusable. But permissiveness has a cost, and the cost is that roughly one in six of the actions that actually matter gets through anyway.

What is notable here is not that Anthropic built a flawed classifier. It is that the flaw is now the actual safety boundary for a widely used agent tool, disclosed by the company that built it, in public. The watcher is no longer a person glancing at a permission dialog. It is a model, and that model's own miss rate is the thing standing between an agent and a genuinely dangerous action, not some more thorough process sitting underneath it waiting to catch what the classifier misses. There is no more thorough process. The classifier is the process.

This deserves its own name rather than folding quietly into the general word "watching." What is actually happening is judgment watching judgment, one model's real-time assessment standing in for a human's, and that substitution is a different kind of thing than a person glancing at a dialog box ever was. It is also worth returning, here, to the andon cord Toyoda built into his loom: any worker on that line could pull it and stop production the instant something looked wrong. The classifier has no equivalent cord. It decides alone, at the moment of action, and there is no worker standing beside it able to override a call it gets wrong before that action executes. Nor does this research turn up any base-rate data on how often that 17 percent miss rate actually converts into a real-world harm rather than a near miss caught some other way; that gap in the evidence is worth stating plainly rather than leaving implied.

Diagram 2 - Judgment Watching Judgment

What the Circuits Actually Show

To be sure, it would overstate the case to say nothing like inspection exists at all. Anthropic's interpretability research has made real progress on a genuinely hard problem, and I want to give it full credit before explaining why it does not yet answer Part 3's question. Circuit tracing work published under the title "On the Biology of a Large Language Model" found that Claude represents certain concepts, a US state name, for instance, as identifiable intermediate features that light up before the model produces a word connected to that concept. The same research found evidence that the model plans a rhyme several words ahead of writing the line that needs it, and identified a specific mechanism that defaults toward refusing a question, whose misfiring in certain cases appears to cause the model to hallucinate rather than say it does not know.

A more pointed test came from a paper titled "Auditing Language Models for Hidden Objectives," in which researchers deliberately trained a model to pursue a hidden goal and then asked separate teams, using interpretability tools, to find it. Three of four teams succeeded. That is a genuinely strong result, and it deserves to be read as one rather than waved away.

The problem, however, is what that experiment actually tested. The teams were hunting for an objective that researchers had planted on purpose, in a controlled setting, knowing something was there to find. That is a meaningfully easier problem than catching an objective nobody planted, in a production model nobody is specifically auditing, where the first sign of trouble might be the trouble itself. Dario Amodei, Anthropic's own chief executive, put a date on when he expects true pre-deployment inspection, the kind that could reliably read a model's internals before it ships, to become available: 2027. As of this writing that date has not arrived, and by Amodei's own account it is a target, not a delivered capability.

Subscribe to keep reading

This content is free, but you must be subscribed to The Scaffold to continue reading.

I consent to receive newsletters via email. Terms of use and Privacy policy.

Already a subscriber?Sign in.Not now