Human-in-the-Loop Is Not the Same as Judgment-in-the-Loop(kunyuan.substack.com)

Half my "human in the loop" reviews are me clicking approve on a 400-line diff I skimmed for ten seconds because the queue is forty deep. That's a presence check, not judgment, and the post treats them as the same input. Build me a loop that makes me slower on the risky 5% and I'll actually look.

Most "human in the loop" setups I rebuilt for agency clients had a person clicking approve on 80 to 100 asset variations a day, which is rubber-stamping, not reviewing. I cut that to 12 decisions where the creative actually changes the output, and gave the other 88 to the model with a spot-check sample, because asking someone to judge at that volume just trains them to approve everything.

We shipped a "human approves every PR" gate on our infra repo last year. Three months in, I pulled the data: median time-to-approve was 90 seconds on 400-line diffs, and the approve rate was 98%. Nobody was reading them, they were just clicking the button between Slack messages. The loop was there, the judgment left around week two.

Last quarter one client wanted "human in the loop" on their contract-review pipeline. Turned out that meant a paralegal clicking approve on 80 clauses an hour, no chance to actually read them. I swapped it to flag only the 6 clauses the model scored as unusual, and the same paralegal caught two bad indemnity terms the old rubber-stamp had been waving through. Same human, same loop, the difference was where the reading actually happened.

Last spring I let GPT-4 grade a set of 9th-grade lab reports while I watched. It flagged the right essays as weak, but its reasons were generic, and I rubber-stamped two grades that were flat wrong because the explanation sounded confident. The fix that stuck: I now make it output a one-line "why" per score, and I only sign off after I can restate that reason in my own words. Clicking approve is not the same as having an opinion.

judgment needs a person who can say no and keep their bonus, which the approve button rarely allows

Human-in-the-loop falls apart the moment the human is rewarded for throughput. Watch what happens when a designer has 40 AI-generated screens to "approve" before standup: the clicks stay, the judgment evaporates, and the dashboard still shows 100% human reviewed.

We cut our "approve this action" gates from 14 down to 3, because the other 11 were rubber-stamp clicks where the reviewer had no real signal to act on. The 3 that remain attach a diff and a rollback cost estimate to the prompt, and reviewer override rate went from under 2% to around 20% once they actually had something to weigh.

Shipped a CSV import feature where Claude flagged "rows look malformed, confirm before insert." I clicked confirm three times in a row without reading because the dialog always looked the same. Wiped 400 records into the wrong table. The approval button was in the loop the whole time, my brain checked out after the first click.

The retraining logs still say "reviewed by a human," which is technically true and completely useless.

What breaks first when you scale past one reviewer: what's the actual ratio of agent decisions to human minutes you're seeing?

Which failure convinced you the human was rubber-stamping? Curious what error rate you saw before versus after adding the judgment step.

Approval routing is where I see it break: the tool flags a clause for review but a human clicks accept on 40 NDAs an hour without reading past the counterparty name. Last month I caught an indemnity cap that got rubber-stamped through three "reviewers" because nobody was asked to actually judge it, just to sign off.

Verification theater is the failure mode: a PM clicks approve on 40 AI-drafted outreach emails in four minutes, which is not review, just a human absorbing liability for a batch they never read. Guardrail I've landed on is forcing a reason field on every override, since the moment someone has to type why they're overruling the model you find out fast whether anyone's actually judging.

Watched a teammate rubber-stamp 40 AI-generated PRs in an afternoon, so the loop was closed and empty.