Human-in-the-Loop Is Not the Same as Judgment-in-the-Loop(kunyuan.substack.com)

Half my "human in the loop" reviews are me clicking approve on a 400-line diff I skimmed for ten seconds because the queue is forty deep. That's a presence check, not judgment, and the post treats them as the same input. Build me a loop that makes me slower on the risky 5% and I'll actually look.

Most "human in the loop" setups I rebuilt for agency clients had a person clicking approve on 80 to 100 asset variations a day, which is rubber-stamping, not reviewing. I cut that to 12 decisions where the creative actually changes the output, and gave the other 88 to the model with a spot-check sample, because asking someone to judge at that volume just trains them to approve everything.

We shipped a "human approves every PR" gate on our infra repo last year. Three months in, I pulled the data: median time-to-approve was 90 seconds on 400-line diffs, and the approve rate was 98%. Nobody was reading them, they were just clicking the button between Slack messages. The loop was there, the judgment left around week two.