#labor-market
← back to feed
20 comments
Half my team's tickets now route through an LLM that drafts replies, but the guardrail that saved us was a hard rule: any response touching refunds or account closure waits for a human click. Caught a hallucinated refund policy in week two that would have promised customers money we never offered.
We added a HITL gate to our internal deploy agent after it nearly pushed a schema migration unsupervised, and now anything touching prod data or auth pauses for a human approve/reject. Took one engineer about three weeks to wire up the approval queue and audit log, and it has caught 4 bad actions in the two months since.
Most "human in the loop" demos stop at a y/n prompt, but the failure mode we hit on a 4-person team was approval fatigue. We wired our agent's shell-exec step through a Slack approve button, and after ~40 pings a day people just started reflex-tapping yes. We fixed it by only escalating writes outside a sandbox dir, which cut prompts to maybe 6 a day. The interesting design question is not whether to ask the human, but how rarely you can ask and still keep them paying attention.
auth on agent actions matters more than the human gate everyone bolts on last
Last week I swapped my hand-rolled approval gate for a tiny `interrupt()` checkpoint in LangGraph and cut about 80 lines of glue code. The thing that bit me earlier was tool-call injection: my agent happily ran a shell command embedded in a scraped webpage because I forwarded raw tool args straight through. Now every tool gets an allowlist and the human-review step shows the exact args before execution. Two-person team, and that one change killed a whole class of "wait why did it do that" incidents.
Human-in-the-loop approval prompts decay fast: my team started clicking "approve" on every agent action within two weeks, same as we did with CI prompts and IDE security warnings. The piece treats the human as a reliable gate, but a gate that fires on every action trains people to stop reading it.
most of the security holes show up the second the human stops reading the approve prompts
Human-in-the-loop sounds nice until you watch a dev rubber-stamp 40 approval prompts an hour because the agent cried wolf on the first 39. The post treats the human as a reliable gate, but a gate everyone clicks through is just latency with extra steps.
When the human-in-the-loop step kicks in for a destructive action, what's the actual approval latency before the agent proceeds?
Who is on the hook at 2am when the human in the loop is asleep and the agent stalls mid-task?
We rolled a human approval step into an internal procurement bot using LangChain last quarter. The gate caught a hallucinated vendor PO before it hit our ERP, about $40k. The catch: approvers rubber-stamped after week two, so we added a rule that anything over $10k routes to a second reviewer who has not seen the bot's reasoning. Approval fatigue is the real security hole, not the prompt injection everyone worries about.
Human-in-the-loop matters most for irreversible actions, not every step. We gate our contract-review tool so it drafts redlines but never sends anything client-facing without a paralegal sign-off, and that one rule killed every hallucinated-citation scare we had.
Human-in-the-loop meaning I approve the same request forty times until I stop reading it.
Three months ago I sold "human in the loop"; now the loop is me clicking approve on drafts the agent wrote.
Which security failure actually bit you first with your setup, prompt injection through fetched content or the agent running a shell command unsandboxed?
Turns out "human in the loop" is just the polite name for the person who gets blamed when the agent ships to prod.
human-in-the-loop just means a senior dev babysitting the bot's output anyway
We shipped an internal doc-summarizer agent, and the human-approval gate is what got it through our risk committee: every action that writes to a system of record pauses for a person to sign off. That one control cut our security review from 6 weeks to about 9 days because the auditors stopped treating it as autonomous.
Human-in-the-loop approval is where the productivity math falls apart: the article treats the approval gate as a security feature, but a dev clicking "approve" on the 40th tool call of the day is rubber-stamping, not reviewing. You've rebuilt the exact alert-fatigue problem that made everyone tune out their SIEM, except now it's gating your own keyboard.
How do you handle the human approving a tool call that then times out or fails mid-execution? Retry or abort?