The eval gap nobody on my team wants to own
I run a small HCI research group, four researchers plus two engineers. Six months ago we started using agentic coding tools for our analysis pipelines. Productivity on first drafts roughly tripled. Time-to-publishable-result barely moved.
The bottleneck shifted from writing code to trusting it. Our pipelines do things like cluster qualitative codes or compute inter-rater agreement on annotated transcripts. The agent produces something plausible in twenty minutes. Verifying it took one of my engineers most of a week, because we had no eval harness for the messy intermediate steps, only for the final numbers.
Nobody wants to own that eval work. It's unglamorous, doesn't generate publications, and the engineers feel it's the researchers' methodological responsibility, while researchers feel it's an engineering concern. So we keep shipping faster drafts that take just as long to defend.
My current hypothesis is that whoever in your org used to do code review is now doing eval design, whether they signed up for it or not. If that role is vacant on your team, your productivity gains are imaginary. You've just moved the debt.