"It's Hard to Eval" Is a Product Smell – Hamel's Blog
When your AI product is hard to evaluate, it's usually hard for users to verify too—and that's a product design failure, not an eval problem.
Read Original Summary used for search
TLDR
• The objection "our product is hard to eval" signals poor product design—artifacts hard for you to verify are hard for users too
• Three examples show how to redesign AI products for verification: data agents that show provenance and assumptions, curriculum builders that diff against vetted templates, medical report tools that surface evidence before generating narratives
• Design for verification by asking: What does the user actually need to check? What trusted thing can they compare it against? What smaller units can they accept or reject?
• Progressive disclosure and provenance (showing where each output came from) make verification tractable without overwhelming users
• Designing for verifiability makes both the product better AND evals easier—it's the same problem
In Detail
The author argues that "it's hard to eval" is a product smell, not a technical limitation. When AI outputs are difficult to evaluate, they're usually difficult for users to verify—and in the worst case, users have to redo the work from scratch to trust the result. The solution is to design products for verification first, before building evals.
Three examples illustrate this principle. AI data agents that only output a final number force users to redo the analysis. Better design shows provenance (which vetted analysis was used), surfaces assumptions (metric definitions), breaks down aggregates (so users can sanity-check distributions), and provides an interactive notebook with the full context. PE curriculum builders that generate plans from scratch require teachers to judge entire lessons. Better design anchors to vetted plans other teachers use and shows a diff of what changed, scoping review to a few edits with explanations. Workers' comp medical report generators that produce fifty-page narratives force doctors to re-read entire charts. Better design works like a research assistant—extracting facts with citations, surfacing contradictions, letting doctors verify incrementally before assembling the final report.
The pattern across all three: show provenance, provide trusted baselines for comparison, break work into smaller verifiable units, and use progressive disclosure so details don't overwhelm. Four questions guide this: What does the user need to check? What trusted thing can they compare it against? What signals do experts use to verify? What smaller units can they accept or reject? Designing for verification makes products better for users AND makes evals tractable—it's the same problem. Pre-AI, verification happened incidentally during creation. With AI, verification is the bottleneck and demands explicit design attention.