Turning vague ideas into evaluation benchmarks requires a level of procedural thinking that many non-technical users don’t naturally apply. You need to define constraints, success criteria, edge cases, and failure modes — basically treating any task like a mini-spec. Once people see that framing, their results improve dramatically.

Detecting hallucinations vs reasoning (2) is also important, but in my experience it becomes easier once users adopt a habit of forcing the model to externalize its reasoning (step-by-step assumptions, uncertainty estimates, alternative paths). When the chain of thought is explicit, hallucinations become much more obvious.

Curious how you see it from your experience.