Ground truth is not consensus, it has to be graded against what actually works for the original goal. Plenty of scenarios with AI and Humans can result in consensus around incorrectness.
An agent can help you create the specification, but it's up to you to know whether it's correctly testing that you got the result you wanted.