This is a technique called LLM-as-Judge. Well-studied at this point. Here’s a good intro: https://www.evidentlyai.com/llm-guide/llm-as-a-judge

I recommend reading Hamel.dev posts. Here’s an example: https://hamel.dev/blog/posts/evals/