I really dislike benchllm's use of yamls for test cases - I'd rather it be in code.

""" input: What's 1+1? Be very terse, only numeric output expected: - 2 - 2.0 """