Who cares about benchmarks?
These things still cost me time because of hallucinations.
These things still cost me time because of hallucinations.
How many times do you think I've heard that over the past three decades? And you know what? They've been right every time, except for this one little fact:
The machine cannot make you give a shit about the problem space.
It matters if it is better than what you have.
If it breaks a cup but is 10x cheaper than a human, go figer