This result is the same as a recent test of the same method+hypothesis from a group at Apple, no? I don’t have that reference handy but I don’t think I’m making it up.
I am impressed by the progress on LLMs but I remain skeptical that they can replace humans.
Perhaps some (distant!) future model but I don’t fear mass unemployment (for example) or even moderate LLM-driven unemployment in the near-to-medium term.
They can clearly complement human labor but there are vanishingly few domains where they can be substitutes.