They then either tell someone to execute it in real world or will upload this to a robot.
Nvidia has already the Omniverse for this and others are working on World Models.
Get one to write programs in a new programming language with novel syntax and semantics that are not represented in its training set, and that might more compelling. Because that demonstrates real knowledge transfer and skills acquisition without task-specific reprogramming.
Heck, even an existing but obscure and unusual one with very little representation in public data sources would be an interesting demonstration. For example there’s CPL, an application-specific scripting language which does have a C-like syntax but lacks structs and its only dynamically sizeable data structure is a B-tree with a pretty rough-around-the-edges interface. Take an off the shelf model and demonstrate it doesn’t already know how to write it but can automatically improve at the task without undue human assistance, and you’d have a plausible demonstration of solving a (semi-) “novel problem without task-specific reprogramming.”
It’s that sort of thing that gets you the G im AGI. Not people continuing to play task whack-a-mole. The G is supposed to explicitly exclude those kinds of tricks.