Depends a lot on how this scales though. If you need 10x as many parameters to get 1% better results for instance, then even if this works it could be impractical.
You gotta be more techno-optimistic. People are just arithmetic and an existence proof that algorithms can be intelligent. People are just math so it's obviously possible to engineer general artififial intelligence with just math and computers. It's all a matter of adding up the right floating point numbers in the right (partial) order (modulo parallelism).
Very valid point. The last papers I read on the subject conclusively show that all current networks do not hit a limit in scaling though. Bill Gates said that GPT five will not be much better than GPT four so maybe things change