I use it frequently with my own custom UI framework. It’s never seen my framework before but it can output new, useable code with just a few examples. If that’s not generalization I don’t know what is.
That said, even GPT4 certainly has pretty significant limitations on what it manages to reason about. But without comparing their capabilities in other aspects, arguably so do most humans. We tend to force our way past those limitations by learning incrementally by doing over and over. Current models don't get that luxury without complicated fine-tuning steps, so if anything what should surprise us is how well they do with the limitation of only context to act as short-term memory.
This is only relevant for the AI-bro fantasy of AI becoming exponentially smarter than humans.
(And btw., if that is what you consider the "AI-bro fantasy" then I'm firmly in that camp. There is no logical reason to assume that's not possible; short of identifying a non-deterministic, non-materialistic source of intelligence in human brains that violates known physics and that we can't emulate, it's just a question of when, not if, and the reason for not going full tilt on that right now is down to a variant of the Wait Calculation, not a need for intelligence to kick it off - that is, trying to kickstart that now is likely to be more expensive and not be any faster than trying to kickstart it next year, and so on, and the question is guessing at when that stops being true)
Another important thing to keep in mind is one paper(wish I could remember which one it was) that showed even larger scale llms have trouble understanding that A=B is same as B=A if they have not seen A or B before