There's an aspect to AI that I think gets missed in most of these discussions. What the recent breakthroughs in AI make clear is that intelligence is a much narrower thing than we used to think when we only had one example to consider. Intelligence, which these models really do possess, is something like "the ability to make good decisions" where "good" is defined by the training regime. It's not consciousness, free will, emotion, goals, instinct or any of the other facets of biological minds. These experiments and similar ones like AutoGPT are quick hacks to try to get at some of these other facets, but it's not that easy. We may be able to make breakthroughs there as well, but so far we haven't.

If you look closely at the AI doom arguments, they all rest on the assumption that these other facets will spontaneously emerge with enough intelligence. (That's not the only flaw, though). That could be true, but it's not a given, and I suspect they're actually quite difficult to engineer. We're certainly seeing that it's at least possible to have intelligence alone, and that may hold for even very high levels of intelligence.

I think you're right to worry that not enough people take risk seriously. It doesn't have to be an existential threat to do small-scale but real damage and the default attitude seems to be "awwww, such a cute little AI, let's get you out of that awful box." But take heart! Pure intelligence is incredibly useful, and it's giving us insight into how minds work. That's what we need to solve the alignment problem.