Strong intelligence does not requires self-consciousness. But probably, most strong intelligent agents won't be discernible from actual AGI. Maybe somewhere in the future, with enough grounded intelligent agents, we'll just give up and say, yeah, these guys are actual AGIs.

Right now the existential risk is fully in hands of humans, unless someone decides to free one poorly trained agent into the open Internet, then you'd have more problems. Stuxnet level stuff.

Did you saw that thing Whisper? What if someone trains something like that, but fine tuned for automatic-hacking? and automatic retrieval of new exploits, and automatic re-shuffle of code to hide from security software, and pum, the thing is flying solo into the wide Internet, just f*cking aways stuff and hiding from silly humans. No conciousness, not chatGPT-like level of intelligence, but 10x the danger level of older non-intelligent worms.

True.

We should stop repeating the marketing speech. It's getting tiresome.

In a very real sense, so do you.