The singularity is the tipping point where the AI can improve itself faster than humans can comprehend, so the software at least wouldn't be human-designed anymore.

It requires a level of self-driven intent and access to their own code LLMs aren't capable of on their own. They might be one component of such a future system though, we just don't know yet. Even then it would probably take quite a while to reach that tipping point. GP is just wrong, talking as if we'd already reached it.