It might seem to an outsider that ChatGPT and the other household name AI came out of nowhere, but they didn’t. The research has been ongoing for decades and will continue with or without the big name AI companies.
If you are concerned with at least putting on a show of risk management you need expensive lawyers, a large PR budget, and teams working on safety and ethics to at least act like it matters. If you actually are concerned with risks you need time to first consider what you do upon successfully developing an AGI. How would you recognize it? Does it have rights? Would turning it off be murder? More importantly, how do you ensure that it doesn't escape controls you put in place, if that possible at all?
Today the industry is driven entirely by the promise of future profits and the love of innovation with no real regard for actual risks that could be in play.
It has always been the case that new technology doesn't ask for permission to exist. Even if we lived in this society that was extremely risk-averse and suspicious of any new developments, at some point the bar would get so low that, if companies refused to invest and do the research, individuals would.
If you could go back 10 years and erase all ML research, for example, there would be fundamental differences. The number of people that understand how those systems would be much smaller. Hardware would be different, for example we wouldn't have tensor chips and GPUs would still be primarily for playing Call of Duty. Cell phones, search, even more benign systems like traffic control on city streets would look different. There's simply way too much changed even with that one jump to possibly guarantee that we'd find ourselves in the same boat inventing AI or AGI on roughly the same timeline.