The schism between AI accelerationists and doomers is reminiscent of the rift between people who are pro- and anti- widespread entheogen use. I see both as a way to contact nonhuman intelligences, and I predict that if anyone cares to do a survey, they'd find that concerns about doing so via DMT will correlate tightly with concerns about doing so via concerns about doing so via LLMs. The major difference being that entheogens are a fairly mature technology and LLMs are the new scary thing.
As someone who is perhaps a bit unbalanced to the accelerationist camp I perceive the argument for caution a concern about too much change too quickly, particularly to existing authority structures. It's easy to dismiss most of those arguments as fear of the existing authority to losing control of a population with different priorities. Are there arguments from that side that aren't vulnerable to that dismissal?