This letter is much better than the earlier one. There is a growing percentage of legitimate AI researchers who think that AGI could occur relatively soon (including me). The concern is that it could be given objectives intentionally or unintentionally that could lead to an extinction event. Certainly LLMs alone aren't anything close to AGIs, but I think that autoregressive training being simple but resulting in remarkable abilities has some spooked. What if a similarly simple recipe for AGI was discovered? How do we ensure it wouldn't cause an extinction event, especially if then they can be created with relatively low-levels of resources?
As far as a pandemic or nuclear war, though, I'd probably put it on more of the level of an major asteroid strike (e.g., K-T extinction event). Humans are doing some work on asteroid redirection, but I don't think it is a global priority.
That said, I'm suspicious of regulating AI R&D, and I currently don't think it is a viable solution, except for the regulation of specific applications.