20 years ago computers basically couldn't do voice or image recognition at all. 10 years ago it worked, but was still very flakey. Today it's super human.
I agree that if progress flattens out very rapidly and very soon then we'll at least have more time, but right now everything would suggest progress is still exponential and that flattening will now need to come quick and hard. This is largely why I've significantly increased my doom probability over the last year. I don't think there's any reasonable progress curve you can draw at this point that would suggest ASI isn't coming very soon. I suspect our only hope is nuclear war or a long tail good post-ASI outcome.
In regards to Sol, I think it's worth remembering that people were saying just a couple of years ago that AI can't even count the number of letters in a word or do basic maths. The fact the bar has risen so much we're saying they're still meh because they're not one-shotting problems is at least worth noting.
> I think one real risk is solo or sleeper cell type nefarious actors having a semi-competent engineering consultant which can enable small-scale weapons deployment.
Yep, the problem with AI isn't that there's any one risk that anyone can reasonably predict with high probability, but that there are many potential risks with signal digit probabilities. When you consider the overall risk landscape, that's when things get worrying.
> So AGI take over, meh, low chance IMO.
Right now I'd put this at like 10-20% probability. It seems unlikely to me, but quite possible if AIs of the future develop their own goals and are sufficiently super human.
The biggest risk is collapse I think. I don't think the world is stable in a post-ASI world. When the magic wish box will grant me whatever wish I like, I might wish something bad. In my mind this is the biggest risk by far.