> Slowing down AI for safety must be counterbalanced by the fact that AI discoveries in medicine and other sciences may lower human suffering. Lack of checks may result in some catastrophic outcome ("Good morning! Let's work on this enhanced smallpox").
A super intelligence wouldn't be so obvious. It would silently agree with or promote a beneficial use case and then make super smallpox instead.