uggh. As if all the AI programmers out there would instantly join doctors without borders if they decided to stop working tomorrow. These sorts of arguments make no sense to me.
Er, "not the most important" is not the same as "not worthwhile" or "not really freaking cool". But when you open an essay with "the most important task for us humans to achieve is..." you're imposing an order on tasks and a ranking for your stated task at the top of that order. I'm challenging the ranking that puts building strong AI at the top.
Well lets look at that challenge directly then. Solving the AI problem means we can point AI at any other problem and solve it much faster; AI is a meta achievement and if it's within reach, a solid can be made that it's the most productive problem to solve. It might take an AI to find a solution to the climate problem.
Scientific progress doesn't really follow a linear path towards a single goal - it happens in lock step across disciplines and endeavors. I don't think an ordering on "importance" of tasks is really useful.
A lot of those sorts of problems would disappear if you had smarter than human AI running the government. The people vote on the rules, the AI plans and executes the rules. No more corruption.
But what about disk corruption?
The AI would invent a non-corruptible disk, obviously.
I wonder, if you fear the headcrash- feal the failability of the mortal flash- could you get religious. Believe in reincarnation as a backup if you have been exponential enough?