17 karma · joined January 13, 2017
I was mostly pointing out the trade off between doing "direct" research on AI safety and actually speeding up the whole field significantly leaving us with less time (while trying to have better sense of the uncertainties involved), and asking for a rationale. Of course, a single PC is one of the least probable sources for AGI, but that was not my argument.
Another of the main concerns before the singularity is the possible social breakdown due to massive unemployment and inequality, the main issues being the transition, not the possibility of a post-scarcity society.
About ensuring most researchers and companies have the most incentive for a good outcome. (This ties back with guaranteed basic income, so that people can work on this unconstrained by salaries and papers citation metrics. Or stealing researchers from Google et al to AI safety, without overinflating salaries. Elon and company should be (seems they are?) dropping as much as needed on this (not just money, but PR and status as well).) Naturally, gym, universe, etc can provide more leverage to do all of this, otherwise researchers feel more compelled to join Google/Amazon/etc, just for the raw computing power and software infrastructure (the data advantage is largely overplayed for advertising purposes; what's useful is the GPU clusters for hyper parameter sweeps (of course, in RL the data reappears as an advantage if there is no open gym)). I realize some of the examples above are naive or incomplete, but they serve mostly as an example to illustrate the point.
In the blog you mention balancing managing people and technology, and I could not agree more. The AI safety problem will have the best odds if individuals are incentivized to contribute in their own short term selfish reward way. Specially among extremely intelligent and ambitious people, the danger of self denial is quite present, one can convince oneself that this is actually in everyone's best interest, when in fact one is looking for the always needed social and intellectual validation. Please do not underestimate this, and try to find ways to counter it.
Edit: This is also related to Conway's Law [0], as I think you make an allusion to (values of inventors).
I'm not saying releasing gym and universe was a bad choice. Honestly, I can't say. But can we see a rationale? One can read this piece and see it as a group of super driven and uber competent people designing a super, faster car, open sourced with a 3D printer for the parts etc, without thinking about the lack of safe roads. It's a jungle out there. And in this case, one wild fast car is enough for a disaster.
I'm sure this is talked about over and over again, but could someone please lead us through the AI safety rationale behind this? With gym and universe openAI is slashing a few months (years?) off the singularity countdown, most likely. What's the upshot? Why is the expected value of these initiatives positive? The uncertainties seem extremely large.
Edit/PS: To put it more bluntly, and be more specific: are there any projects that openAI is choosing NOT to pursue even though they would be very useful/cool for the research community (à la gym and universe), but where it has had to explicitly restrain itself, because the expected value from an AI safety perspective is negative?