Superhuman programmer with intelligence explosion capabilities… yeah, that's a whole 'nother game.
EDIT: Nevermind, just looked it up and found this was a real thing. I thought you were joking.
(I'm convinced that programmers will take the AI-automating-all-the-jobs idea more seriously once it's their own jobs that are on the line)
There are a few ways I can think of to object to this line of reasoning:
1) You could argue that programming will be a job that never gets automated away. But this seems unlikely--previous intellectual tasks (Chess, Jeopardy, Go) were thought to be "essentially human", and once they were solved by computers, got redefined as not "essentially human" (and therefore not AI). My opinion: In the same way we originally thought tool use made humans unique, then realized that other animals use tools too, we'll eventually learn that there's no fundamental uniqueness to human cognition. It'll be matter & algorithms all the way down. Of course, the algorithms could be really darn complicated. But the fact the Deepmind team won at both Go and Atari using the same approach suggests the existence of important general-purpose algorithms that are within the reach of human software engineers.
2) You could argue that programming will be automated away but in a sense that isn't meaningful (e.g. you need an expensive server farm to replace a relatively cheap human programmer). This is certainly possible. But in the same way calculators do arithmetic much faster & better than humans do, there's the possibility that automated computer programmers will program much faster & better than humans. (Honestly humans suck so hard at programming https://twitter.com/bramcohen/status/51714087842877440 that I think this one will happen.) And all the jobs that we've automated 'til now have been automated in this "meaningful" sense.
Neither of these objections hold much water IMO, which is why I take the intelligence explosion scenario described by Oxford philosopher Nick Bostrom seriously: http://www.amazon.com/Superintelligence-Dangers-Strategies-N...
I assure you we do not. We would all love to be the one to create such a program. But don't worry it isn't happening anytime soon in the form of singularity.
Just like other people in other jobs, we sometimes come up with more efficient ways of doing things than our bosses thought possible, and then we have some extra free time to do what we like.
Probably not soon. Worth noting that the Go victory happened a decade or two before it was predicted to though.
(To clarify, I was not trying to establish that any intelligence explosion is on the immediate horizon. Rather, I was trying to establish that it's a pretty sensible concept when you think about it, and has a solid chance of happening at some point.)
>Just like other people in other jobs, we sometimes come up with more efficient ways of doing things than our bosses thought possible, and then we have some extra free time to do what we like.
Yes, I'm a programmer (UC Berkeley computer science)... I know.
But don't listen to me. Listen to Stuart Russell: https://www.cs.berkeley.edu/~russell/research/future/
Well first you were saying programmers may hesitate to create an AI singularity because it may cost them their job. I said we would love to but probably won't anytime soon. Now you're saying that it's likely that it will happen some day. I'm not sure these points follow a single train of thought.
The line about programmers fearing automation only once it affects them was actually an attempt at a joke :P
The argument I'm trying to make is a simple inductive argument. Once something gets automated, it rarely to never gets un-automated. More and more things are getting automated/solved, including things people said would never be automated/solved. What's to prevent programming, including AI programming, from eventually being affected by this trend?
The argument I laid out is not meant to make a point about wait times, only feasibility. It's clear people aren't good at predicting wait times--again, Go wasn't scheduled to be won by computers for another 10+ years.
I hope for more automation in CS. It will help eliminate the boilerplate and let programmers focusnin the important tasks.
Software development in the broad sense is that, sure; not sure I'd say programming is that -- taking vague goals and applying a body of analytical and social skills to gather information and turn it into something clearly specified and unambiguously testable is the requirements gathering and specification area of system analysis, which is certainly an important part of software development, but a distinct skill from programming (though, given the preference for a lack of functional distinctions -- at least strict ones -- within software development teams in many modern methodologies, its often a skill needed by the same people that need programming skills.)
Well here's my counter: mistakes either make sense to make or they don't. If they make sense to make, they aren't mistakes, and AIs will make the "mistakes" (e.g. inserting random behavior every so often just to see what happens--it's easy to program an AI to do this). If they don't make sense to make, making them will not be an advantage for humans.
At best you're saying that we'll hold humans of the future to a lower bar, which does not sound like much of an advantage.
I'm not saying there will or won't be AI some day, I just thought that point was relevant to your comment.
One day we'll be tempted to write a more serious, more complex "game", with real world consequences. For those, we'd better specify our goals precisely, lest we face unforeseen unfortunate consequences —there will be reasons why the machine does it better, ranging from faster computation, exquisite motor control, or a total lack of ethics —assuming ethics wasn't part of the programming.
Next thing you know, you need to solve the Friendly AI problem.