If you're the first kind of programmer, do some programming with some of the tools that interest you to learn new things and broaden your experience/understanding. If you're the 2nd kind, pick and area that interests you and roll your own. It doesn't matter if there is a big commercial code or even a big open source version out there, chances are if the right person does their homework they can build a better mouse trap. Just because something is big doesn't mean it's the best that can be, some things just have a lot of momentum.
But no, I don't really see much changing other than the list of things people build and then glue together. That basic pattern (build tools, apply tools) seems to be very persistent. Rather than worry about the future, you should open your eyes to just how wide the field is today.
Application engineering is getting easier and easier to the point where people no longer need a degree or even extensive training to be able to do application engineering work. Employers are aware of this, startups especially, and often prefer unskilled individuals (or people with only domain expertise) who are willing to work extra hard to teach themselves the skills on the job. They often pay these people less as well. This was explicitly stated as a good strategy by a ycombinator partner, and the executives of several successful, well-known, mid-size companies.
Honestly I hope this isn't true, maybe someone else can shed some light on this.
What will change is who is doing the work, how many people are employed, and how much they're compensated for their work.
I also want to mention that a number of my friends have gotten job offers from Google, and they're much lower than the offers other friends received just a year ago for similar positions (at least $20k less).
What? It's vastly easier. 10 years ago, you'd often buy servers. Getting a HA DB required a bit of knowledge (especially if you weren't using SQL Server's easy cluster stuff). Backups were another hassle.
Now? Click "New Database" and select a performance and HA level. Done. Look at StackExchange - they just scaled the DB up cause RAM is cheap enough to put a TB in a server. That's vastly easier than dealing with scaling out. And even to scale out, "cloud" DBs like SQL Azure have sharding and auto-scale built-in.
On top of that, servers are MUCH faster. Look at the RPS that some sites are claiming. They simply aren't that high. 1000 RPS? 10 years ago that required some planning. Now, so long you don't do dumb things, it's not an issue. (Source: I ran a system that did hundreds of millions of transactions a day, and the hardware requirements were rather light. Each transaction ended up in an ACID DB UPDATE, in addition to archival, auditing, HTTP requests, etc.)
It's simply incorrect to say it's not easier to scale today. It is.
There has never been a positive correlation between the ability to build/maintain applications and having a degree or going through extensive training. The job has always and will always consist in learning things by yourself, as and when you need them. If someone needs hand-holding in order to learn a new language, a new API, a new tool, or anything else that we use, he is simply not suitable for the job. You cannot count on receiving "training", because with relatively new technology, there may initially not even be any training around. In other words, there is rather a negative correlation between seeking knowledge in degrees or extensive training, and the ability to do the job. The person may simply never have learned to figure things out by himself. This would entirely disqualify him for the job. Furthermore, application engineering is only getting easier and easier for a certain group of people. For everybody else, it is getting more and more difficult.
The difference is that computer networking has become more seamless and ubiquitous, and so the process that you describe of hiring unskilled (by traditional measures) workers doing domain-specific programming work for low pay has become a more rational one to do, because of information accessibility.
So, all the "two week programing camp" developers launch their 12-hours-from-idea-to-launch apps on top of the work of _actually_ skilled people, and all the reward goes to the final layer, not the underlying support.
Right now we're in an environment where, if you do build a better mouse trap, you get $1 while people using your better mouse trap to build Social Mice Death Sphere get $1 million just for adding fancy graphics on top of other people's work.
Is the only reason our entire zero-reward open source system stays alive is because nerds are economically irrational and just want to have fun while turning a blind eye to being exploited?
I would say no, because at least to me it seems like those who are just building applications by "gluing libraries and APIs" need to be compensated more because their jobs are much less interesting both in terms of the actual work and also to future potential employers.
I would definitely take a pay cut if I were to be allowed to work on something more interesting provided I still have enough to survive.
For those "nerds", why should it bother them that other people are profiting from their work? If anything I would think it should make them happy to know that what they built is helping others all the while keeping them entertained.
I think it's disingenuous to think every open source software component should be a VC funded company operating off support contracts to get any reward. That's essentiality the only avenue to "greatness" right now.
In a proper world, VCs would recognize all their darling, bespoke, hand-picked companies depend on the public infrastructure of open source (and unaffiliated) developers to super accelerate the growth of their portfolio companies.
In a sane world, VCs would see it's in their best interest to create a "public benefit" pool of cash where they fund open source developers to create, maintain, and grow open source projects/platforms at full time senior-engineer salaries. Such a pool of cash (contributed to by multiple VC firms) would be administered by a neutral third party (to avoid nepotism as VCs are wont to do) and have "board-like" oversight and project prioritization (maybe each portfolio company gets X votes towards ranking which features to create next) so resources (cash/salary/infrastructure) go towards projects companies are using and needing to grow.
A different option is to encourage VC-funded companies to set aside, say, 5% to 15% of each round for hiring full time open source developers. Open source developers working at a company would be loyal to their open source projects first and foremost, but would accept direction and feature prioritization from the sponsoring company.
Just because something isn't a company doesn't mean it's not valuable and doesn't deserve compensation for services rendered, but in the current environment that's the only way forward until more fundamental change appears.
1. Fun is it's own reward of course.
2. Producing open source code you get easy marketing and no risk. Try to sell the same thing requires a whole new skill-set, capital, and you will probably end up with the $0 you'd get from open-sourcing it anyway.
3. Not having a boss (or customer) makes it more fun and allows you to be super-opinionated about how you make the framework.
4. I agree though in principle nerds should be more mercenary. For non-fun stuff, get your employer to pay you to write open-source code, instead of doing it on Saturday at your own expense. For fun-stuff ... well it's fun! Try to make a $ out of it somehow.
It seems that machines are much faster at learning how to do low level stuff(compilers, generic problem solvers, etc) - than they are at doing the high level stuff - understanding the problem and build a high level spec/system for the solution.
And of course we're becoming more capable of letting the domain users enter more and more details of the system themselves, either by visual programming, scripting or giving data to machine learning systems, etc.
So if those trends are to continue , professional programmers will specialize in creating more and more , highly abstract generic solutions.
Low-level optimizations are the least automated process I can think of.
The share of programmers who are doing high level stuff(understanding users and encoding that ) versus those who optimize the low-level is always growing.
Maybe it's ~90% today , vs less than 10% in the early years of computing when every byte counted and systems were less complex.
> Low-level optimizations are the least automated process I can think of.
And compilers automate low-level optimizations every day.
I try to avoid the word "System(s)" because of confusion between what we're talking about, system administrators, system architects, and control systems folks. But if you're certain the person you're talking to uses the same definition then it's fine.
The problem is that software developers are largely detail managers, we work out the finer points of everything and translate it into something someone who doesn't need to think or care about the finer points can use.
For example, you could design a building yourself - but at the end of the day you could just tell an architect what to do and he/she will manage the details for you. A good one will even stop you making obvious mistakes, and save you cash by suggesting better materials etc.
"You will never find a programming language that will free you from the burden of clarifying your ideas."
So if you need to sort an array, instead of programming quicksort, you will simply specify that you are looking for a permutation of the input array T such that for all i, T[i] < T[i+1] and the solver will search and find a solution to your problem (i.e. a sorted array) using whatever technique it likes.
Many constraint satisfaction and optimisation problems are already solved this way, there's a possibility that this generalise to other problems.
There's many advantages to this: first you need less (no?) testing, because if the solution doesn't match the problem specification it can only be a bug in the solver.
Second, there's a chance that solvers will eventually be as fast as human programmers at finding solutions. Just like the compilers are almost as good at programmers at generating assembly, and often better. If this happens programming in general will become as useless as programming in assembly today.
[1] http://en.wikipedia.org/wiki/Boolean_satisfiability_problem#...
Don't get me wrong. Constraint solvers are useful and powerful tools for declarative problem solving, but I don't recall them ever being viewed as general-purpose, especially not to the level of being a generic "autocode" (what compilers were once called, funnily enough) system.
Genetic programming seems like a more likely candidate (with some combination of SAT solvers and statistical learning), but again hypothesizing about such a major thing doesn't appear fruitful at present.
> Why hasn't Prolog programmed us out of existence, then?
I would say because of its (low) speed.
> Constraint solvers are useful and powerful tools for declarative problem solving, but I don't recall them ever being viewed as general-purpose, especially not to the level of being a generic "autocode"
They are not autocode, but they are autosolve-my-problem which is in fact what you really want. I don't believe automatically generating code makes any sense. Source code is a very inefficient way to represent a program (i.e. a function) that is only useful because it can easily be written/modified by humans. There's no point in using source code as an internal representation for a program.
[1] AFAIK state of the art SAT solvers already do some level of learning: http://en.wikipedia.org/wiki/Conflict-Driven_Clause_Learning
A lot of the other things that are going on are fun things though, and being an intern and working on a project - from scratch is a rewarding experience.
When I started this career a couple decades ago, this was not the case. Most people worked in C, with the majority at least having had exposure to, if not direct professional experience at the assembly level. The abstractions we work at today existed, but were were just written off as toys and script, not "real programming".
I think this will continue - most software developers will be re-using components, piecing them together like legos. Actually writing code to make those components happen will be less common.
There will always be people who can work at all levels of the systems... but the crowd of coders just making a living will move higher and higher in terms of abstraction.
And then I'd work as a garbage man or something, and write good old code at home. ;-)
Scripting is not more abstract than C. You can perfectly-well script in C too. Add a garbage collector. You no longer need to call the free() function. Add a few dynamically-growing structures such as a hashtable ("map") and an arraylist ("vector"). You no longer define struct data types and instead, store everything in (JSON-style) hashtables and arraylists. Define a union of all primitive types and call that "var". Now you only use the "var" type. That's it. Except for the obvious lack of syntactic sugar, that should hide the ugly details, you are now scripting in C.
But how can you start working on low level and get at least the same amount of money than in other fields / areas in IT ?
And do not forget that it is extremely tied to either big corporations, asian HW manufacturers or other big companies (like car makers) that simply requires this type of knowledge.
There was probably a zero chance for me to dive deeper into this in my country. Yes at uni they teach you some C and algorithms but it is the top of the iceberg.
Did you use search engines before Google? It was obviously a shitty experience. And ads? And don't even start on the folks that thought they could categorize the web by hand, or use META tags to sort it all out.
So while the Google guys might not have gone around polling people "What can I build that you'd like and would make me rich?", it was far from a dream of a product no one had thought about. Tons of money was going into search engines and portals by the time Google started.
And Bitcoin? The dream of e-cash has been around for a while as well; it wasn't a brand new idea. The implementation was a breakthrough, but the idea has been around for a while. (Nitpick: And at current rates, Satoshi's coins are probably worth much less than a billion dollars. And AFAIK, there's been no movement on the blocks people thinks he has.)
Like spelling and maths, programming is a technique to achieve a goal in an area. A couple of centuries there were very few 'literate' people and even fewer mathematicians, maybe a couple percent of the population. As education progressed, literacy and maths became a normality and now an average 16-year old has an arsenal of tools and equations to solve very complex problems that Pythagoras or even Newton struggled with or had to invent.
The same thing will happen to programming. We're talking about maybe 10-20 years when everyone will know how to program and the tools they will have (programming languages or environments) will be so powerful that they will be able to build their domain specific solution in a very short period of time.
There will be of course people focused on languages or low level operating system stuff, but those areas are in themselves entire branches so it will be more appropriate to call them 'compiler scientist' or 'performance engineers' rather than the generic term 'programmer'.
What I was trying to say is that there are lot of problems that are already solved and new generations can build on top of that. There are always going to be ever more complicated problems and ever-expanding areas, but in many areas we've pretty much figured things out so people can just use the solutions in their domain specific problems without having to re-implement them. I'm talking about open source libraries, higher-level programming languages and even visual programming which can be used by people to write programs without needing to understand the deeper aspects of things.
Doctors visualising data, artists making interactive art or chaining visual effects, lawyers querying databases - things you needed to hire programmers for will be done by people who aren't doing this full time, but can do it if required.
And on the other hand ,similar capability, i.e. machine learning is being used by the researchers to crack TOR mostly just by feeding data to the algorithm.
So basically ,strong tools will be available to everybody, greatly equalizing programmers and non=programmers.
On a more serious note, if program writing neural-nets develop much further I can see the field putting more of an emphasis on the theoretical aspect, and leaving implementations to the AIs -- a bit like the way mathematics is heading. Less time spent on proof-checking (and one day perhaps, construction), etc.
You can really answer your question according to the observable trends of the field. The role of the "programmer" will shift, that's all.
Regarding what a programmer needs to know, I think you'd be hard pressed to find something that wouldn't be useful in some way. Personally, I am trying to read as much of the "Old Masters" as I can. I've started with Claude Shannon and have tried to hit all the big names along the way. Bernard Widrow's work (http://www-isl.stanford.edu/people/widrow/publications.html most articles are free) got me started.
It's possible to do this already. Although working on application software at the moment which doesn't have any special assurance requirements, I've started making use of an interactive theorem prover to help sanity-check some complicated bits of logic. It's the future, from what I can see.
I see programming as a way to model reality. All programs essentially created to do something with objects of the real world or model existing world in some way. And that is where the complexity comes from. As we go higher up the stack of abstractions and tools, some things get easier, but the problems and tasks remain complex just like the world behind them. Performance gains we get come from the fact that many of the modeling and decision making has been done "at the level below" so we can shift into making higher level decisions.
Just imagine a "magic AI" which can create any program for you simply based on your description. For any non-trivial program which does not previously exist you would need to specify so many details that your list of requirements essentially becomes a program (in DSL). The more "defaults" and existing pieces you can use, the easier your job will be.
So, following that logic, the programmer of the future would be using some sort of AI for sure, to take care of the parts of the system which can be reused and recombined (taking care of the common details). And that means that anybody who can describe what they want will be able "to program". But for new problems, it would need to go into deeper down the stack and domain knowledge to be able to build new things.
In terms of "what you need to know" it means having domain knowledge so you can describe what you want. And for new domains - being able to dive deep and learn them, which means the more basic stuff you know (math, statistics, algorithms, whatever), the more efficient you will be in learning and ultimately building.
$code = '$z = 0;' . PHP_EOL
. '$y = 0;' . PHP_EOL
. '$x = 0;' . PHP_EOL
. '$x = 0;' . PHP_EOL
. '$y = 1;' . PHP_EOL
. 'for ($count = 0; $count < 10; $count++) {' . PHP_EOL
. ' $z = $x + $y;' . PHP_EOL
. ' echo $z;' . PHP_EOL
. ' $x = $y;' . PHP_EOL
. ' $y = $z;' . PHP_EOL
. '}';
$this->assertPhpAndResultEquals(
$code,
'123581321345589',
$this->asPhp(
'sätt x till 0 och sätt y till 1 och upprepa följande 10 gånger:'
. ' sätt z till x + y'
. ' och visa z'
. ' och sätt x till y'
. ' och sätt y till z'
)
);http://en.wikipedia.org/wiki/Technological_singularity
In my opinion, the next programmers generation will do quantic computing. Crypto algorithms, protocols, softwares, we will have to rethink everything to get it working with qubits. (http://en.wikipedia.org/wiki/Qubit)
If AI ends up replacing most programming tasks, then our jobs will likely shift to something more akin to AI-centric psychology mixed with high-level software engineering. Much like how only the most hardcore "Real Programmers(TM)" today write their programs in assembly or machine code in this day and age of optimizing compilers and interpreted/JIT'd languages, only the most hardcore "Real Programmers(TM)" of the future will actually write software from scratch, with most programmers instead opting to give their AIs descriptions of what they want their programs to do like how we give compilers or interpreters source code files today.
I expect that there'll be a heavy emphasis on TDD/BDD in such a future; you'd write a specification or (inclusively) a suite of integration tests, and your artificially-intelligent program-maker will create software that meets the specifications provided and satisfies the provided tests.
Whether or not these will be "programmers" in the traditional sense is a bit fuzzy; it's conceivable that - someday - non-technical pointy-haired-boss types could come up with specifications if AI-driven software-building tools can sufficiently understand natural languages. At that point, though, it's fuzzy whether or not PHBs will exist, or whether humanity will exist (or AIs, if a Dune-Butlerian-Jihad-style anti-computer revolution manages to happen).
Nice try, Skynet prototype.
Personally, I'd be shocked if this happens in the next 100 years. But it's such a "new" field, that trying to predicit 100 years into the future is a crapshoot.
100 years ago it was 1915. How much has technologically progressed since then? Do you have a sense we have leveled-off (besides single-core frequencies)? Is technology like air travel where we will reach max capability soon after creation and just have to give up improving? Of course not.
We're still in exponential growth phases of technology. Don't discount how quickly everything can change out from under us.
> can program without human input.
Much of programming is just trying all possible endpoints then seeing what works. How much of programming today is just asking Google, finding examples, then tweaking them? Most programmer jobs these days could be completely automated away given current technology if someone built it.
For example: would you be able to write a program that can recognise people faces with 80% accuracy? Probably not but AIs can do it.
If humans cannot program AIs, who else is going to do it?
AI will probably dominate the everything where the process is well known and BORING. Even if better mechanisms (security, processing, protocols) take place, there will always be issues between humans and machines.
For example here is a good example, about how computers and humans make each others situations more complex.
https://medium.com/backchannel/how-technology-led-a-hospital...
When I say better way, I don't mean touch or type, I mean better connected to our thoughts.
That's post-Singularity, and can't be meaningfully discussed.
Since the idea of the singularity has gotten conflated with one particular possible outcome ("rapture of the nerds" is merely one of the better possibilities, not the totality of the idea), let me remind you that the core idea of the singularity is that it is when the future is so different than it is now that our ability to predict it has dropped to zero. There's a certain amount of relativity to it... we're post-singularity for a cave man, for instance. AIs that are smart enough to program themselves will result in a world so radically different that there is no meaningful way to plan for what to do from here in 2015.
In the future before then, which for all we know may well encompass the entire lives of everybody here, and assuming no major civilizational hiccups (which may not be "singularities" but are also fairly difficult to predict around), programming will be one of the few fields that probably will continue to grow more or less indefinitely for the forseeable future. More stuff is going to be programmable, and the advantage for a company to have better programmer better than its competitor will likely also grow.
In fact I think there's a decent chance that demand for programmers, and especially good programmers, is likely to rise beyond the base level of people who are even theoretically capable of becoming that level of programmer. I suspect we have not actually seen peak salary in this field for good programmers.
(Interesting tidbit... programming has the tendency to "eat" other fields, as others have noticed. Even if genetics and biology manage to do everything that they've ever promised... the top level of biological designers are going to look an awful lot like programmers, moreso than biologists, precisely because the only way we'll ever have that level of control over biology is to abstract the issues away until we have a programming language on top. Nobody can work at the raw gene level, at scale. Oh, it won't be C or Haskell or Python, but it'll still be a programming language. Probably declarative.)
I'm not just trying to say good things to make us happy; this has been my assessment of the field's prognosis for a while, even since before the bubble pop, and I've seen nothing to make me change it, even the bubble pop and the 2008 crisis. Even if we are in another bubble now and it pops, it would have to epically pop, like, the entire country, person by person, decides this whole "Internet" thing is a total waste of time forever for all things, for me to change my mind.
The fact that I feel this way about this field and that I work in this field is not a coincidence. Per my comment [1], when in 1996 I mentally did what I suggested in that post and "computer programmer" popped out in my second slot for Jobs I'd Like To Do, I was pretty grateful, and, again, have seen nothing to make me change my mind since.
I read it as, some AI are already writing code. In the present day.