> how can software--particularly free software--create jobs?
Ok, let's take Linux as a specific example. Take a gander at this: http://lwn.net/Articles/547073/
So major companies (Oracle, Red Hat, Google, IBM, Intel, Samsung, Fujitsu, Texas Instruments, etc) are contributing to Linux. That creates jobs for all the kernel hackers who probably started before kernel knowledge was a hot commodity.
But that effect is minor compared to what people are building WITH Linux:
- Red Hat (a 1B company!) wouldn't even exist if it weren't for Linux. They don't "sell Linux", they sell services around Linux.
- IBM invested a billion dollars in Linux, and it paid off. They recently decided to invest another billion. Their support for Linux sells Mainframes and their expertise sells services.
- Google wouldn't be able to give away their search engine (over 10 million servers) if they had to pay Microsoft for every server.
- Companies like TI and Intel can add Linux support for their chips (a tiny expense compared to creating a new CPU), which will enable new devices to be created easily. Some of those new devices (TiVo, Android, Tomato WRT, RPi) will be successful and cause their companies to hire more people.
Giving away software doesn't compete with programmers -- it only competes with companies that sell software. Only a tiny fraction (maybe 5%?) of programmers work for companies that sell software directly. The vast majority of programmers write internal line-of-business software, or sell their software indirectly (SaaS).
So any Open Source software (Drupal, Apache, etc) is far more likely to help lots of companies save money (therefore have money left over to create jobs).
> In fact, for every programming job there may be manual jobs becoming obsolete.
That will make programming jobs be more "in demand" compared to regular jobs, therefore command higher salaries.
> So what can sustain high hourly rates and create new jobs?
You are thinking about it from the wrong end. The question is "can a bit of software be worth millions/billions to a company?" The answer is clearly "yes", since we've seen tiny teams create billions in value over and over.
Search around for the writings of patio11 on HN. He will open your eyes. People charging $200/hr are playing a different ball game than people charging $20/hr. They are not 10x better, just better at demonstrating the value of their software to the client.
> not everyone can become a specialist.
Just the opposite. There are so many new technologies and branches of science coming out that everyone will be a specialist in the future. In fact, programmers are already highly specialized. No matter what technology you pick (COBOL, Java, .NET, Linux, Microsoft, nodejs, Ruby, Python), you will find that some subset of programmers will refuse to work on it because they refuse to learn it.