Is There a U.S. IT Worker Shortage?
spectrum.ieee.org
spectrum.ieee.org
Network operations salaries, from what I have seen, are actually lower now than they were when I was in college. Network architects and engineers still command pretty good salaries, probably because there are so many legacy networks out there that need to be replaced.
In general, salaries in product oriented positions are going up, and salaries in service oriented positions are going down.
IT has one of the highest unemployment percentages of any industry (14.9%, last I read).
While I don't believe there is an actual, honest to goodness shortage, I do believe that there are three critical problems in the industry right now:
1. Hiring - Companies can't hire the right people, good people can't get past HR, companies are afraid that they're going to hire the wrong person, and in general there are no professional organizations that are making this any easier. This has resulted in ridiculous salaries (in both directions) because companies don't know what they're paying for when they first hire someone, and are desperate not to lose the people who they have hired that know what they're doing. In addition, positions stay open for an inordinate amount of time because of the difficulty of hiring, and overall employee mobility is reduced because the interview process is slowly moving from a multi-day process to a multi-week process. I got my first programming job on a phone screen and a lunch break interview. That would be ludicrous today.
2. Training - Companies don't want to hire people who have 85% of the skillset they're looking for, because they don't believe that training for the extra 15% is going to pay off. I think this is partially because the value proposition of a new graduate with six months of solid experience is probably double of the same graduate before their experience. I think this is something not strictly limited to IT at the moment, but I'm seeing it more in IT than in any other engineering discipline.
3. Placement - In short, people aren't getting positions that are most suited to their abilities. Unskilled, first-time "CIO"s are running startups into the ground, and former "CIO"s with plenty of experience are consulting because consulting, right now, actually seems more low-risk than being an employee. I see more and more of my friends moving over to consulting not because of the money, but because they were switching employers so much before they started consulting that it doesn't make sense not to go that route.
Keep in mind that I am providing minimal evidence - although I know that the 14.9% employment rate is recently accurate, within a few weeks, but in general I'm seeing these trends everywhere and it's worrying as someone whose job it is to make systems function well simply because there is so much waste.
That contributes to making it harder for supply of people and the demand of positions to be matched when not collocated.
1. Workers are not a commodity and not interchangeable. You can't a take a network-layer programmer who got laid-off from Cisco and ask that person to work on big data and reasonably expect success.
2. I suspect a lot of tech workers are not motivated by money as long as it's above a threshold. I know for a fact that I've taken jobs that paid me as much as 25% less because I got to work on "cooler" stuff. This also fits in with the stories of Google's early days when they actually paid less in dollar terms than MS but were able to lure workers away from MS because they had better "perks" and more interesting technology.
2a. If progammers were optimizing for money, most of us would be working for wall Street, and we'd all be contracting in our free time instead of building open source apps. Clearly this isn't happening.
3. Anecdotally, I know for a fact there is huge demand for competent programmers/engineers because I have standing offers from multiple employers. The reason for this is that a good programmer is orders of magnitude more productive than an average programmer so good managers will move heaven and earth to get their hire. On top of this, an average programmer in a good team will have effective negative productivity. So restricting your supply pool even a little bit can leave you with a drastically less productive team.
I have a qualm with that statement, I think. Do you have anything to back that up? It doesn't make logical sense to say that "If I have four programmers and they give me nine megawarbles of productivity, and I add a fifth programmer who can only create one megawarble of productivity, I will have a team that only generates eight megawarbles of productivity."
Too often, programmers are drawn into this romantic idea of being a 10x programmer, but I can honestly say that, in my experience, there are more 10x programmers out there than there are 10x programming problems. The IT world needs more data janitors than it needs data scientists.
0. There is a cost on the communications between team members, if the team is larger the cost is larger 1. There are costs on integrating (in some cases re-working) sub-par code that does not fit with the rest of the team
If the programer is just average, the impact is not that terrible but it exists, but if the programer is actually bad, I can tell you from personal experience that the costs will be very significant.
A programmer who is worse than everyone else will tend to produce bugs that they themselves can't fix and also make poor design decisions. Both of these things will make things harder for the rest of the team.
In my experience there's a lot more bad programmers who produce lots of buggy code than produce low quantities of good quality code.
I used to work with one guy who was an absolute wizard at performance tuning. Helpless as a newborn lamb tho' when it came to setting up a new test environment, he was a 10x guy at what he did, but a 0.1x guy at "restoring last night's backup from tape" despite having all the privs he needed to do it...
I think the big difference between 1999 and now is back then anyone who could spell HTTP could get a great offer, because there was a lot of dumb money chasing people. The industry has learned it's lesson. Now it's very hard for a Marketing person with a personal website to pass themselves off as a web guru.
It's also much harder for people with "bad signs" on their resume to garner interest. Most of the top companies don't look at 10 years of internal IT work at the same firm as a plus. And when you get 20 years, like it or not (and I don't!) there is age discrimination.
Net - there are lots of new jobs for talented new people, but it's wrong to assume that people losing their jobs can naturally fit into them.
I think you massively underestimate the complexity of routing and equally overestimate the difficulty of "data science" there. I'll wager a guy that can program router firmware can turn his hand to anything.
If we double the amount of IT workers available, do we still have a shortage? How do we determine what a shortage is?
I think it's all perspective. From the manager/business owner who is looking for the lowest cost and highest-payout labor, there is a shortage. To the actual laborer who is competing with other laborers, there is an oversupply. So this talk about there being a shortage like it's an objective fact really strikes me as bizarre.
You can't say that e.g. a rise in the price of pencils means it's "easy to tell" there's a shortage of pencils. It could mean that:
1) more people are getting better utility from pencils.
2) pencils are getting harder to produce at the current scale.
3) an input supply to pencil construction is rising in price
Applying these lessons back to the market for developers, it's pretty easy to see parallels to ALL three of the above scenarios. Which explains the rising price.
Unless you're using the word "shortage" to mean "you can't get what you want at the price you want", in which case the statement is almost tautologically true.
And, like many basic economic principles, only works in the artificial environment of a fictional, deliberately simplified economy.
House prices, for example, have a very strong dependency on the cost (and availability) of credit; people will spend as much money as the bank is willing to lend them. Turn on credit, price of houses goes up. Close off credit, price of houses goes down. This is but one example of many; the extraordinarily simple model of a commodity price, "supply and demand", is not inherently wrong, but is so simple that to rely on it alone in the complex economy we have is wrong.
I'm talking about the fact that to apply the extraordinarily simple supply and demand model to the price of houses, and expect it to work, is wrong. Other factors are more important.
If we were talking about something very cheap and fungible, it would be far more applicable. We're talking about houses. The majority of people cannot afford to buy a house. They don't have enough money.
If there was no credit, we'd see a simple supply/demand model at work; I could take 20 percent off the cost of houses, and most people still wouldn't be able to buy them because they don't have enough money, but a few more would be able to, so the demand (or rather, the number of people who want one and can now buy one) would increase. However, there is credit.
If I added 20 percent to the price and also arranged to lend people enough money to buy the newly expensive houses, they would sell. If I doubled house prices and lent people the money, they'd still sell. The price of houses is so dependent on the availability of credit that it swamps the simple supply and demand model. Credit doesn't just play a factor; it's the majority of it.
They're also a veblen good; the simple supply and demand idea that as prices go up, the demand will drop, isn't just wrong - it's the complete opposite. As prices go up, people want them more; they fear that if they don't buy one now, it'll cost them more in the future ("don't miss the boat" is what often gets trotted out during the housing booms) and/or they think of it as some kind of investment and think they'll make more money since they're going up. The simple model that prices going up leads to demand going down is inverted. This works as they go down in price too; as prices drop, demand drops. Nobody wants to borrow money against something that will soon be worth less than they owe on it. Dropping prices cause their own positive feedback, every drop pushing demand lower, in turn dropping prices further and amplifying the effect; classic boom and bust behaviour.
Houses are a veblen good, dependent on the availability of credit to buy. Simple supply and demand is utterly swamped. Look at the last housing boom (and indeed, the ones before it). Was there a massive reduction in the number of houses available? No; in fact, many housing companies were building them as fast as they could and the supply was increasing even as prices bubbled beyond sanity. Did hordes of people who had been living on the streets suddenly decide they wanted to live in houses? No. It was people suddenly able to buy one because someone was willing to lend them the money.
We already know what happens when companies do this kind of introspection - obvious problems are confirmed, and in most instances are either too difficult to fix, or any change wouldn't be cost effective.
In my opinion, the "Big Data" demand is simply being driven by modern business managers who have an unhealthy lust for statistics. While I can't deny that there are some amazing applications of "Big Data" out there, a lot of what I'm seeing makes me say to myself "You wouldn't know what to do with that information, even if you had it."