The Bogus High-Tech Worker Shortage
pbs.org
pbs.org
If anyone is curious where the mistake is in this article's reasoning, it's in the assumption that people are more or less interchangeable, and that all you have to do is train them to be Xes, and you can have as many Xes as you want.
Whereas the reason there are open positions for programmers at high tech companies is that these companies want star programmers, and you can't train people to be stars. Someone asking why we need to hire programmers from other countries might just as well ask why Real Madrid has to hire so many non-Spanish players. Can't they just train more Spanish players to play at the level they need?
Incidentally, when I first started to see these articles a few weeks ago (I don't know if it was this one or another one like it), I was curious why anyone would put so much effort into saying something false. It turns out the authors work for something called the Economic Policy Institute, which is funded by a consortium of labor unions. I was a little puzzled that unions would care about this, since none of these programmers are unionized. I suppose they must see it as the thin end of the wedge.
When demand exceeds supply, prices should go up until both reach equilibrium; conversely, when supply exceeds demand prices should fall until both reach equilibrium. In a real market, there will always be buyers who are overpaying or trying to underpay. You cannot cherry pick these instances and make any sort of valid assessment of the market.
There's a lot of room on all sides to improve the quality and availability of good technical workers.
And, in the US at least, the financial industry siphons off a lot of otherwise excellent talent that could be going towards more productive ends than shaving milliseconds off automated trading.
Can you elaborate on how you envision things like hacker school to solve the problem of stars, if those can't be trained? I am not trying to be semantically nit-picky - I don't think I understand how star programmers come about, and maybe you have better insight.
The only thing I can think of is that you might imply that being a star programmer is an innate talent that needs to be developed, and things like hacker school would allow for this talent to be discovered. Is that about right?
Stop whining and enough already with the arm-chair generalship, easiest thing in the world.
Remember this? http://www.youtube.com/watch?v=VpkJZuxSayo
Of course, the growth in graduates may not keep pace with the growth in companies!
"Star programmers" are a completely separate problem, as Joel has mentioned before (e.g., are your stars really "Stars" if they're just the best 21-26 year olds your network knows of that are willing to live in your geographic region and not interested in graduate school?). But it's the horribly poorly managed firm that can't turn an A student from a top-fifteen CS school who did any Big Project at all succesfully into a very productive team member.
What about the assertion that companies are struggling to fill these roles because (a) they demand more from potential candidates while (b) offering no more in wages than what we saw 10 years ago (edit: 15 years ago, according to Salzman et al. [1])? For example,
"Or is the hidden truth quite simply that large supplies of guest workers allow many firms to swap out higher-paid, high-skill domestic workers for lower-paid, high-skill guest workers?"
The article spends very little time arguing for the training of Xes and much more time arguing for higher wages for potential Xes. This gets rehashed on HN all the time: offer Xes more, and maybe they won't roll over to finance instead of accepting your offer.
---
RESPONSE TO PG'S EDIT: discrediting the source is kind of misguided for two reasons: (1) their wage data comes from the Census Bureau (see pg. 19 of [1]) and (2) the Brookings Institution essentially agrees with the EPI [2]:
"[I]t is likely that the extra supply of foreign-born workers does bring downward pressure on the wages of incumbent workers, as research suggests."
I don't think you could accuse Brookings or the Census Bureau of having a similar agenda.
---
FINAL EDIT: PBS quoted Brookings out of context, and I took it at face value. The paper doesn't support the EPI's findings, and the full Brookings quote is below:
From a theoretical standpoint, it is likely that the extra supply of foreign-born workers does bring downward pressure on the wages of incumbent workers, as research suggests.20 Yet, it appears that demand is so strong relative to supply that even the inflow of H-1B workers is not enough to meet the demand of U.S. companies and push wage growth down to normal levels.
Ugh. Apologies to PG and HN for citing this blindly.
[1] http://www.epi.org/publication/bp359-guestworkers-high-skill...
[2] http://www.brookings.edu/research/papers/2013/05/10-h1b-visa...
"If you're a high math student in America, from a purely economic point of view, it's crazy to go into STEM." The best of these students flock to Wall Street and corporate law firms and the rest end up employed in non-STEM jobs which often pay less and underutilize their skills.
If SV companies could draw star programmers away from Goldman, we'd then be talking about a finance shortage, no?
Search 'finance companies lobby h1b visa'. Nothing Then search 'bay area companies lobby h1b visa'. Tons of results.
It's a Red Queen race; whoever runs the fastest front-running operation skims all the rent from naïve institutional and individual investors. If everybody is 200ns slower, everybody is in the same position as before. All the talent invested in saving a few ns does no good to the world; it just changes the relative position of finance companies.
They might even be happier if there were fewer techies they could hire.
To illustrate, imagine that IT workers used to get paid a $100k real wage in 2000 and a $50k real wage today. Imagine computer scientists and programmers got paid a $100k real wage in 2000 and get a $150k real wage today. Bundling them all into IT would lead you to mistakenly conclude that a computer scientist would expect to make as much today as they did 10 years ago.
Can someone confirm this logic is correct?
I don't know what the real stats are, but you haven't provided them. I also notice that the Brookings article you linked states that real wages have increased for those occupations in the past few years, but I don't how if at all they are bundling professions differently.
"Or is the hidden truth quite simply that large supplies of guest workers allow many firms to swap out higher-paid, high-skill domestic workers for lower-paid, high-skill guest workers?"
The Brookings article you cited states the opposite. Foreign workers get paid more. If there truly are enough good programmers and computer scientists available in the states is there a conspiracy to subject oneself to the nightmarish visa system and pay high legal fees to be able to hire foreign workers and pay them more than Americans, all the while not offering high salaries to qualified Americans? What would motivate one to do that other than a shortage of qualified candidates?
You're right that the Brookings article suggests that my quote isn't the case. I wish we had more information on how they controlled for different effects. IIRC BLS etc don't generally control for cost-of-living, and Brookings doesn't discuss the issue except for in the conclusion:
Likewise, the bureau should also consider how demand and supply play out in regional or metropolitan area labor markets, since job search and recruitment often happen locally.
I'd be interested in seeing those results, as I would expect a greater concentration of H1B workers in large metropolitan areas (SF/SV, NYC, Boston), where wages are much higher than the flyover states (where I'd expect less racial/ethnic diversity).
From a theoretical standpoint, it is likely that the extra supply of foreign-born workers does bring downward pressure on the wages of incumbent workers, as research suggests.20 Yet, it appears that demand is so strong relative to supply that even the inflow of H-1B workers is not enough to meet the demand of U.S. companies and push wage growth down to normal levels.
You're absolutely right--what lazy work by PBS (and me, for assuming they fact-checked). Ugh.
Real Madrid puts a lot of effort training Spanish players. As do other teams. This is why they won the World Cup.
And no. There is no draft day but in some countries there are unions.
Just sayin.
Ref: http://www.globalsoccertransfers.com/info/regulate.html
Very well said.
In the U.S., the reason why is "salary caps" in professional sports leagues.
I'm just saying take the polarized glasses off and you might see more stars.
Lots of people also want to play pro ball. They can't, and no matter how much they train, they never will be able to. Their body just isn't built for it.
Similarly, there are people who can study their ass off. They'll probably become pretty good, not great, but pretty good.
Then there are a tiny, razor-thin minority of people who were born as some sort of intellectual freak, who will also study their ass off, and become great.
I've known a few people like that. It's frustrating, because I feel like I'm working just as hard, but not getting as huge of results.
p.s. I am an entrepreneur and have had significant hiring challenges for my company. So not spewing out stock lines from Lou Dobbs and his ilk.
I really don't understand why this is a difficult concept to grasp.
The basis of outsourcing at most large corporations is that people are more or less interchangeable.
Someone asking why we need to hire programmers from other countries might just as well ask why Real Madrid has to hire so many non-Spanish players. Can't they just train more Spanish players to play at the level they need?
Sorry but that is a ridiculous analogy. Most programmers here on H1B's are boiler room grunts. They work for large consulting companies where their key talent is looking like they know what they are doing. There are very few remarkable Cobol programmers working for InfoSys at Fortune 500 companies.
Don't you think they already would if they could?
"There is no poverty. just shortage of people with money"
:Facebook/Google/Twitter/Microsoft/Amazon aren't the major users of H-1B Visas. Consulting firms are.
http://www.npr.org/blogs/alltechconsidered/2013/04/03/176134...
The truth, of course, is that those rockstars account for a very, very small portion of the sought after visas. The vast majority of the allocations are for average professionals at companies far from the bleeding edge.
It seems odd to me that we would put in such an effort to import average workers when there is both not an insignificant number of unemployed American tech workers, and a large number of tech-educated American workers pursuing other vocations. This smells more like a failure of employers to lure the workers they need than of a true shortage.
I do think top companies should be able to scout internationally for top talent and bring them back to live and work in the country. But top talent demands top dollar, so there needs to be a bar. A company must be willing to pay them significantly more than their position's average salary. If they're the best of the best, they're worth it.
To stick with the soccer analogy, how much more do you think someone like Lionel Messi gets paid than a junior league player, or even an average pro player? It's understood in sports that you must pay top dollar to get top talent. I don't think it's too much to expect industry to do the same.
Spain 14
France 3
Germany 2
Portugal 3
Croatia 1
Argentina 1
Brazil 2
Given the Schengen agreement, that means that 22 out of 25 players (88%) are not guest workers.
EDIT: 22 out of 26 (85%)
Maybe they should try paying like stars. What percentage of engineers that aren't founders or within the first five employees make over $300,000 a year? The money is in Silicon Valley, Google's 2011 net income was $299,874/employee [1], but the vast majority of engineers aren't going to be the ones to get it.
How about paying a market wage, instead of offering a "free lunch" to people that cannot negotiate. I am not interested in playing video games at work, I have a house for that. Last I looked at it, the difference in pay between finance and stem jobs is enough to hire a personal chef to deliver lunch to you at work every day, and cook for you at home, and buy all the games you want, with a bunch of money left over. Apple and Google are both posting ~25% net profits because engineers are willing to work for peanuts, and a free lunch. On top of that, cost of living in the bay area is outrageous, we're talking $35,000/year for housing, unless you want to live like a college student.
An average (median) engineer should be making $200,000 a year, and the "rockstar" (>80th percentile) engineers that everybody is trying to hire for $85,000 and 0.5% of equity, should be making $350,000+. These are hard jobs that few people can do at all, and fewer can do well. Great engineers can create millions of dollars of value a year.
It seems that the industry has shifted focus, now that the collusion agreements have been squashed, toward importing cheap labor. This problem has been solved in other industries through partnerships and professional societies (e.g. in the legal industry, with large partnership firms and bar associations). In the interim, it seems like the best way to capture value is to build companies to flip in talent acquisitions, while planning to leave to do it again as soon as you're vested.
[1] Goldman Sachs 2011 net income was $162,913 per employee. Average pay was $367,057. It's harder to get data on Google, but it looks like the mid-career median salary is $141,000.
Google's Gross Profit (Total Revenue less Cost of Revenue) was $24.7B in 2011, and GS's was $24.5B. At the same time, Google had 32,467 employees to GS's 35,700. Yet, the Google net income was $9.7B compared to $4.4B on the GS side. The difference seems to be made up entirely by the difference in employee compensation.
Interesting point, and really indicates that companies are not willing to pay enough for talent when hiring them individually. Even a startup that is nothing more than a bundle of talented individuals has enormous leverage just because of that bundling.
But what's being proposed is a vast expansion in the number if visas available. Most of the new ones will go to the same mediocre outsourcing company drones that fill the current quotas.
It's just an expansion of the current system that rewards blizzards of similar applications from cheap workers with faked credentials. (Or credentials that document some kind of educations and experience without reflecting any talent)
A bidding system would wipe out those drones in favor of American workers. Meanwhile, everyone who needs stars would get them. Of course, bidding hasn't been on the table since the early 1990's. Republicans were inalterably opposed to it then because it would support middle class wages and Democrats oppose it now because they expect a mass of immigrants to eventually vote for them.
It could raise a non-trivial amount of money. If there are only 65,000 H1B visas (there is talk about doubling it, and with exemptions the actual number is almost double currently)and the average winning bid is $20,000, that would be $1.3 Billion total!
A tuition free university, University of Helsinki, is what Linux came out of. Free CS classes. A star was trained there.
The mentality of "you can't train stars" is bogus...it's more "we don't want to invest anything and want everything handed to us free and easy, and if we don't get it that way we'll complain about a shortage of stars"
It's great that it's been deduced that organizer labor is behind this article, and thus it's not neutral and has a hidden agenda. I'm sure someone who takes a 6% stake of high-growth companies for $14,000 is completely neutral with no hidden agenda. Also it's false that "none of these programmers are unionized". Programmers are represented by unions such as the IAMAW, CWA etc. It's possible 3 person startups have no union representation, but that's not out of the norm.
-On the claim of unfillable positions, "It's not bogus."
-"companies ... can't find people"
-"the mistake ... in article's reasoning is ... "
-"the reason there are open positions ... is ..."
-the writer's "put effort into saying something false" because ...
Can these statements be backed up? Is there is a metric for star programmers that companies can measure? If there is an obviously star programmer, but in some obscure language, do they get job offers? Perhaps in your capacity as startup advisor, you have inside knowledge?
We have been small groups, not pursuing H1-B visas or anything and it's possible H1B's are being given grunt work... However, the graph in this article, and really the central tenent that "we have so many tech workers not being hired, hire them first" is complete malarkey. I've gone to Undergrad and grad school. I've TA'd... The fact that even two thirds of those students find jobs (As the article claims) astounds me. Thats a higher number than I'd imagine. In two years TAing at a large research university I think I saw maybe 4 students I'd hire go through the program.
Some schools give good industrial training; some are still bemused that machines exist that perform the computations that are so fascinating to study in the abstract. We need both. It's true that schools can't manufacture stars, but they can help train those with latent ability, just as Real Madrid players are certainly helped by coming up through smaller teams.
Likewise, some places (YC startups, or forward-looking splinter groups in larger industry) need rock stars who can do a lot; but there's a huge swathe of megacorps that just need warm bodies that can tab-complete API calls; the problem with debates like this is that arguments are so rarely anchored by a clear context. Are we doomed to always argue straight past one another?
Speaking of context, in many discussions on this and related topics here I see many contrasts like this raised. There is a vast gulf between startups that (truly) need rock stars and corporations that need IDE monkeys. There is a vast gulf between top programmers who read CS papers for fun and implement problems in 8 different languages out of curiosity, and "programmers" who can't even write FizzBuzz in pseudocode. In these discussions, I get the impression that this segment is either nonexistent or is so small it doesn't matter. As someone in that segment, it make me question if I should be in this industry. If the industry truly has a programmer shortage, I would think that is a poor impression to give.
Please note that his work is backed up research, please consult the links on his site.
At the end of the day it's much easier to import top talent to fill the gap than to cultivate more of it in the U.S. Some companies like HackerSchool are popping up to address this need, but there's a long way to go.
In fact, the vast majority of foreign worker visas aren't snatched up by the tech goliaths. They're snapped up by low-tier consulting companies. [1]
Google may be forced to look abroad to find rock star programmers, but the overwhelming majority of foreign workers brought in to the country are not rock star programmers. They're rather low paid consultants.
1: http://www.npr.org/blogs/alltechconsidered/2013/04/03/176134...
I am interested in the salary thing, though. I think there is at least a little bit of companies bemoaning the fact that they can't find a person to fulfill rock-star expectations on a junior salary. It's macro 101, if there isn't enough supply then increase the price.
The laws are written so that guest workers are basically indentured servants to the companies they work for. Furthermore, we basically give guest workers 6 years of well-compensated on the job training after which point we say "go home and enrich your home economy." If I were to say the things that need to be fixed:
1. Once you've been working at a company for a year, you should be free to remain in the country whether you have work or not. Companies shouldn't be able to hold you hostage. Companies should probably even have to sponsor you for some sort of H1B level 2 to keep you around past 1 year. (Bear in mind, I'm not looking for the current bureaucratic nightmare, I'm looking for companies to have to give guest workers their freedom well in advance of the current six-year mark.)
2. If you've been gainfully employed in the country for three years under an H1B, you should get a green card automatically. No waiting around, no nothing.
If we do this, wages will rise naturally, and we'll see whether companies have a legitimate shortage or they're just looking for cheap labor.
That said, points 1 and 2 are definitely worthwhile. But you're devaluing your points when you compare a high paying job with some annoying visa clauses, with indentured servitude. Indentured servitude is really serious: it's like when people talk about "raping" the environment. Don't make the comparison, it serves no-one.
More importantly, the answer to the shortage in the long term should not be to try to get more of the same. Our real goal as computer scientists should be in making our own jobs more efficient. We need to empower more users to have control to manipulate the machine they are using instead of entrenching us deeper into this never ending hole of proprietary software that continually re-invents the wheel and never works precisely how the user would like.
More concretely, we need to automate programming by making it more accessible to everyone and eliminate our own jobs.
This is why there's such a staggeringly large number of tech-all-stars whose education & experience comes from non-traditional places.
Has any study not sponsored by the Industry shown that there is a "labor shortage" in STEM fields?
http://www.epi.org/publication/bp359-guestworkers-high-skill...
seems to say yes:
"Analyzing new data, drawing on a number of our prior analyses, and reviewing other studies of wages and employment in the STEM and IT industries, we find that industry trends are strikingly consistent: ... Wages have remained flat, with real wages hovering around their late 1990s levels."
EDIT: if you're skeptical of EPI, Brookings also agrees:
http://www.brookings.edu/research/papers/2013/05/10-h1b-visa...
DOUBLE EDIT: to echo what I've said in other comments, Brookings disagrees with EPI's findings. PBS selectively quoted them.
" In recent years, from 2009 to 2011, nominal wage growth for U.S.-born workers with at least a bachelor’s degree has been high for the most prominent H-1B occupations. The average native-born worker experienced flat annual growth in wages over that period (0.0 percent), but wage growth for those in computer occupations—the largest H-1B category—grew by 1.3 percent each year since 2009 and 2.7 percent each year since 2000 for those with a bachelor’s degree. Wage growth was even higher for engineers, with 2.1 percent growth since 2009 and 3 percent growth since 2000."
Yet flying in the face of that is this idea of a "high tech worker shortage". If there really could be a shortage of high tech workers for longer than a short period, than the bedrock idea of supply and demand are bogus. Even miniscule hurdles like visas would mean nothing - if there is a mass of talent in India and a worldwide demand for that talent, then if visas were a problem, multinationals and trans-continental consulting firms would just set up there.
There is no high-tech worker shortage. There's a bunch of "idea guys" sitting around who think how much money they could make if they could hire a bunch of rock-star programmers on the cheap.