What STEM shortage? Electrical engineering lost 35,000 jobs last year
computerworld.com
computerworld.com
Similar people doing similar things for society are suddenly software people, not hardware people, despite the fact that the software they're writing targets a physical device rather than an anonymous Turing machine in the cloooooooooooud.
Consider the employment profiles of "companies selling electronic locks" in 2002 versus 2012. These days it probably ships with a microcontroller. That means, congrats, software people now outnumber hardware people, at least as the BLS measure things. (If you really want to get wonkish, read up on what the tiebreakers are if your work implicates multiple SOC classifications. If you spend more time with a mouse in your hand than a soldering iron you're probably going to get bounced out of engineering occupations entirely, a result which is -- to put it mildly -- not a 1 to 1 mapping with our intuitive understanding of the claim "US employment of engineers just decreased by 1.")
Would it be fair to say that some percentage of that percentage probably aren't qualified to find work as developers, and that contributes to the elevated number?
Looking at any shortage by the number of applicants or job-seekers assumes that the bulk of the applicants are qualified to do the work.
This is not even remotely true in practice. A large proportion (over 50%, honestly) of the developers I've interviewed in my life can't tell the difference between a tree and an array, much less apply them intelligently. The majority cannot articulate the performance and storage differences between arrays and hash tables. These are the most basic qualifications for writing code.
Without excluding this massive, massive pool of yahoos, I don't think the statistics here are ultimately that meaningful.
That seems like a very strong assertion. I don't have formal CS/data structures training; I've read about all of these structures in the past but probably wouldn't be able to answer your questions about them to the level you seem to expect. (I could tell the difference between a tree and an array, because they look different, but frankly I have no idea where it would make sense to use a tree; they aren't offered as a native type in any of the languages I use day-to-day and I've never missed them. I know a bit about the difference between arrays and hash tables because it's one of the most common criticisms of PHP, but again; to the extent that I use PHP--and I avoid it like the plague--their implementation of "arrays" using hash tables has never once affected me. It is the least shitty part of that language.)
So anyway, I confess my ignorance. And I'm very open to learning more about the structures you're talking about (seriously). But I (and many of the programmers I know) have been successfully writing programs that work for people who want it (and gladly pay for it) for a relatively long time without meeting your most basic qualifications.
When I say that someone can't tell the difference between an array and a tree, I really mean someone does not know what a tree is. The number of interviews I've conducted where the candidate has heard of trees before, but cannot articulate, even in a basic way, what they are, is depressing.
To be more clear, the basic level I'm expecting is "it's a way to store data, where one object has children objects, which in turn have children objects", or something along that theme. Really, if you articulate in some way that it's stuff pointing to other stuff, you're already further along than half the people I've interviewed.
A lot of people can't even get that far.
These questions aren't designed to defeat you, they're questions that have depth so that you can get some idea about the candidate's level of understanding of data structures. Once you get past "it's an object with children" we can go deeper - what is the complexity of fetching an object out of a tree? What of sorting? Disadvantages and advantages in application? Cycle detection? All of these are nice to haves - but the basic requirement is that you have to know what a tree is.
EDIT: I'm more distressed by the number of applicants I've come across who've never written an SQL query, or have no idea what a database schema should look like (never mind noSQL stuff). I generally prefer working with programmers who understand and appreciate big picture design principles over the ones who sweat the details (because in my experience the former are usually able to pick up details pretty quickly, while the latter are not always able to appreciate the big picture design decisions being made) but in bigger teams there is presumably room/requirement for both types.
I've worked with a lot of academics who write code for math and stats without any real computer science knowledge (including no more than a rudimentary understanding of the most basic data structures). I have a business mentor who makes millions of dollars with a piece of legacy web software he wrote himself--I'm certain he could tell you that certain fields are indexed in the DB because the SQL performs better that way, but I doubt he would use the word hashmap/hashtable at all and that doesn't undermine his credibility in my eyes one bit. I have friends who write embedded code all day but couldn't write an SQL query to save their life; that is obviously fine for what they do, and it doesn't make them any worse of a programmer.
My example in the parent comment was obviously relating to web programmers, where the necessity of being able to write a usable SQL query is of far greater importance than the ability to explain how the database is actually executing that query. All I'm saying is that if we are able to deflate our own self-importance just a little bit (and accept the idea that the role of an employer might also be to act as a mentor), suddenly the pool of qualified applicants becomes bigger.
An example that received some discussion on HN last year was the employment numbers for the movie industry. Since many (most?) people who work on movies are technically independent contractors working under contract, they are not "employees" and therefore not included in some measures of film industry "employment".
I am confused as to the methodology by which IEEE extracted that conclusion from BLS data. On noticing my confusion, I attempted to verify every number in the article. I have just spent an hour with OES data from 2012 and 2011, since I believe the IEEE was looking at the 2012 data released in 2013 rather than the 2013 data which be released later this year.
I remain confused as to how to generate any number that the IEEE has quoted in this article.
I have considered the possibility that the IEEE has accessed to unpublished data from the BLS for 2013. However, if I interpret the numbers they are quoting as from 2013 with the delta from the year previous, I am unable to reproduce the BLS' numbers from 2012.
I am confused by the magnitude of the discrepancy. For example, the IEEE claims that "The number of employed software developers, the largest IT occupation segment, increased by only 1.75%, to 1.1 million, a gain of 19,000." I have difficulty reconciling this with the BLS numbers from 2012, which report that employment in that segment is 1,397,780. The one-year gain in that segment is, according to the BLS, over 100,000 jobs, again assuming that we're looking at 2011 to 2012.
I have taken the liberty of repeating the analysis while intentionally making several easy processing mistakes in interpreting the 2012 data, for example looking at the numbers for only the private sector. I have not yet found any combination of data sources or mistakes which produces the numbers the IEEE is claiming.
An interesting aside is that I think the type of software I'm writing benefits from having a deeper knowledge of the underlying hardware.
Have to say that's the evidence I find in much of the developed West. I watched a program in the UK last night where they were flouting the Government encouraged idea that over 100K new STEM graduates were needed a year - many of my friends leaving University with STEM degrees or thinking of enrolling on such degrees haven't bothered because job creation at that scale doesn't exist.
With outsourcing and offshoring by the bucketload it's refreshing for this Ron Hira guy to point out the reality for many.
Added link to summary of TV program if anyone is interested: http://www.itv.com/news/2014-01-16/tonight-is-britain-back-i...
I always found the idea of a post-industrial economy rather perplexing as it relies on the idea that we can design and manage the manufacturing of the rest of the world, but the skills required to do that are dependent on having manufacturing experience, which will die out in a generation without an industrial base in which to get the required skills.
Like the US, the UK still manufactures loads of stuff - it's just there aren't many people employed doing it and it's a much smaller percentage of the economy than it used to be.
e.g. http://www.theregister.co.uk/2010/02/22/manufacturing_figure...
The program I watched was also interesting as it followed HSBC who had brought call centre jobs back from India to the UK - what it failed to mention is that the higher paying STEM based jobs are moving (rapidly) in the opposite direction!
FinServ is a great target for offshoring, even better than software IMO. The tasks are more rote/routine and well defined which makes it much easier to send it to a cheap team in India. The big concern is the competitive advantage of the information but if you just work as a clearinghouse then it's not as big of a deal since the information is not proprietary.
A friend of mine works at a financial research offshoring company and they were just acquired by Moody's.
The problem is not that STEM workers can't be found, it's that they can't be found for the wages business owners want to pay. And what they want to pay for skilled labor will always be: "less".
Being from the US, with an Italian wife and a daughter born in Austria, and working with people from all over Europe and the US, yes, I suppose I am some kind of "open borders weirdo" in that I tend to see the value in other people as something inherent and not depending on where they happen to be born or what their passport says.
Granted, I don't think Lichtenstein should probably open its borders 100% to India tomorrow, but in general I don't believe there is a fixed amount of labor to go around:
http://en.wikipedia.org/wiki/Lump_of_labour_fallacy
Immigrants have brought a lot of great things to the US and elsewhere. For that matter, the economy is a lot better now than 100 years ago when there were far fewer people.
> the economy is a lot better now than 100 years ago when there were far fewer people
The era of strongest wage growth and increase in living standards coincided with the period between the 1924 immigration restriction act and the 1965 immigration act, resuming immigration. In that interval net migration was roughly zero. It's tough to argue the USA was badly suffering for want of more foreigners in that time.
Can you think of any other events in that time period that might have had some effects on the economy? It's not exactly an experiment with a control...
> I also see the value in other people, but in addition I see the value in nations and rooted associations and loyalties.
I suppose it's a bit harsh, but IMO that kind of thinking is what breeds mentalities like those that created what the NSA has become. My loyalty is to my family, my friends, of whatever nationality they happen to be, and after that it gets a bit vague. I'm lucky to have been born in the US, certainly, but it's one country of many, with good things and bad things.
> mentalities like those that created what the NSA has become
A cohesive citizenry with a voice in policy leads to illegal spying? WTF?
Do you want to live like that ? If the answer is no, you must find a way to avoid it, and not being an "open borders weirdo" is one of the more common suggestions.
The reason STEM is pushed is the same reason the US has the H-1B visa - to drive down wages and increase labour supply in STEM fields...
He wasn't able to find another EE position, except at a small company as a part-time tester. He's retired now, mostly because he gave up trying to find fulfilling work.
Now, the kicker is that the neurosurgeon is in his 60s.
I think he missed the implication that if programmers must stop by 40, but neurosurgeons can keep going and keep up into their 60s ... neurosurgery is easier than programming?
(I don't doubt that in all fields there are people who slack off, but also in all fields there are smart people who keep on learning. It is very sad that 50's engineers are exculded on the assumption that they are slackers.)
The STEM shortage is so true, while I agree outsourcing saves money, H1B are most often not, more likely, it's because the company can not hire the right STEM candidates and they have to go through the painful H1B hiring process, while paying for their very expensive immigration attorney bills to keep them.
More high-quality STEMs, less outsourcing, are the real solutions for a booming local job market. When we have more good STEMs locally, the H1B will become a non-issue.
Anyway, as a matter of national policy it's remarkably short sighted to import large numbers of skilled workers to a large nation like America. Can't people see that addressing any "shortage" this way perpetuates it? People have got the memo loud and clear: If you are a technical worker you are a fungible commodity of little status. Your prestige and earnings will be attacked with immigration policy, visas, and outsourcing. Why go in for this as a career if you have options? The most able will go elsewhere.
Why would outsourcing be worse? The detailed analyses show that bringing people here has large costs externalized to tax payers. People use schools and when eligible chain-migrate low skill relatives.
If businesses feel the business environment the citizenry of America have built up is not worth the costs, well I'm fine with them choosing other places to operate. What they shouldn't be allowed to do is import a new citizenry.
Usually H1Bs' kids excel at school, and their parents pay tax on that. For their poor relatives to immigrate 22+ years later(roughly 5 year for greencard, 5 year for citizenship, 12 years for relative immigration waiting period), it's a immigration policy issue, however it's still all legal, at least 10X better than those who came here illegally.
Keep jobs local is vital, outsourcing R&D is so short-sighted, with more STEMs, H1B will be irrelavant.
Above is also the reason good (not great or rockstars) developers are in high demand and almost never on the open market. Once you work with people and vet yourself with other good developers, new jobs are usually just a phone call away.
Every job I have ever landed, I've had to go through the full HR rigamarole, from finding the posting through multiple rounds of interviews. I've never quite been able to make a friend--or even acquaintance--that has subsequently been able to backchannel me into a job. I've had several interviewers say directly that I appear to be one of the most technically qualified candidates they have, but after I walk out the door, I never hear another word from them again, not even to say they picked someone else.
The hiring system is broken. And the damage is at least in a load-bearing member, if not the foundation itself. There is no financial pressure for anyone capable of fixing it to actually do so. And so we suffer as entities with far more political experience glide smoothly among the sharks, depressing our wages today, and tomorrow, they quietly dismantle the dreams of young people passionate about our field.
It sickens me to the point that I feel like there are no good employers left in the U.S., in any field. I have no reason whatsoever to buy a house, befriend my neighbors, and invest in my local community, because at any moment my gracious employer (or their new owners) could decide to fire everyone and move out. That would leave me again in the position where I have to also move, just to find a job not quite as good as the one I just had.
And now that I am in one of the few locales recognized as a "tech center" in the U.S., I find that most of the jobs here are military-industrial complex tax-leeching jobs, where industry best practices are routinely ignored, because the primary goal of the contractor is to produce billable hours instead of permanent solutions, and the overseer-boss got his job via military good-ol'boy networking rather than having one inkling of competence.
What's the solution? I don't know. But making me retype my entire resume into your stupid web forms to better facilitate its automatic rejection for lack of required hidden keywords is completely antipodal to it.
If you don't know what Ruby Gems or Active Record are, you've likely not touched any web technology in depth.
But if you can't code common data structures on your own, you'll probably have trouble coding the more advanced data structure that you found a paper on describing (possibly described in part by replacing parts of a more common data structure) with some code, but not in your language of choice, that would be, from its description, far better than the common data structure for your problem, for which there is not a debugged library for your platform of choice.
It's the edge cases that separate a good engineer from an average engineer. When that good engineer runs into that problem, he/she will solve it quickly and move on. An average engineer will spend days or weeks struggling with the solution -- costing lots of money and potentially delaying schedules.
This is a learning task that might take a competent developer all of 2-3 days. If you're rationalizing not figuring out how to write a heap or a hash table, consider asking yourself whether you actually want to be a developer. You might feel OK about what you're doing now, but a career spent redesigning unit test suites and coming up with ever- more- elaborate HTML templating schemes is unlikely to be fulfilling.
It's very possible to make a positive programming contribution in a professional context (and in a wide variety of fields) without having ever implemented a hash table (I had not, before just now). Obviously it's more important in some contexts than others, and that's fine, but the suggestion that programmers who don't have that experience are not programmers or that they're incompetent programmers is nonsensical--the context in which they work makes the difference.
Now obviously, it's been a bit embarrassing making essentially this same post multiple times recently, so I just implemented a hash table and read a few sources for heaps and binary trees--yay! But this isn't rocket science--if I ever encounter data which doesn't conveniently fit in the basic structures available in my environment, I go find out what works (so, for example, I'm working on a project where I need to find the closest points in a data set according to Euclidian distance, and discovered that VP trees fit the bill).
It is entirely possible that the ability to synthesize a simple data structure is a valuable signal of programmer competence for people with a formal computer science education (because they were presumably taught and tested against that knowledge multiple times, and if they can't reproduce it afterwards they might be bad at learning) but my suspicion is that the implementation of domain-specific algorithms is more often limited by a lack of domain knowledge than a lack of CS knowledge.
Well, do they? Being able to code simple data structures is not a pre-requisite to being paid to write code.
I can't base this statement on cold hard numbers, but I feel comfortable stating that the majority of dev positions in the world today do not require this particular skill set.
Thus : no, they (programmers) don't need to be able to understand or be able to code simple data structures.
If the inverse were true - namely, that a large percentage or majority of dev work required knowledge of and the ability to implement basic data structures - then you would be quite correct.
> You might feel OK about what you're doing now, but a career spent redesigning unit test suites and coming up with ever- more- elaborate HTML templating schemes is unlikely to be fulfilling.
That's a bit dismissive. Also, are you speaking generally or referring to me as an individual?
The realities are more stark in our breach-infested world. Sloppily implemented systems put companies in the headlines.
Examples abound, not only with data structures, but also with algorithms. Someone who has written a compiler is far less likely, in my estimation, to try to "parse" a URL with a regular expression, with negative consequences. Someone who has written a hash table is less likely to be surprised when the insertion behavior ends up being n squared.
Are programmers better off having a deep working knowledge of data structures, algorithms, compiler & OS design? Sure.
Do they need to, in order to be (paid, professional) programmers? No.
If I were to poll all my SE colleagues, present and former, and ask them when they last manually coded a linked list or b-tree - I can assure you the answer would not be "on a product we shipped".
Well they don't want to brag, but an online community where you can be anonymous if you want seems like a good place to reach out for help if you're in trouble.
Beyond the lump of labor fallacy: http://en.wikipedia.org/wiki/Lump_of_labour_fallacy , did you consider that those people have every right to do their best in this world as anyone else, no matter where they happen to be born? That might mean coming to the US or Germany or whatever. If that's easier for them, fine by me.
Nobody's proposing any anti-"doing-your-best" legislation, nor blaming the cheap Indian labor, who is often exploited terribly.
Say you're born in someplace really bad like the Congo, as an extreme example. No matter how you try, you are not going to do your best there.
I live in Hamilton (which is Canada's Pittsburgh). We've got a great STEM-oriented school here, and Waterloo (RIM's home) is only an hour away... it took a long time to find a job that didn't involve suffering through a two-hour commute in some of the most miserable traffic on the continent (Ontario's corporate tech-hub is Mississauga).
Ontario is rough. Most job offers are in the range of 65-75k for 10+ years of experience and highly-specialized skills. The cost of housing and living in Toronto and surrounding regions has sky-rocketed in the last 10 years. And the traffic is terrible.
All of the STEM grads get shipped off to SF or the Valley. There are American recruiting agencies on the campus' of every major STEM-focused university in Canada and are at every job fair. The choice for these grads is pretty easy: stay here and make Wordpress themes for a social media marketing agency and get paid 35-45k or go work at Google/Twitter/Whatever and work with the best-of-breed tech, get paid 90k to start and be making 120-130k within a few years.
We could do better up here. The visa situation is much more permissive here and there are a plethora of grants and tax benefits for tech companies available. VCs are even finally starting to warm up a bit too.
[1] http://money.usnews.com/careers/best-jobs/computer-programme...
[1] http://money.usnews.com/careers/best-jobs/software-developer
[1] http://money.usnews.com/careers/best-jobs/computer-programme...
There's no shortage in the more creative jobs creating new apps, cutting edge web dev, network engineering, game dev, etc.
The kind of stuff that inspires most kids to go into comp sci.
It seems, the big 4 get more resumes than they can shake a stick at.
I could be wrong. Just my thoughts.
And yet the wages being offered for these jobs haven't gone up. It's pretty hard to have a labor shortage if the remaining workers aren't seeing wage increases as demand exceeds supply.
Isn't it possible the jobs just aren't worth that much?
There could be hundreds of unfilled positions, but not filling the positions is only costing say ~$50k in revenue. It's not worth paying someone $51k for that position
The nearby Costco pays better, has realistic working hours, and offers actual benefits.
I can't remember when it was that I saw such a job that paid a living wage.
Engineers insist on truth and struggle with the fact that most people who succeed in human affairs are extremely dishonest.
"Shortage" just means, "we can't hire talent at as low a wage as we'd like to pay". It is not an objective condition. It might qualify as a true shortage if companies were reducing the pay of executives and non-techs to make space for engineers, but that is happening almost nowhere.
Did you mean to do that? either way, it's pretty funny.
Do what? I'm not sure what you're asking.
Yes, I know what the word "phallic" means.
It does sound like it could have come from an Agile consultant. "Are your iterations flaccid?" But nobody actually talks that way... right?