Is There a STEM Crisis or a STEM Surplus?
blogs.wsj.com
blogs.wsj.com
Companies are not hiring lots of people because they perform poorly in the interview. From what I can tell, a large fraction, if not the majority of those people actually lack the skills the employer is looking for, despite a 4 year degree in a related field.
This large pool of people who apply for jobs makes hiring more expensive, and also causes companies to apply low-sensitivity, high specificity tests to the pool of applicants (e.g. I'll interview only people who graduated from these 10 schools).
Hiring biases create a lot of problems. I can speak to this issue. I interviewed and was turned down by several well-known Silicon Valley tech companies, from big boys to trendy up-and-comers.
I did that because I was undecided about accepting a faculty job in a STEM field at a major university. It has worked out for me because not only am I enjoying being a professor but some months later I joined a non-SV startup who was happy to have me. My startup is on a trajectory to outdo all the trendy SV startups where I "wasn't a fit".
Without even launching into to biases in university hiring/admissions (so many it's shameful), I can say with good confidence that a large part of the problem is the employers.
https://blog.codinghorror.com/why-cant-programmers-program/
Remember, everybody that gets hired stops interviewing, while everybody who doesn't get hired keeps on applying for jobs.
I suggest we stop kidding ourselves on how hard most jobs are. Most jobs aren't hard and can be trained with a surprisingly low amount of effort. Though most people instead choose to spend that effort sifting through candidates holding out for the 10x engineer. I think we'd be better off seeking out determined individuals and getting on with the actual work.
Not to mention the compounding effects of the work experience catch-22 as for any job, AND a hidden requisite of unofficial apprenticeship experience.
It’s often difficult to find someone that doesn’t outright lie on their resume. It’s hard to find someone that will be able to contribute to your team rather than make a negative impact.
I try and hire engineers and we have to go through dozens of candidates to find someone who just seems motivated and competent
This has been our experience as well.
This statement is dependent on one's definition of motivated and competent. Additionally, I think in general a company gets the candidate stream that its job description/offered salary deserves. Pay peanuts, expect a line of monkeys applying.
While that might be true, if you pay six figures, all you get is an even bigger line of monkeys applying.
Git
Html
JavaScript
SQL
Web frameworks
Many people fresh out of school skilled in computer science.. they have no programming experience. They can code a linked list but that isn't actually useful.
Remember the pain of learning all this
Companies should accept this and be willing to invest in their employees, not just use them as cogs.
I've personally seen two bootstrapped companies locally who invest in college graduates and pay them slightly below competitive wages; they have some of the most loyal employees, and more competition for job openings than they could ever hire.
It's not ideal for every company, for sure, but it does work.
On the other hand, in most or all of the companies I've been somehow related with, more experienced developers have naturally invested their time in helping juniors learn.
Vs 6 months of training at fulltime pay, would be maybe $70k cost to a company?
Yes, it is far cheaper to interview tough vs train employees (for right or wrong)
Luckily there are plenty of Big Co jobs... but most still seem to ask for 2-3 years of experience, aka we can pay you crap AND not train you, a win win.
The jobs actually need git/js/SQL
The interviews are mind benders and school coding challenges.
Or perhaps they havent coded in a whiteboard in years?
Or perhaps they havent had to think about RB tree vs trei for a while, so can't code them perfectly.
What?
Intro Systems: demonstrate working knowledge of Assembly. Write C to exploit spatial and temporal locality. Learn to cope with concurrency.
Operating Systems: write process scheduling, virtual memory management, system calls, and filesystem caching into Pintos [0].
Networks: write an IRC server, user-land TCP implementation, and IP router in C from the relevant RFCs.
Advanced Networks: write Byzantine Generals, Raft, and Paxos from the papers.
Programming Languages: write several interpreters.
Parallel Programming: write efficient concurrent algorithms, implementing concurrency primitives yourself.
Software Construction/Engineering: learn to collaborate with a team. Lose your fear of merge conflicts by immersion therapy. Experience the pain of people not pulling their weight.
Intro Security: given a pseudorandom function, write your way up to a PRNG, stream cipher, RSA, Diffie-Helman, and an authenticated encrypted channel.
Databases: write most of a database engine.
If graduates are passing these classes but can't write code, or find it remotely difficult to get to a passable level on any of the tools you listed in a matter of hours/days, then the CS program at the schools they come from are not nearly rigorous enough and should lose reputation in favor of those that are.
A graduate of a decent CS program should have a decent idea of how to create these tools, and be capable of doing so, though with considerable difficulty. Consuming them should be pretty easy.
[0] http://web.stanford.edu/class/cs140/projects/pintos/pintos_1...
Other than Intro to Systems, which is required, these classes are representative of a list from which you choose 8. There's also 3-quarter intro sequence (all programming) and the 3-quarter theory sequence (all proofs, some pseudocode). You could get away without taking some of these, but then you would do similarly substantial programming projects in other domains (compilers, machine learning, 3D modeling and rendering, visualization of scientific datasets, etc).
Our department tends to be derided as "very theoretical", so I'm surprised to hear that we do more programming than others.
To be fair, you could also load up ~6 of your 8 with proof-based math classes about CS theory (graph theory, combinatorics, mathematical logic I-II, whichever of complexity or formal languages you didn't count towards the required theory sequence, etc) but the people who do that are on tracks towards PhDs in math or the subset of math that is CS theory, not the programming job market.
HN threads about CS education tend to posit that it shouldn't matter where you go to school because ~all CS programs are the same, so I assumed my experience would translate. Perhaps this is not so?
I do think that the more selective schools graduate better people largely because the start with the better people, but don't think the school makes zero difference.
I learned
1) Very crude Java (not enough to do anything else, just the bare bare basics).
2) Very crude C++ (just enough to mess up pointer referencing, not enough to code anything useful)
3) A tiny bit of LISP (mostly enough to make homework work, to this day I am still not good at LISP).
4) lots and lots of Math I have not used since college
5) Lots and lots of Physics I have not used since college
6) How to log into a SPARC box and use VI (I do use the VI portion, and the *nix skills, but not so much the SPARC stuff)
7) Nothing about source control, testing, html, css, databases
8) Nothing about security
9) Nothing about design patterns
I see sibling comments complaining about the state of CS education, but new graduates actually have among the best signal-to-noise ratio; it's the people that have hopped from job to job trying to hide their incompetence that are the worst.
It would be interesting to know how many H1b visa holders are employed in STEM positions at institutions that are exempt from the cap. What if it were something scandalous like 800,000 or 1,000,000?
If the numbers were this high, it might have something to do with the fact that there's so much double speak about STEM shortage / STEM surplus in the United States. More American STEM degree holders might be able to find jobs if they weren't forced to compete with foreign nationals.
In fact, it was the nation's science policy establishment that started this whole mess in the late '80s or so when they apparently decided they were paying too much money for scientific labor in universities and akin non-profit research institutions, as I recall the National Science Foundation got the ball rolling,
(But my focus on that "industry" back then might have caused me to overlook the same in the E and T in STEM, and certainly "consulting" and the safe tax harbor for it were a very big issue at the same time and cost me my job then, see Section 1706 of the Tax Reform Act of 1986, https://en.wikipedia.org/wiki/Tax_Reform_Act_of_1986#Tax_tre... .)
Fascinating. I hadn't seen that before.
Or just work directly for The Man, one company I worked for in '93-4, for a specific, time limited contract they were fulfilling (a cool NASA science Data Distribution Facility), had their default body shop refuse to hire me for some reason, so I was made a no/small? benefits hourly worker. Anyway, it suddenly smashed the working setup of many, many technical professionals, and very seldom for the better. And rather strange to single us out, yes, an indirect bill of attainder perhaps?
Further anyway, starting in the late '80s national/Federal public policy turned irredeemably hostile towards us, and you can see it in the static real wages we've earned since around that period. We may be viewed as essential, but....
a) actual shortage of specialists (shocker!) b) new projects that has to be staffed quickly (we need 50 Java EE developers in Phoenix, AZ, for the project starting in 6 months) c) moving existing projects to maintenance mode after feature completion: existing expensive team is downsized, and lower-cost maintenance engineers are hired to fix occasional bugs and user support (have to be hired fast and in large numbers, hence the shortage and need for H1B). This creates a lot of protest and drama, but looks like a perfectly valid business decision to me: some projects actually do reach completion stage, and keeping all expensive developers afterwards doesn't make any sense.
To hire somebody via H1B route, you need to: a) pay at least average salary for this position; b) prove that there are no local candidates available for your opening, c) prove that it is not because of your low pay rates, but because of genuine shortage of specialists with these particular skills, d) pay processing fees and wait for at least 6 months, perhaps more.
Common sense says if someone cannot freely move between jobs, who faces immanent deportation if fired, who will have much better prospects of living well in US than their home country, then this person has much less leverage and would be paid less. It is easy to test this hypothesis: do H1B holders have a much higher rate of changing jobs after obtaining visa than their peers.
Come on now. You can't actually believe this is how it works in real life.
http://www.cringely.com/2013/07/18/so-thats-how-h-1b-visa-fr...
>> The EPI study, written by University of California-Davis computer science professor Norman Matloff, compared American-born college graduates holding degrees in computer science and electrical engineering to their foreign-born counterparts.
>> There are, of course, reasons for this that have nothing to do with intelligence. More than 80 percent of H-1B visa holders are approved to be hired at wages below those paid to American workers for comparable positions, according to EPI. And because H-1B workers and green card applicants are locked into jobs with whatever employer sponsors their visa, they have less less leverage to push for raises and promotions. As Matloff puts it, "the worker becomes a de facto indentured servant."
http://www.motherjones.com/mojo/2013/02/h1b-visa-bloomberg-f...
Also:
http://www.motherjones.com/politics/2013/03/microsoft-stem-e...
Now you can argue that postdoc and research positions don't pay well enough to attract local talent[1], but I don't think you should conflate this issue with the typical Indian consultancy companies working their way around H-1B constraints.
[1] Which seems true from what I've seen. I know people at the Berkeley Lawrence Lab who would triple their salaries working in the industry.
The school I went to had a decent sized group of students from China, and all but one of them is back there, despite wanting a job in the US.
To be consistent then if we are nationalistic, we should either strive to keep the graduates here, or not allow them in our universities at all. Conversely if we are globalistic, then we should be fine with immigrants working here.
University is not 'tranining'.
A STEM degree doesn't really prepare you for much.
At a startup, you have developers, product, product marketing, marketing, finance, ops, HR.
Most devs are needed for their practical skills: experience in programming, getting through nightmare builds, tons of APS, grasp of server technologies etc. etc..
Most devs are not dealing with highly scientific or 'researchy' things.
I think STEM grads are often much, much better placed to do Dev, and if they take the time to learn the skills, I'll take an Eng/Science grad usually over others because there are a bunch of other intangible benefits, but smarty pants arts major could easily do the work if they were motivated, and sometimes that has advantage today given focus on user experience which is sometimes a little too intangible for some to grasp well.
Depending on location, there may be mechanisms to preferentially hire a postdoc on as non-tenured faculty. It became more common in biomedical sciences after the recession when grad students stayed on for "postdocs" with their advisors when they couldn't find a job, and were then aged out of the postdoc.
https://www.uscis.gov/sites/default/files/USCIS/Resources/Re...
So clearly there are lots of people working outside the cap. The approvals listed there would indicate the maximum possible number, I think you'd have to combine 3 years of reports to get an estimate of the total population.
Do you think these numbers affect the STEM shortage / surplus arguments?
I knew a phd who did a couple of post docs on a J1 for a lot less than the average salary given in the report there. Just an anecdote, but it points away from H-1Bs competing for science jobs.
I found the paper to be more detailed and have a higher signal:noise ratio.
For IT, a master CS degrees (+5 years after highschool) have still not proven to be better than 2 years of apprenticeship. (Sackman/grant 197x)
The global cost of not working (wasted potential outcome hence taxe) is ~ 10% of the potential cumulated income a nation could have.
Then, STEM is also used as a regulative barrier to some jobs hence a de facto limitation on competition of workforce to access a market, thus diminishing the competition.
Least but not last, the access to the data of employability per diploma is not accessible to future graduated creating an opaque market which access is based on your capacity to either be born rich or to have a crippling loan. It is then creating a market where some workers because of their vulnerability are in poor position for negociating their earnings (thus diminishing the overall potential wages of all workers).
And, least, the non reproducibility of scientific experiments yielded by STEM education is growing up, being a clear signal of a «cheating» that arise when stakes are too high in a competition.
I would say there is neither a STEM crisis or surplus, but bank and university are clearly fueling a «diploma bubble».
So basically most of my «scholarship» was about learning more than one way to do the same thing other and other again in up to 4 distinct notations (Einstein with 4D vectors), Nabla operators, quaternions ...)
Whereas working in labs was a total different story. It was basically «telescope building».
I learned computer programming for physicists and to be honest most of CS are pretty clueless about memory, and how to make flow of data kick computers in the balls.
So, I really think that even «hard» science as an heavy load of «apprenticeship» and that scholarship have an heavy load of fat vs muscle.
If we «optimized» training not to achieve a social status based on how much more years someone studied but based on what is required, the whole society would be improved.
So I stick to my gun, for me studies and their indirect cost of «non availability» on the market is excessive and every one (except banks and universities making an awesome lot of money out of it) are losing.
There's of course good economic reasoning for this viewpoint. It forces job candidates to learn skills on their own dime, and companies don't end up training someone who then gets a higher paying job elsewhere.
This in turn is a consequence of the modern view that people only spend a few years at any one job, rather than decades. This encourages employers to see employees as a resource to exploit, rather than to nurture.
There's two major tech companies who do this locally, and they have some of the most loyal employees I've seen anywhere.
“The STEM labor market is heterogeneous. There are both shortages and surpluses of STEM workers, depending on the particular job market segment.”
...is something that somebody actually paid to hear/read. Whatever happened to common sense?
I realize this is pulling things out of context, but in what context exactly is this mega-obvious statement contributing anything at all to any discussion?
It's an honest question and not an attempt at trolling.
I make 2 to 3 times a person on H1b makes. I have close to 100 examples. I am good but not three times good.
H1B is the modern day indentured labor with some American niceties to it. In all honesty as an immigrant I will tell you America treats its immigrants best than the rest of the world. So don't be surprised if there is a never ending line of people wanting to come to america.
As for the issue at hand H1B for surely suppresses salary for americans any spin on it is just spin. It does not give americans who want to change their career into IT options. It makes it harder for anyone to change jobs. Newcomers have a lot of competition with people some with real resumes and mostly fake ones. As you reach late 40's it is impossible to find any STEM jobs since no one is willing to hire you. This holds good for naturalized citizens and locals.
Since most companies who hire H1B know that the H1B will not leave them and go for a minimum of 10 years they happily sponsor. This is due to the fact that a green card roughly takes anywhere from 10 to 12 years. Cheap labor is always good for business.
The government does not care since America needs people to keep growing so they are fine with it. Legal immigration is better and taxable than illegal. Our congress is business friendly so they don't care.
This process will continue till the point where in the salaries of STEM jobs will be so low that most of us will prefer flipping burgers. At that point things might balance out since most of the americans will be looking into other fields other than STEM so no one around to complain.
Also ask the question differently when there is a serious shortage of doctors in all parts of US especially the smaller cities and towns why aren't they bringing in more H1B's there. What is stopping that from happening? This is the STEM curse.
Suggestion for all average americans there is still time try and switch to something where u don't have to compete with H1b you already have a 10 year head start. Here is an idea start a business hire H1B's.