There is no talent shortage
apaitch.com
apaitch.com
From what I understand it's due to two things-- at 34, I'm older than others going for junior positions. I spent a decade abroad after graduation and have only a year of experience at a tech start-up in Asia. I can kind of understand the pursuit of someone who already knows what they're doing, but on the other hand everyone has to start learning somewhere. Sure my bohemian background and experience starting and running a non-tech business and learning multiple foreign languages doesn't translate directly into usable skills here, but I would have thought that it would have been a strong signal I'm capable of working hard and learning quickly.
Not to overstate things, I am getting a lot of interest and a few interviews and even repeat interviews. It's just that things are moving very slowly, and I'm not getting a great return on the time I put into it. Also, SF is a very expensive city to live in off of savings earned in a developing country!
I think the shortage isn't so much general talent; it's a shortage of people who already have X skill a given business wants right now.
If you want to find work, make yourself easiest to get a hold of. ;)
Also, start hacking up a portfolio.
The barrier to entry for programming is very low. Anyone can get started and build things. They can put their code up on github and start displaying what they've learned. They need to keep learning and challenging themselves though.
The problem is that there isn't a lot of demand for "smart" or "talented" people who can't contribute quickly. Someone who writes bad code and adds bugs to production software is a drag on everyone else's productivity. In most companies there just isn't the time or resources necessary to teach basic programming skills.
You probably don't need to know skill X to land a job, but you need to show that you can learn it quickly. If you can learn it quickly, then start learning it on your own. Most companies in SF don't care how old you are, they care that you can be productive quickly.
Also, SF is expensive, and if you are staying somewhere expensive in the city while trying to find a job, you're doing it wrong. Move out to Oakland/Berkeley or even Hayward/Fremont, one of the other cheaper places and stretch out your money til you find a job... Then be prepared to look for a place to live, which is harder than finding a job in my experience... Especially if you are out of money by the time you get a job and don't have enough for the deposit of wherever you live.
Be prepared for things to take a long time, and with that in mind try to stretch out your money.
(the exception being for VP/COO at larger companies where a board would need to sign off)
I've got a $500/month place in Chinatown, and really haven't found anything near that price with English speaking landlords. It's not to the point of missing rent or anything that dramatic yet, but I have taken on some $11/hr writing and translation work via various freelancing sites as well as some similarly priced work on a rails project. It's the surest way I see of surviving long enough to beat the chicken and egg problem of getting relevant experience.
I've been actively seeking something paid at a higher rate using platforms I've already worked on (iOS or JS/CSS), but it's going slowly. Translation is good for helping me continue to use and strengthen my 2nd (or 3rd) language. The rails work is priced as it is because I've never used rails before and I'm spending at least half the time reading docs. It's not a good long-term money-making solution, but I am learning and I'm adding something to my portfolio.
What would someone with good judgement do if their funds were running out and finding a real gig was clearly in sight but moving slowly? Missing rent would bring all kinds of pain I don't want to deal with and wreak my credit.
From all that I have heard and read, do NOT move to Oakland.
"From what I understand it's due to two things-- at 34, I'm older than others going for junior positions."
I hope any HR people who have had your resume cross their desk are sweating at the thought of age discrimination scrutiny.
It may be there's a shortage of HR talent, not tech talent.
Did Glinda flit in from Oz and wave her magic wand, correcting the past two generations' worth of age discrimination problems in the Valley? Did the EEOC's jack-booted thugs suddenly start raiding offices last week and I didn't read about it in the Murky News?
>It may be there's a shortage of HR talent, not tech talent.
But they not only have certificates, they have college degrees in HR now! A shortage? That's unpossible.
One thing I'll add is that, at least for me, the problem of employers only looking for people with X skill right now is not as bad in SF as it is elsewhere. I'm in Washington, DC and the only interest I've gotten has been from companies in the Bay area.
Of course, those aren't super exciting startup-like jobs, but they are places where you'll get a bunch of experience.
For the contractors I've been talking to, when I tell them that I write python when I can, PHP when I'm patching stuff I wrote a few years ago, and javascript when it needs to run client-side, and that I spend my days maintaining systems running Ubuntu server I get a response something like "yeah... that's not really what we're looking for.
And maybe I've been too quick to dismiss working in a Windows/ASP/VB shop. Those folks have challenging problems to solve too, and reasons for using the tools they use. Thanks for reminding me of that.
See books: Outliers, The Talent Code
When your developers cost $75 an hour including overheads, paying people for those 10,000 hours is an extremely costly proposition.
Everyone wants to hire master engineers, but it's not like master engineers are available for hire.
For programmers, most of the initial chunk of that time is through hobby programming or through formal training at a college or the like. That's 3000 hours that the company didn't pay for. That's probably the experience most programmers walk in to any company as a Jr Developer, and they are paid accordingly. Nobody is being offered $150k/yr with that level of experience. Maybe $30-$40 depending on where they are working, and then are promoted and given raises above that.
Of course these things are company-dependent, but I'm a comparatively junior developer and for purposes of internal costing my employer costs my time at $75 an hour.
In all seriousness, this is ridiculous, of course natural talent exists, what gladwell et al are really getting at to me is that it's not enough to have natural talent and execute once. That isn't how success works in the real world, even if it outwardly appears that way. I have Los of friends who are artists and I moved cities last year from one where the local ar school was very practically focused to one where the prominent art school is obsessed with concept and object art. The art and quality of artists in the new city is laughable, apart from those who have been trained in and practiced their craft elsewhere or in their own time however. It's an inescapable fact that if you want to be good at something you need to practice and fail.
Well then, what is natural talent, how do you objectify and measure it in a person, if it so obviously exists?
Success is best correlated with time invested in skill development. Natural talent is bullshit, it's like attributing height to a basketball player's success.
My point was, skill and hours practiced are far better predictors of success than a voodoo attribute like natural talent. In the above case, I tried to use height as an example of a "natural ability" person could be advantaged in.
There are some areas where I am quite frankly talented. Once I get Perl coding on accounting software, there is very little that can stop me and quite frankly I can outperform nearly any programmer I have ever worked with.
On the other hand, there are also things which I cannot do very well no matter how hard I try. When I was in college for example I used to be able to help struggling students in areas like analytical geometry understand the problems and go from getting C's to getting A's. But I couldn't get above a C myself.
I don't know why you can't score A's at math though. It makes no sense to me as to why you can help others get good grades but can't get good grades yourself. It probably means you lack the hours applying through repetitive practice.
Similarly in chemistry lab, I could help anyone else but I would invariably run all the right tests on all the wrong samples. This made me a great lab partner but a disaster on my own.
As I have gotten older I have learned coping mechanisms but some things I can't do and I pass those onto others.
At the same time I have always been top of my class in many things, while putting in quite a bit less effort than average. These ranged from history to some of the sciences, to some cases of math (calculus, some algebra).
As far as I can tell, pretty much everybody graduating that wants a job gets a job. Naturally, not everybody goes to work for Google or Facebook, but this includes people even worse than the presumably hypothetical "John".
Also, people like "Norman" are just as likely to get snapped up by Google and then still work in relative obscurity. Well-paid, enjoyable relative obscurity with great benefits and a great culture, but relative obscurity nonetheless.
Really, the talent shortage works like this: you, as a company, would love a "Norman". You'd be happy with a "John". You'd probably be content with anybody who can program. Instead you get "Barry" who can't write a FizzBuzz program in any language. All this puts upward pressure on programmer salaries and benefits (good) and motivates recruiters to spam me about openings for senior Java developers (bad).
Also, a lot of companies do claim to want very good engineers. But I've found this to be empty rhetoric as often as not: for every company claiming that and actually having a stringent interview process and difficult technical problems there's a company willing to accept anybody to work on their CRUD app but wants their ad to sound cool.
Anyhow: there is some shortage of programmers of any but minimal competence. That is what "talent shortage" actually means, and, as far as I can tell, it's actually accurate.
Not that it's wrong to evaluate based on that criteria. However, for people like the hypothetical John it's quite hard to keep up with a good CS program and manage to do impressive things on the side. So you end up with people with strong potential, much of it not yet actualized, who spend the first years of their careers in a job of questionable usefulness (excel macros?). I think there are more of those people than commonly assumed, and I do think they can contribute well in much better places. They may be slow at first, but give them 6 months and they will be strong members of the team. Granted, a startup may not have 6 months to give, but once you've got some resources under your belt a green-but-talented engineer can be a great investment.
I don't have a lot of advice besides "practice".
Well, I do have some. Don't be afraid to open up the tool's source and read it. Try to design the tool yourself and you'll get a better sense of how it's likely to do things.
But mostly, practice.
Anecdote: I like to think I can write fizzbuzz in a couple languages.
I have actually avoided HR. I've been too lazy to send out resumes, and I only really want to work at small startups (at least for now). All of the interesting offers I got had some concrete spark independent of my education: for example, I got one after doing well at a hackathon and another from an HN post (but largely thanks to my knowing Haskell and having some interesting projects on GitHub).
I think that if you don't mind not working for an established company, you will have to worry much less about your education (but, probably, more about your actual skills). Of course there are good startups and bad startups, but I've found the good ones to be very good not just from a technical standpoint but also from an employment standpoint.
However I ran into a bunch companies that did not know how to hire and I also found that the more successful the company was the better they were at the process, defiantly a correlation there.
I think our industry sets a high bar for "minimal competence". It isn't fizzbuzz. I went to an interview where I was asked to write, on the whiteboard, a recursive solution to the problem of printing all possible permutations of a string. I blew it, and of course I went home and worked on it a bit and solved it (the way I usually program, try it a bit, run it, modify, test, iterate), and I emailed them my solution, but that's how it goes: no offer. I want to be very clear about this, I am not complaining. This company has every right to decide what their interview process will be. But of course it irritates me to no end to hear about how there's a shortage of programmers beyond "minimal competence", especially when the people doing the hiring what to hire whip-smart engineers at roughly half the rate a top MBA or law grad would command.
I've had such discussions on this board with people about the the concept of a "shortage" at the "market rate" when everyone is experiencing hiring troubles at this rate. I think it's an absurd notion - if demand clearly exceeds supply, this is not evidence of a shortage, it is evidence that salaries need to rise to reach an equilibrium. All I can say is that some very intelligent and rational people simply disagree with me on this.
As a personal anecdote - I was in grad school at Berkeley during the first dot com boom, when industries were claiming a severe shortate of engineering graduates. I saw reports of average starting salaries out of the MBA program, the JD program, and the various MS/PhD engineering grads... not a single engineering degree program, even PhDs in CS, were at the MBA or JD level. Why is there so much resistance to paying engineers what it will actually take to get more top people into the field?
But Biglaw/MBA Consulting pay so much because the firms are oligopolies with gentlemen's agreements not to compete on price.
That's not going to happen in a field with mesurable results.
While there does seem to be some basis for questioning the data, I do think that the employment reports support the notion that the average starting salary for someone out of a "Top 5" law school greatly exceeds the pay for an MS or PhD grad of a top engineering school.
Is this something you'd disagree with? - ie., do you think that the job placement reports from the various grad schools at UC Berkeley and other top institutions are so unreliable that we can't draw general conclusions from the numbers?
I also completely agree with you about biglaw and consulting. Law actually does resemble a cartel in many ways, and oligopolies certainly exist in consulting and drive up pay. But to me, that's beside the point. If smart people can make gobs of money in other fields (never mind why, the important thing is that they can), why should we be surprised that they won't become programmers for 80K a year?
Note: I'm not talking about on-the-job training from someone who doesn't know what a computer is to a master programmer. I'm talking about allowing people who are less immediately capable into the workplace and mentoring/growing them into strong contributors.
I'll admit I haven't seen a lot of explicit on-the-job training (besides that dealing with internal technologies), but I have seen a lot of people who's first job is less challenging than their later ones, who use it as a growing experience and a stepstone.
The real question is whether a large number of programmers don't reach their full potential or do so significantly later than they would have with proper mentoring. A difficult thing to find evidence for either way.
Just look at the difference between grads of those school, who basically get money thrown at them, and the rest of graduates, who have a hard time finding a job at all.
Honestly, the idea that someone can make it through the CS program at UIUC, and end up shunned because he/she didn't graduate form MIT, is a perfect example of what's wrong with hiring practices these days.
Is it the case that a CS degree is somehow considered second rate if it is not from one of these universities? Is it a prestige thing or do all the other universities suck for CS?
Obviously we have similar in the UK to an extent with Oxford/Cambridge etc but CS degrees from most universities are well enough respected.
I've done some of the online classes from these 3 universities as well as watched youtube videos from some lectures. For the most part they don't strike me as being especially more difficult than what I did for my CS undergrad (in most cases the difficulty level seems about the same).
Funny.
The part of me that flatters myself tends to think of geeks as more logical and individualistic. And less likely to follow a clique. And less likely to be biased in such silly ways like preferring someone just because they came from your university.
But reality keeps reminding me that nope, intellectuals can be just as stupidly cliquey and biased as anyone.
[1]: http://news.ycombinator.com/item?id=4140048
Largely though "Norman" is more attractive than "John" to businesses because he displays strong signals both in talent and work ethic. "John" might be a hard worker and have talent himself, but because he has no obvious projects or other public displays of talent his signal to employers is weak. What "John" needs is something that sends a comparatively strong signal. In a way, this sounds like a call to action to figure out how programmers can build a sufficiently strong signal for employers to see. How do we separate the signal from the noise?
Interestingly, this signalling problem arises with employers as well. Look how many articles are posted here about "How to attract the best talent."
A number of posters in this thread assert that anyone who can write code competently[0] can get a job in this economy. I personally, as with the OP, don't see that. I see two broad groupings of companies[1]. The first group only wants Normans or near-Normans, and complain that there are not enough of them to fill their slots. The second group is happy with Johns, but drops a laundry list of (sometimes conflicting) skills and experience requirements that filter out almost all of the Johns, and then complains that there are not enough of them to fill their slots. I have also encountered an overlapping subgroup of companies willing to compromise somewhat for local candidates.
Unfortunately its difficult to move beyond anecdotes on this matter. I have some that support the OP's point, many here have some that refute it. It would be really interesting to see distrbutions of these anecdotes by geography, experience level, and undergrad program, though.
[0] "Competently" goes beyond writing FizzBuzz in this case. [1] I am not asserting that every company falls into one or the other; my current one does not, though my previous one does. I'm asserting that most do.
I did a bit of casual freelance work building basic PHP/HTML websites and fixing/building PCs before I started college, but never anything I would have dreamed of incorporating a company for. In fact I'm not sure how many jurisdictions would even let one start a company at that age?
In fact I would be surprised if someone with that level of accomplishment so young would even consider taking any job with any company, more likely they are already putting together an MVP of the thing that will replace google.
But I do know a person who's been programming since ~12, wrote a 3D game in C++ at ~16 (multiplayer one, thousands of players), and wrote his own OS, and happily employed.
Shame all I can find is that youtube video and no download links etc, that game looks utterly insane and quite fun.
Technically it's only really a 2D game though , still very impressive for a 16 year old.
I'm not sure what I'm supposed to gather from this John/Norman comparison. If John has the same amount of natural talent as Norman, after a few years of full-time professional development work he's likely to be just as valuable as Norman. There's plenty of time to become really competent and valuable even if you start very late (like in your 30's or 40's, which I assume classifies as "freaking ancient" in many companies' eyes). I don't see how one person having some imagined head start figures into whether there's a shortage or not; but then, I don't live in the valley, so perhaps this is a cultural thing there.
I do agree, though, that there's a lot of companies out there that seem to want the top 1% (all the while offering bottom 25% rates).
Is a physicist just as valuable after a few years of doing research as a full tenured professor with 30 years of research? After a few years of full-time professional development work John will be talented at working with a very, very limited set of tools that his company uses and developed the ability to solve a very small set of problems. This means he is perhaps as valuable as Norman to one single company and almost not valuable at all to most other companies.
I think you're wrong, there. Building software is, 99% of the time, like installing carpet.
> Is a physicist just as valuable after a few years of doing research as a full tenured professor with 30 years of research?
Terrible analogy. There are grad students who are more valuable than professors on the edge of the retirement. Comparing software development to academia is a specious argument.
In my opinion, one thing that can happen is the following:
Norman gets easily a job at a great company where he'll continue to improve his skills amongst other 1% programmers; while John takes a job that isn't exactly what he wants, in a so-so unknown company, where he isn't challenged as much and isn't learning as fast.
After a few years, Norman changes job easily because he has worked for Google/Facebook/whatever and that acts as vesting to move him along quickly in the interview process at the new hot companies of the time. John has yet again a harder time because having worked for that unknown company doesn't give any special credit. He might have had to take some jobs that were not exactly what he wanted (e.g. support, QA) and that is questioned when he has interviews. Even with his additional experience, he still has to go the extra mile to pass the ex-Google, Facebook, etc.
Lather, rinse, repeat.
Of course, I'm forcing the trait, but I think this scenario happens fairly frequently. Also, it's certainly possible for John to break the cycle by working on his own things and putting himself "out there", or by just "waiting" a few years until things even out a bit; however it feels that the extra efforts required are not reflecting the initial difference between Norman and John.
That said I'm now actively building a portfolio by working nights on real products and leveraging what experience I do have as heavily as I can. I also try to present how quickly I picked up the skills I do have in things like Ruby, coffee/node and all that 'sexy' stuff to show that I can pick up anything anyone throws at me really quickly.
Hopefully by the end of the year I'll be where I want to be.
as proven by 1) the fact that the percentage of of the general population who have an aptitude for "brainy things" in general is pretty much the exact same as it was 20 years ago and 2) the percentage of companies with a need for said people is growing exponentially as software eats the world.
basically, all of the new-need companies from point 2 are bottom-feeding all of the people from point 1 which applies upward pressure on the talent pool.
because a talented accountant-turned-programmer still leaves a hole that needs to be filled.
feelin' me?
m3mnoch.
This article is arguing that the current situation is because companies wont employ below a minimal technical competence.
New, brand new, out of college only done programming in college programmers can expect massive starting salaries, the best equipment, and probably lunch and accommodation with a strong chance that they'll get a higher salary the year after.
Few companies require you to actually have used their technology or their architecture, if you have you might have already priced your way in to a job above the one been offered.
If the industry contracts as it surely will programmers are going to be scratching their heads at articles like this.
I've got a great job now programming all sorts of things, but it's not purely CS, there's a lot of other stuff there too. Perhaps more BS-CS degree earners should look into industrial applications, they're not ashamed to hire people from small colleges.
I certainly think that the "talent shortage" could be reduced by making computer science mandatory in high school, or at least on equal footing with other sciences such as biology or physics.
I didn't. I loved playing around with computers. But no one in my family was technical. None of my friends programmed. I literally had no idea what programming was. Most people don't, even those who are otherwise very effective with computers.
Now I've started, much later, and I love it.
This xkcd comic is relevant: http://xkcd.com/519/
I'm sure if someone had shown me what a programming language or a terminal was, I would have started 10 years ago. Instead I learned about other things.
I heard about Python / C++ / etc, but didn't try "diving in" until college with CS1. Basically, IT knowledge instead of CS knowledge.
I can't help but feel that anyone that is <22 right now is getting a radically different introduction to hacking than those who are >=22. The <22 grew up on GUI in the Windows monopoly, and probably never needed to dig in to computers to get things "working" as much as the elders did, which would actually introduce them to CS much earlier.
I'd say I have a decent amount of talent, and spend probably 75% of my waking hours increasing said talent, but I still feel sort of late. I often lie awake wondering if I'm already behind in my career.
* you can replace code with almost any other employable skill
"Talent shortage" usually refers to people who fall in a).
Every company claims to want to hire only a) but most companies need b) or c). Very early stage startups ideally need a), but they don't always manage to hire people in that category. They can temporarily bypass this requirement by hiring people in b) and using frameworks like (Rails, Django, etc) built by other people who fall in a) until they get to a point where a) types will be willing to join.
When people talk about shortage of talent, they usually mean shortage of a) people. This is mostly because two of the three things that people in b) and c) need to become a) are hard to acquire and come with 'experience'(for those who try).
3 things you need to become a):
1. Acquiring a good base - data structures, algorithms, design patterns etc - learning languages/specific skills - interview skills
2. Acquiring good mental models - working with smart programmers - reading code by smart programmers - learning to make tradeoffs - actually building stuff
3. Acquiring reputation - Building stuff - Talking about stuff you build - Having other people talk about you - Collecting badges that other people recognize i.e. working at well known startups/companies, winning programming competitions, going to Berkeley, etc.
Most people manage to succeed in 1) eventually. 1) is also where most college/other training programs focus. But, 2) and 3) are much harder to acquire.
If a company is big enough they will assign an a) to mentor someone who is a b) or c), but this is often not viable for startups (and, imho, slightly unreasonable to expect from a lot of startups).
If we figure out a way for b) and c) people to learn 2), we can solve a lot of the talent shortage. But, this requires either a b) or c) person to spend time on their own looking at work by a) people (not always possible and extremely time consuming), or an a) person mentoring the b)/c) person outside companies (100x faster, imho).
Until we build social systems that enable b) and c) to transition to a) easily, the alarms for 'talent shortage' will continue to ring.
Caveats: 1. A lot of jobs only need b) or c). Employers could adjust expectations. 2. Luck is a huge factor. You don't actually have to be a) to be hired or considered a) because perception(3) plays a big role.
What you need is someone for a) is someone who is both a software engineer (i.e. thinks hard about design) and is a coder/developer. The amount of experience doesn't really matter though experience is an important guide. What really matters is attitude. How does the programmer approach design questions? Is the individual someone who just wants to code? Or is the individual one who is always addressing his or her code for maintainability and pursuing perfect in that regard?
I have seen projects built by people who coded for a dozen years and never got to the point of actually thinking hard about design and maintainability. I have even been stuck in the nightmarish position of maintaining such code (and of course deciding to refactor with a chain saw!).
Again a lot of it is a question of priorities and, I hate to say it, ego. One has to both have an ego enough to question prevailing wisdom but have a small enough one to understand that one's coding style is never perfect. Such individuals are rare, in my experience.
Attitude is exactly what companies look for when they hire people with less experience. Even for more advanced programmers in a sense it is essential because you never stop learning. This is why there is some bias towards younger people as they are supposed to be more willing to learn.
But, as with any behavior, attitude becomes less necessary if a clearly defined system for mentoring/acquiring the necessary mental models is in place. Someone who graduates from a top school and ends up working at a company where a) types mentor new hires has to try significantly less than someone who graduates from a less well known school and has to figure out some way to learn from a) types.
Btw, the way a lot of the best companies bypass the HR problem is they hire based on employee recommendations. This is another contributing factor to 'talent shortage'. There is a hiring channel at any company that is much better at recognizing potential - existing employees. The high effectiveness and limited reach of this channel adds to perceived 'talent shortage'.
I have worked with a lot of programmers and I find you can basically throw advanced programmers into three groups:
1) Those who think about and figure out how to write maintainable code (these are a small minority btw!)
2) Those who believe that writing maintainable code is a matter of following coding conventions they are comfortable with, and
3) Those who just want to get stuff done.
I think that #3 will only make competent maintenance programmers. You do not want to be stuck maintaining a major app one of them built. #2 will be fine at extending existing frameworks once they become comfortable with them, but #1 are the ones who can create new framework code and make it work.
As for mentoring that's key (and was for my own development as a programmer and software engineer) but many people just never "get it" even with all the mentoring in the world. Either they are not listening or they are not drawing the right messages from it. This is tough because in order to get to #1, you have to be willing to question what your mentor says. If you mechanically follow it, you end up in the second category.
Regardless, it does suck to have to maintain/extend apps written by #3.
You can't say that the person who just goes to class is going to do as good of a job as the person who completes side projects, learns on their own, etc.
The moral of the story I would take from it is that if you want to program make a name for yourself, and do so without regard to your employment.