Job hunting is a matter of Big Data, not how you perform at an interview
theguardian.com
theguardian.com
> The data suggested that the success of teams had much less to do with experience, education, gender balance, or even personality types; it was closely correlated with a single factor: "Does everybody talk to each other?"
> Ideally this talk was in animated short bursts indicating listening, involvement and trust – long speeches generally correlated with unsuccessful outcomes. For creative groups such as drug discovery teams or for traders at financial institutions, say, the other overwhelming factor determining success was: do they also talk to a lot of people outside their group? "What we call 'engagement' and 'exploration' appeared to be about 40% of the explanation of the difference between a low-performing group and a high-performing group across all the studies," Pentland says.
> It was important that a good deal of engagement happened outside formal meetings. From this data, Pentland extrapolates a series of observations on everything from patterns of home-working (not generally a good idea) to office design (open and collegiate) to leadership. "If you create a highly energetic environment where people want to talk to each other right across the organisation then you have pretty much done your job right there."
So true.
It'd worry me that, although this guy says you need to increase behavioural diversity but minimise value diversity, that you're effectively just minimising the pool of potential employees, rather than figuring out ways that a larger pool of people could fit. Or in other words, whether it's Myers-Briggs, Belbin's roles, IQ tests etc, it seems that evaluation tools are trying to quantify the diversity of people, and pick off, with increasing accuracy, the exact archetype that aligns with organisational goals. But if you assume that people are diverse, and that employment roles are diverse, and that there's a large pool of both, would it not be a better idea to focus on quantifying the differences in attributes required for your employment roles, so as to maximise your pool of potential applicants?
I'm willing to bet that someone who's slobbish and lazy and unpleasant could play a valuable role if you could quantify the requirements and goals of positions in your company, for example. And does value alignment matter for all roles in all companies, or is it just a phenomenon arising in the last decade, being an intuitive way to maximise employee investment and increase profit? Why would financial staff need to want to 'change the world' or some bullshit to work at crappy Startup X?
Yeah, but the positions of CEO and HR are usually filled at an early stage...
That hits on one big problem I see with all this: How do you know what the company actually wants? All/Loads of companies claim to want the same thing, and have the same vision of "changing the world". But some companies are run by abusive wankers who want cheap developers who'll be happy working 24/7 for low money. You can tell by the company's actions, not their words. If the only pay peanuts, then they only want cheap labour.
Will this big data be turned around to the company, so employees can see what sort of place the company is?
This sounds like what Glassdoor does. Even small local companies get reviews on it.
In other words, hand-wavy big data aside, this is yet another HR tool.
I really think this informal "who you know" is how many jobs actually happen. Something like speed dating but for employment.
He was arrogant, confrontational, couldn't answer simple questions and was half an hour late - and not apologetic at all. He was just weirdly and inexplicably unpleasant to everyone.
Perhaps he would have been great in some role or would have aced a psychological test .. but I'm glad we did an interview so we could reject him out of hand.
I'm sure there is a place for widespread empirical evaluations .. but ultimately if you are employing someone to work with you there has to be space to say: "Err, No! - I don't care what the computer says".
We are huge believers that this sort of testing is purely a pre-interview step. You get everyone to take it and you save hours and hours of digging through resumes. That's the big money saver.
It strikes me you could easily record the interview and run it past a machine looking for the tempo of the conversation, the tone of voice and so on - and see whether that correlated with employment a year later.
I'd be for much stronger data controls, and that might rule something like that out just as collateral damage. Fair enough, it's not a very strong principle if you violate it the minute something you approve of comes along.
That said, I'm vastly more creeped out by the idea of someone collecting email addresses and looking up linkedin profiles for recruiting, or looking you up on facebook when you apply for a job, than I am with the idea of someone recording an interview. The latter is in a limited context, where you expect to be examined anyway. The former is an intrusion on your private life to one degree or another.
I'm not saying, by the way, that it's an acceptable thing to do behind someone's back - that would be extremely scummy.
I just don't really see the problem with running something past a computer. Would it make a difference to you if there was a typist there and you ran the transcript past the computer? Is it just the emotional ick of it being your voice they have on record?
Edit: Is it the possibility of mass data collection? I can see that being creepy with things like car number-plates because there the difference in speed and ability enables things like continual tracking of individuals day to day lives. But I'm not sure how that would be a concern here.
[1] https://twitter.com/DEVOPS_BORAT/status/288698056470315008
"@DEVOPS_BORAT got heavy duty @dev ops gig available in San Mateo,ca and Seattle area! $130-$140+k doe email me Ryan.Lum@greythorn.com"
does this mean it works?
I replied saying "I may be" and what size was their "big data", as usually I don't see the point in taking it out of a traditional relational database in most cases. I don't think they like my attitude, as I never got a reply.
Unsavory political views? Get blacklisted.
Don't get along with a relative? Get blacklisted.
Indulge a porn habit more than the HR manager likes? Get blacklisted.
I cannot stress enough how dangerous this is.
The algorithm to hire people in this future is a competitive asset of a company and they would spend lots of time & money to narrow down the actual important factors. I highly doubt companies would just all settle on the same algorithm given the incentives to compete.
Also if you are concerned about this info impacting employment, just don't post it online. Not much different than today.
Yes, but you forget the old banking maxim: it is better to fail conventionally than it is to succeed unconventionally. I think it's very likely that hiring managers would pass up the potential increased talent of an employee if that talent came coupled with increased risk that the employee would do something to embarrass the company or do something to make the company look bad.
>Also if you are concerned about this info impacting employment, just don't post it online.
So what do I do when the lack of online information about me in itself becomes a drawback?
- Good analytics (I'll combine the three terms into 'analytics' for the sake of simplicity) requires an understanding of the tools, as well as a significant understanding of statistics so that you know which analysis to pick. But in addition, it requires a lot of creativity (see my examples below) and a significant amount of time to analyze/slice/dice data in a zillion different ways.
- This is a huge opportunity. Much, much bigger than people realize and much bigger than past trends of new technologies like client-server in early nineties or web apps of 4-5 years back. Why? Because it has the power to affect business processes very powerfully.
- Example 1: I spent 10 months working for a $5B shipping company analyzing data from their Marketing department. I combined it with several hundred global data sources. I worked on over 100 hypotheses. At the end of it, I came up three specific actions that their existing customers take about 6 months before going to a competitor. The Marketing department was thrilled. They spent $17 Million coming with a plan to tackle this. It has been a few months since then; and they have not lost a single customer. This is a powerful proprietary competitive weapon for them now.
- Example 2: I analyzed 10 years of power meter reading data for a large utility company. I combined it publicly available data sources of power consumption of major appliances and census data on family composition/wealth for various neighborhoods. I was able to reliably predict the lifestyle of every family, down to whether the person living in the house streamed a movie on Friday evenings and a whole lot more. So the company decided to use this analysis to change their Direct Mailers with very specific, personalized offerings. Their response to the first test mailer sent to 10,000 people? Twenty seven percent!!! They predict that a significant portion of their profits would come from DM's.
What level of sophistication did you have to reach in the shipping company case. Was it, hypothetically speaking, the client starts using more than one other alternative shipper (fairly low sophistication) or was it multi factor stuff at levels of statistical skill that you need a Phd to grasp?
And thank you - these one off comments are why HN is such a valuable place to contribute to.
- When you start, you are taking a leap into total emptiness. You explore a thousand different avenues, most of them are dead ends. Your day consists of massive amount of mental effort to stay focused, to stay sane, and to make good assumptions. Then you change tack/analyses continuously. You keep adding/deleting datasets. This is not theoretical statistics but sometimes you use arcane things; so you definitely need to be very strong at Stats. My personal dream is that one day, when I have time and money, I will use these methods to come up with a Meta- Statistics approach to empirical analysis -- i.e., to use Analytics to predict, based on the problem definition and available data sets, which methods to use.
- You have to have patient clients. Jumping 10 months into a project with absolutely NO guarantee of success is a huge leap of faith, financially speaking; but the rewards can be huge. One day you are still nowhere, and literally the next day, everything clicks, you check your conclusions once-twice-thrice, make a presentation to the client, and, boom!, your project is over.
So how are you planning to go from tenuous projects to repeatable revenue (sorry someone asked me some hard questions today - paying it forward!). I am guessing that a 5bn dollar shipping firm that just stopped multi-million clients walking out the door is getting some serious payback - even if your daily rate was enormous.
So what about things like multiple clients simultaneously (hiring interns), building a capability maturity matrix to help clients increase their data sophistication (and naturally you charge a monthly retainer)
Just interested in knowing where you are taking this.
Also, some of the numbers in the article really make you scratch your head: Achieving more than 95 % accuracy when ranking a large number of student teams in an eight-months long business plan competition, based solely on the results of a simple online questionnaire taken at the beginning of the competition? This just seems too good to be true considering the data sources they have at hand, even assuming that they use the most advanced machine learning in the world.
Of course, if you test your algorithm many times at different competitions you will achieve a perfect or near-perfect prediction accuracy for some of them (by pure chance), which however doesn't mean that you can achieve this kind of accuracy consistently (which is where the business value lies).
My first thought exactly.
It sounds interesting in theory. That is, until I got to asking about how they quantified softer qualities that employers look for, like an applicant's social skills or potential for a client facing role. Apparently, to determine this, they look at the number of "check ins" people do at locations that are not their home city while employed. Their algorithm assumes that the person is traveling for business and is therefore trusted to meet customers.
There are so many assumptions in this one example that it makes me question the integrity of the whole system. An algorithm is only as good as the person designing it. Maybe Evolv really is better than these guys at finding quantitative markers for softer skills, but I remain skeptical.
His response was basically that they are "trusted partner" (quotes because I can't remember the exact term, but that sounds right) of Facebook and so they get all the data somehow. Maybe they pay for it? I am not sure how it would work because it couldn't be anonymized for their service.
Anyway, he basically said that, when you combine the Facebook location based posts and FourSquare checkins (and probably location anchored Tweets too as well as others I am missing), there are so many millions of these happening everyday that some of those people are bound to be qualified for the position a company is hiring for.
...Which brings us back to problematic assumptions. Garbage in, garbage out.
But yes, you end up selecting people that:
1) Have smart-phones
2) Use them, and not have their kids mess with them
3)Use many of these services
4)Are not caught up in the SV non-compete bubble
5)Are good enough nonetheless
6)Are looking
This limits the use of this idea incredibly. Not to mention all the regular Federal issues of race, sex, creed, and orientation. If I were hiring, unless this were incredibly cheap, I'd stay far away from it. It's just plain creepy.
I'm currently working on my CV and there's about as much conflicting advice as to what the 'ideal CV' is as there is about what the ideal diet is. It's somewhat frustrating but at the same time it just shows the amount of chance and variation involved in the whole process.
At least in the industry I'm going for (games / Unity3D for what it's worth), actions & side-projects seem to speak louder than words.
http://www.dilbert.com/dyn/str_strip/000000000/00000000/0000...
I can hardly wait for the day when a baby is born and "garbageman" or "engineer" is stamped on it's head and it becomes futile to argue with empirical truth about "best fit".
Actually, kidding aside I think companies and employees may both benefit from this research if it is applied properly. But if a better form of these tools were available to our current system it would likely produce some very bad effects. I think maybe we have some important decisions about humans and their role in society coming up soon. Because technology never seems to go back in the bottle.
Then folks can take the same test and see if they should run screaming from the interview
In other words: expect a massive increase in job hopping as people find the only possible way to negotiate with this program : quit.
> By morning, he says: "If a customer has thousands of people in similar job types, our system can predict accurately on a given day which individuals are most likely to quit." In response, Evolv then offers employers "what-if types of analysis" by which if they change certain incentives – a bonus, training scheme, change in environment – they can see exactly what effect it is likely to have on a particular person's behaviour. In this way Evolv advertises average reduced employee attrition rates among its clients, who include one fifth of Fortune 100 companies, of up to 15%.
This sounds horrible. It would force employees into quit-to-improve-working conditions dynamics. Constantly interview, at a non-ridiculous rate. If you get offered better conditions, either Evolv will offer you the same at your current position, or you should quit.
Of course that's already mostly true : my advice working for a fortune 100 company that isn't Google or Facebook : prepare to quit after 1 year or less. Regardless of whether you want to stay or not, have a serious discussion with your boss about quitting after 6 months at most.
I wonder if it would defeat the negotiation tactic used by "Evolv" here. If you can call it a tactic, that is.
To get a raise you mean?
Seriously, almost any type of 'optimization' on the half of companies/employers right now is bad news for employees and job seekers. We call it optimization because its nice and clean. But the effect of every optimization is to squeeze out every last bit of productivity out of a given work force as possible.
Honestly, I hope this type of thing is never developed and deployed to the degree to which some of the proponents in the article wish it to. They can paint starry eyed pictures of a future where everyone gets to work the job that fits them the best, but all I can see is a future where everyone who has a job is scared shitless of losing it, since it'll blackmark them forever.
As for it going into production I don't see why it is a bad thing. You could turn the whole thing around and help job seekers find companies where they know they would be happy and productive. After all it is a similar kind of optimization but from a different perspective.
For instance - lets say that I realise I have a problem with turnover that I want to fix, because my costs of replacing staff are too high. To address that I want to spend 1M on retention activities in a year. Should I spend that on additional vacation days, or should I spend that on extra events for the staff, or on extra training opportunities? If I spend that money, will the impact on my turnover costs be positive enough to warrant the spend?
That's the kind of discussions that always pop up. Being able to quantify the impact would make it easier to do the right thing that both benefits employees and the employer.
But let's be real - there are likely more people that jobs that they are willing to do in the US for the short to medium term. As the 'jobless' recovery showed, companies were perfectly able to squeeze out additional 'productivity' (ok, I'll agree that the way we measure market performance isn't really that great, but it's the metrics that we largely agree to play the game by) while cutting labour force, and maybe even per-worker pay, and likely driving down employee satisfaction, other than how glad they were to still have a job.
As long as 'hey, look be glad that you have a job' is a legitimate threat, for the majority of workers, they're really out of luck, cause again, let's be real. The right thing that benefits both employees and the employer is not the same thing as the right thing that benefits both the employees plus the guy you just fired and the employer.
An algorithm can't 'negotiate' with a strike, or other more extreme forms of trade union actions (like burning down the compeditors, shunning scabs, etc.)
You are claiming that the human cannot beat the algorithm, because the algorithm can always adapt. But humans are still better than computers at some games (Arimaa, Havannah, Hex, Go) and worse at others (Chess, Checkers). So your claim needs a lot more justification.
The comparison is worrisome: depending on how good these algorithms get and how zero-sum the competition is, we could be screwed. But we don't know.
If I can get awesome people to work at my company just by offering them a reasonable amount more than at Evilcorp, I'm just gonna do that. If it forces Evilcorp to raise their wages, then that's good for me - EC is spending more to get what they'd get anyway - and if they come work for me then that's good too.
Wage fixing, is a problem, don't get me wrong. However, it's a problem regardless of how efficient you are in hiring.
Funny you should mention the 6 month rule. I've kinda stumbled on it through trial and error but it seems like a pretty good checkpoint for salary discussions.
After 6 months, your employer can more accurate judge your work and it isn't necessary to wait until the "annual" review. It's an opportunity to adjust salary upwards because you've proved your value. Remember, the original negotiated salary was based on pre-job leverage (skills, knowledge, etc) and not how well you exceed in the role. You could prove to be a bigger value-add than was expected.
You don't have to pose it as an ultimatum (Pay me more or I'll quit), because you can also use it as an opportunity to find out how to adjust in order to get the raise at the 1 year mark, which is in another 6 months.
Money isn't a great motivator. Lack of decently accurate compensation is a very good de-motivator. Same goes for a bunch of other things - I dont want to have to think about them, but if they aren't thought about it will annoy and demotivate me. Find a way to script that and it will increase my happiness by decreasing irritants.
The job-hopper stigma isn't about imputed low skill or merit. It's about social status. The person who is presently unemployed has (temporarily) low social status. The person with 5 jobs in 6 years, it is perceived, failed to achieve high social status at any of them.
The problem with humans is that most don't make decisions based on value-add potential, but on social status. They see Harvard on a resume and want to hire that person, to be socially "closer" to Harvard. It's not about whether Harvard graduates are better hires or not; that question is irrelevant.
Job-hopping might seem like it could be a high-status behavior, in that the best people get bored quickly and always have other opportunities, so they don't put up with abuse. After all, the serially fired job hoppers are maybe 1/10 of that set. It's not so, because the people who make hiring and promotion decisions are in corporate in-crowds, and part of being an in-crowd is the necessary assumption that everyone wants to be in an in-crowd. The job hopper may be individually excellent, and it may be that he'd be a 5+ year employee if given high-quality work and colleagues, but all his paper says is that he never stayed long enough to join a corporate in-crowd, and that even if he was invited into one, he made the "wrong" decision to leave it.