This is exactly what we've been thinking about for the last year while we've been building http://skillsapp.com. Exactly.
Love it when lucid, articulate writers take my mess of a brain and put the right words to it.
Love it when lucid, articulate writers take my mess of a brain and put the right words to it.
* Most of the really good devs I know and hire do not bother with easy metrics like number of tweets, followers, or StackOverflow points. They are perfect to paste together on summary reports and look "complete," but their absolute values or essence don't mean anything.
* The high numbers on public sites seem to be for two classes of devs -- celebrities, who you already know are celebrities -- and people with way too much time on their hands (if they don't work for an open source company, which most people don't). Most people have average values. If someone is on LinkedIn, I'm more interested to see if they have X hundred followers per years in industry up to some maximum lifetime value. I generally ignore recommendations.
* See http://news.ycombinator.com/item?id=3771286 for the discussion about what the number of points actually measures on SO, for example. It's a great site, but it's struggling to handle both breadth and depth of material well. High numbers does not necessarily correlate with expertise, just dedication. Maybe I want that skillset for a customer service rep, but I haven't yet wanted it for an engineer.
* I have shipped 20M loc in my career in 32 regions, and none of it is available on github or sourceforge. Most engineers are like that -- again, the public numbers are biased to certain individuals. Maybe there is a way for candidates to quantify the projects they have worked on?
* What I would love is a way to automatically assess FizzBuzz on candidates. I know there are some sites like interviewstreet.com that try to do this, but they don't yet have the critical mass like linkedin. And I fear, the moment they get critical mass then they'll become a new SAT or other standardized test that becomes too easy to game.
We don't think number of tweets or followers is a relevant means of assessment at all—quite the opposite. Ashton Kutcher is not an interesting candidate for a rails job, and that should be reflected in our report for him (were he ever to apply for such a position.)
We obviously gather a lot of metrics (hey, storage is cheap) but the ones we actually use in our scoring mix right now are tied to two simple concepts: does this person care about their craft, and do they enjoy any amount of professional respect for what they do.
We aren't offering shoe size; what you need in early-stage hiring decisions is an indicator: something more fast than perfect. Resumes suck at this because everybody is trained to write sterile, "professional" resumes stuffed to the brim with keywords. We can tell a much more interesting story—today!—with the kinds of data that most developers in our industry are already sharing. SO reputation isn't a story, it's a data point. What tech people ask questions about versus what tech they answer questions about reveals something more interesting. They know Python very well but seem to be exploring JavaScript? Interesting. Maybe a hybrid in the making.
Skills isn't for everyone and every industry; it's for one industry and a very specific breed of candidates that has this kind of publicly-visible track record. We think there are enough companies that are hiring in this space that there's a business to be built here now, and the Internet trends towards more public data. Do we really think that we'll still be using resumes in ten years to assess whether to spend more time on engineering candidate X?
We've thought about automated coding assessments but for where we are right now, it doesn't feel like the right direction to swim in to get to viability. Any kind of exam this early in the hiring process will sour the best candidates on applying. If you are a solid developer with a good track record, chances are that you wouldn't want to work with a company that can't figure that out about you with a brief googling. Anybody using that sort of tool to replace resumes is locking themselves out of the top talent.
Finally,mwe've definitely been thinking a lot recently about moving to a model without any kind of scores—just visualizations of data that help you figure out who a person is, maybe get a little bit of other personality in a way that resumes will never give you, and help you make that quick move-forward-or-not decision. We're not sure if scores really contribute something to that, and we'd love to hear what you think.
We're still trying to find the best approach to this, but in essence, our mission is making sense of this data. Some of this will have nothing to do with popularity - say, we might give you a concise overview of someone's code or professional thoughts. Other metrics will use social activity only as a rough validation. For example, if someone has 20 watchers on GitHub, they're definitely not a celebrity, but they have crossed a very meaningful threshold of "someone cares about this person's work".
Does that sound like a good direction to you?
If all you care about is that subset of developers, I'm sure your approach is fine.
Perhaps speaking more practically, the market for developers is segmented. Not all companies want to hire all developers, and some companies specifically look for developers who do take their work online. It's those companies we want to help right now.
Take us for example. We just hired Yuval Adam (http://y3xz.com/) for a project, and his online activity had a huge part in raising our interest.
Check out his site. It's essentially a collection of links to professionally relevant online content. Sure, these are no guarantees. If you were hiring a developer though, wouldn't your interest be raised?
Want to post your personal hacking online under your real name? Nope, sorry--"everything you do belongs to the company, and we can't take the chance that you might accidentally post some uber-valuable trade secret." So rather than risk losing the job, you just find some other, not-publicly-visible way to spend your time.
Maybe this group of developers isn't viewed as interesting to many employers, and if that's the case my argument isn't worth much.
Our goal is to look at any kind of professional activity online, not only code. What do you think?
I don't really get into the social media aspect of development even when I'm allowed to contribute to open projects, so I doubt I'd be very visible (or very attractive if I were) via any social media outlets. Whether I'm the norm or not, I don't know.
Edited to add: By the way, I do appreciate that somebody's out there trying to help developers be more easily visible for doing such things, so don't take my comments the wrong way. :)
I'm not on GitHub because I code for my employers. That work doesn't get put out there like that.
I don't tweet much at all about the languages I code in or cool bits of code I just wrote or any of that. I'm one of those weirdos that tweets pictures of my lunch and funny crap my 4yr old says.
I'm "on" StackOverflow since I have a log in... but I rarely comment on stuff and have never posted a question. I probably should I suppose but I don't.
I don't think my candidate brief on Skills would be a very accurate indicator of my actual skills.
(1) It's not our goal to make candidates like you look bad, not at all. When candidates have professional stuff online, we think employers should be interested in getting a good view of it. That's all we try to do. We wouldn't give you a bad score - we'd give you no score at all. Essentially, we're telling employers: "we can't know anything about this candidate, you should look at other stuff".
(2) I checked out your HN profile, and you have a ton of comments up. So, you do interact professionally online, at least in some way. I'm curious to hear your thoughts - say you applied for a job, and we did our best (algorithmic) job to make your HN footprint interesting to employers - what would that do for your attractiveness as a candidate?
To be quite honest, if I found a job post that wanted me to go fill out a profile on Skills I would probably not do it unless I was getting desperate or I really wanted to work for that particular company. I would move on to the next job post.
One thing to consider: I read that you're geared to employers at the moment and that they would be funneling candidates to the site as a way to apply. I would be more inclined to fill something out if it was just me doing it as a way to make a summary of me disconnected from any particular job app. I don't know how your system works right now, but I would be hesitant to just fill in the info and hope the results are in my favor. I'd want to see my brief and have an opportunity to adjust it before letting employers see it. But I imagine that dilutes the meaningfulness of it. Obviously if a person entered all that info and found their Twitter presence is not as good as they thought it would be, they would just remove that. But you can probably see where I'm coming from.
There are many using social channels very rarely or in read-only mode. They don't because they just don't have time to tweet or their project is just confidential to be public on Github. Rather they work 12 months non-stop and quietly on a project and at the end of this period they raise with their work 2 million or got millions of user or whatever. And THAT are the relevant achievements and signals a recruiter has to identify—real achievements and no social streams full of "buzzword spam and unverified claims" (using your own words).
When you say "there's so much noise in social channels and generally you find many people there who look rather for attention than real long-sighted achievements and it's hard to recognize the real gems in that noise.", could that not make room for a company that aims to do just that - recognize the real gems?
Right now, you sign up, creat one or more positions. Each has a unique landing URL, and anyplace you'd normally publicize "send us your resumes," put that link—job boards, advertisements, recruiters, etc.
Candidates go through a very, very (sub-five-minute) short form to collect their personal contact details, their recant social networks and a 500-char-max tell-us-about-yourself and why-are-you-a-good-fit-for-this-job. You get a brief, and the rest is up to you.
We don't publish the details of the algorithm but in effects it would be fairly difficult to game. Either you have a solid history of consuming and contributing relevant content or you don't. If you are able to manufacture that fraudulently, well, chances are that you're somebody interesting to talk to.
We're thinking about allowing some form of "self indexing" but we aren't there yet.
Thanks!
Our mission it to extract the most meaningful data from someone's professional activity online, and we're currently exploring how to best do that. If you're a job seeker, it's our goal to show employers the best version of you.
Question: do you think your current activity online can make you look good to employers? If so, what would you most like to show them?
Thanks!
Twitter tags are based on lists, but with some amount of tokenization and stemming so we count "djangonauts" and "Django" as contributing towards the same tag.