Data Scientist: The Sexiest Job of the 21st Century
hbr.org
hbr.org
I wonder if one of the goals of a good Data Scientist is also to be not too accurate, lest the product create an eerie feeling among users! (remember the Target pregnant girl incident?!)
One issue is that from an end users perspective it makes it obvious how much information is being captured about them. While most people are aware that their information is being captured, seeing it plastered all over their facebook feed makes them confront it.
Worse than that are the questions that come with these ads - "Why am I seeing ads for baldness cures?" Is it because I'm a 30+ male, or is it because they have analysed photos I'm tagged in and detected my thinning hair? Sometimes it just feels mean!
This is primarily a challenge of data science working in a marketing environment and doesn't really permeate through all areas of data science, however it is the form of data science that is most visible. Therefore much of data science and the big data we work with gets lumped in with sleazy marketing.
In particular, it's creepy if someone knows something about you and you don't know why.
So if you don't want data mining to be creepy, you have two options:
a) Explain why you know something.
b) Wait until data mining is so commonplace that people take it for granted, and standards of etiquette shift.
Because that's all a "data scientist" is... but without the experience to realize there's already a job title for what they do.
More seriously though, the requirements to be able to hack up a prototype and talk to people are probably what hold back a lot of people who otherwise have the skills to be good "data scientists", or just scientists.
My current employers told me at interview that they had no data, and in the three months I've been there I've been slowly discovering that they have loads of it, unfortunately in multiple incompatible forms and jealously guarded by different departments. It is rather funny, though a little sad that they were essentially drowning in data and didn't realise it.
There is a difference. It isn't a difference in fundamentals so much as it is a difference in focus.
Business Analysts give reports to CEOs about customer segments or the projected amounts of signups. They arn't even close to DS or statisticians.
Statisticians tell you about how a drug reacted with a control group or how likely it is that a population feels a certain way given the results of a survey or trial.
Data scientists harness data. They impact every user on a site. "Watch this video" "Follow this user" (recommendations) or "Silently ignore this user's impact on the algorithms that manage where this piece of content should go" (graph analysis) or "What exactly is in this photo" (object recognition) or "What combination of widgets leads to the maximal amount of engagement" (optimization) or "I have this paper that I really like, show me more that are just like it" (recommendations, document classifications, NLP).
It is different. The focus is on users and what they will do or should do or should see. To call them statisticians leads to much less understanding of the value that DS bring. Put me in a room with an actuary from an insurance company. Neither of us could possibly do each others jobs. Neither of us have the others skill set.
Now, both of us could learn and get up to speed on how the other works, but a sys admin and a web developer could swap roles more easily than an actuary and a DS. Yet nobody is complaining that we call devs and sys admins different titles.
If you ask me, the phrase "data scientist" is recruiter-speak. I have all of the skills required of a "data scientist". I've done the job of a "data scientist". And other than object recognition, I've developed all of the different product features you mention in your comment. You know how I got the skills necessary to do those things? I was trained as a scientist, and there's no such thing as a scientist without data. A person properly trained to analyze data should be able to effectively and fluidly transfer those skills between domains -- otherwise, they're not actually good at it. There's nothing special about internet products that precludes competent people from doing effective data mining on their logs.
I suspect that the real problem here is that "data science" is Internet Hipster for: "someone who has already worked at an internet company, and knows some statistics". Because when it comes right down to it, your average statistician, chemist or physicist is more skilled at data analysis than 99.9% of the "data scientist" types you meet, but they don't easily press the comfort button for hiring managers at consumer internet companies. Why hire the "risky" ex-scientist, when you can hire the guy who claims to be a designer, a software engineer and a statistician?
Eventually you have to distinguish new fields from the old, even if they have a lot of commonalities.
The problem here is that "data scientist" adds no semantic value above and beyond "scientist". A scientist of data, you say? However will we find such exotic creatures!?
Here are the three general problems with submitting print views:
1. For most sites, the print view results in a small font and lines that extend all the way across the page. This makes them hard to read. Sometimes, on a desktop, with a bit of fiddling they can actually be made legible to those of us who are older than 40. On mobile, they are often simply not possible for many of us to read.
This particular site is OK in this regard, as they appear to have actually set the line width and the font size so that it comes out reasonable on the screen. In fact, their print view is quite pleasant to read.
2. The print view often omits comments, sidebar links to related stories, links for sharing, and so on. Some people actually might want to use those.
3. There is often no evident link from the print view back to the normal view. Sometimes you can figure it out by playing with the URL, but sometimes the relationship between the print URL and the normal URL is hard to figure out if all you have is the print URL to work with. Note that the normal page, on the other hand, does generally have a link to the print page, so those who prefer the print page can easily go to it.
For these reasons, in almost all cases the submission should be to the normal page, not the print page. Ideally, the submitter can add a comment that gives the print URL to save time for those who do prefer it.
Note that some sites have an "all on one page" option, that puts the whole thing on one page, but leaves comments, social links, and such. That's the best to use if available.
I think the same applies here.
A data scientist is a fancy way of saying a "statistician who can code (should be required in stats programs now anyhow) and who can communicate effectively"
Based on a few people I've kept in touch with, it seems like it hasn't changed all that much at the undergrad level. The grad level was where the problem sizes and difficulty really forced you to use better tools.
I wonder, do statisticians actually use graphing calculators to do stats?
>So far Linkedin's Friend Suggest is one of the biggest success stories.
I don't agree with this. Google is basically a big data sciences company. 'Data science' may be a new term, but it describes something companies have been doing for decades.
>'Data science' may be a new term, but it describes something companies have been doing for decades.
This is not what most articles say. They actually try to frame it as something "new and sexy".
Looks like a tech company list looking to hire data scientists - essentially a sneaky job advertisement wrapped up in a fluffy HBR (aren't they all?) article written by a consultant who probably wants to get in on the new new thing.
I got one call from a recruiter who thought I was in a different city. Ain't so sexy from where I'm sitting.
I consulted for a client that used those technologies, updated my LinkedIn profile afterwards, and the amount of incoming requests from recruiters and principals has been nothing short of phenomenal. (Anecdotally, 20 InMails in 10 days, of which 14 of them converted into a phone interview with the principal.)
Are there as many data scientists who don't work on Big Data?
There are plenty of data problems out there already warehoused by small-cap and mid-cap firms; I honestly don't see a need to go Web-Scale and all that jazz for its own sake if your use case doesn't need it. There's also shortcuts like sampling to kick the can down the road, but that's another discussion in and of itself.
Or, I guess programmers and engineers could start using the big data tools even though they are not needed. Has anyone ran Hadoop on a single (multi-core) machine for this purpose?
Check out http://www.kdnuggets.com/ for links to large data sets to work on and there are also some on amazon.
Also, yes you can certainly run hadoop on a single instance, but once you get into "real big" sizes you'll need a cluster to demonstrate expertise, be it on your local machines at your house or on a set of VPS or EC2 or whatever.
As a math/stats guy who picked up more programming along the way, I personally think it's MUCH easier to train a DB guy some business sense than it is for a a business analyst to have Hadoop drilled into them. Of course, the downsides of a coder without sufficient savvy are harder to detect than a numbers guy who can't make his program work, and therein lies your problem.
- Business Analyst: Data Scientist - Systems Analyst: Growth Hacker - Public Relations: Social Media Evangelist
What else?
* be very good at working with large datasets with computational tools (hadoop is an example)
* be a decent programmer, scripter, and hacker
* have a decent background in statistics
A good data scientist:
* has a good intuition and business sense
* can explain insights to non-technical people (usually through visualization and plotting)
* knows machine learning and predictive analytics
It's a vague term, but purposefully so. There's tons of stuff you can do with data, a data scientist knows what to do and how to do it.
I recently moved to SF and am currently interviewing for data science positions - particularly ones involving social networks and applied graph theory - so drop me a line if you know anyone who is dealing with that problem space.
2. Be smart with an eye for economics (there is way more overlap than people give it credit for).
3. Start by talking to people and telling them what you want to do. Most founders want to help people reach their dream.
If you have a github account, email me and maybe you can start with us over at 500px here in Toronto.