Hacker News Reader

Comment by michaelochurch | Hacker News Reader

Data science seems, for now, to be what software engineering was supposed to be: a career where you choose your own tools and problems (with some constraints) and that gives you the freedom to move about in different industries instead of being stuck to one in the way that most programmers now are.

In many organizations, data scientists are full-time programmers but who get the dibs on the most interesting projects. I identify as a data scientist as code-word for "no-hire if the work's not interesting". There's plenty of hard engineering (in addition to traditional data science, where statistical intuition is more important) in data science. There are plenty of data scientists working on OS hacks, compilers, and other "hard engineering" topics. The difference and advantage for a data scientist is that your boss doesn't think he could do your job if he wanted to. If your title shows that you actually know math, you're not "just a code monkey".

View on HN

I've seen you write something similar to this before, but I don't agree.

Counterexample: a good friend of mine specialized in computer graphics where he did a lot of math-heavy research. I specialized in machine learning where I also did a lot of math-heavy research. We are both full-time programmers who choose our tools and flexibly work on interesting projects, but he would never get hired as a "data scientist", whereas I did. I think anyone who has specialized enough in something that's useful to a company can find a good, flexible job (e.g. my graphics friend and me).

The fact that the term "data scientist" is so ubiquitous simply means it's cooler/more useful than other specializations right now. Some companies may get away with abusing the title because it's so vague, but it does mean something to companies that actually know they need one.

>>There are plenty of data scientists working on OS hacks, compilers, and other "hard engineering" topics.

Can you give some examples on this? Seems very interesting.

Nothing comes off the top of my head, and it's not like they get to specialize in compilers. They mostly end up doing one-off hacks to make an existing algorithm more performant.

What's fun about machine learning is that it touches so many other parts of computer science. You could be at a high level writing DSLs in Clojure to make it possible for statisticians to specify their models directly, or you could go to the low level and write GPU code.

The general rule is that if your boss thinks he can do your job, you lose. If he doesn't think that, you win. When you're a data scientist, your odds are much higher of coming out in the second category.

To your last point, it depends on what role you currently serve to your company. If you're among statisticians, you make yourself valuable by knowing more about computer science and programming than the rest of the group. If you're amongst engineers (probably more common than the former--at least on HN), knowing probability and statistics gives you that edge.

I encourage you to peruse the Github page of the R celebrity Hadley Wickham. While trained as a statistician and employed as a professor of statistics at Rice, he writes quite a few R packages that deal with more traditional development roles (unit testing and debugging, not to mention Plyr and GGplot): https://github.com/hadley?tab=repositories

Also, Duncan Temple Lang, another professor of statistics, has done some really amazing work with R compilers and CUDA interfaces. https://github.com/duncantl?tab=repositories http://www.omegahat.org/

Rob Zinkov (https://github.com/zaxtax) is a first-rate hacker. He has a popular ML blog (http://zinkov.com/) and organizes the Los Angeles machine learning users group.

Reply on news.ycombinator.com