Comment by srean | Hacker News Reader

Comment by srean | Hacker News Reader

Parent Full thread

I cannot speak for zmmmmm but I can think of two reasons (i) performance, not that R cant be fast but you have to pull out heavy machinery such as parallelization much earlier and (ii) its inconsistent and confusing language semantics, and silent and unexpected type promotion/demotions that make it very error prone to develop and debug code in. To me the one redeeming quality about R is ggplot and, ok, availability of push button libraries

I am not so sure about the scope of the 'bleeding edge' part. It is popular among old school, statisticians. Another population (no pun intended) that R is popular in is one where one knows neither statistics nor machine learning but just wants to try out a laundry list of canned methods without a need to understand: pretty plot goes up, yay awesome; pretty plot goes down, ok try the next algorithm (or the other way round). Here I think R is pretty unbeatable in its breadth.

It is not that popular among machine learners. Part of it is cultural, a typical machine learning person will be coming from a CS background and then R grates more.

It has been claimed that R is lisp meets stats, I think that would be Julia now and Lush then. http://lush.sourceforge.net/. To quote Yann LeCun:

"Lush combines three languages in one: a very simple to use, loosely-typed interpreted language, a strongly-typed compiled language with the same syntax, and the C language, which can be freely mixed with the other languages within a single source file, and even within a single function."

You definitely have some good points - I agree on points (i) and (ii) - these are shortfalls of the language and in many cases (especially when building production level code) may be reasons to switch).

I disagree with you premise that R is not popular among machine learners and the implication that has for the "bleeding edge" comment. Perhaps it is less popular with ML people coming from a CS background - but it seems to be the most widespread language for ML (or "statistical learning") among statistics people. And that fact is not to be discounted - perhaps the preeminent book on machine learning (or at the very least one of the most popular) - "The Elements of Statistical Learning" - is written by statisticians (and in fact uses R exclusively!). The Journal of Statistical software, which features papers detailing many ML libraries, has far larger coverage of R packages than any other language: http://www.jstatsoft.org/ . I would suggest this is the evidence for the "bleeding edge" comment. Other languages' packages, say Python's ML libraries (scikit-learn, pyBrain, etc.), do note even come close to the breadth of capability in this space.

Coming from a statistics background,- I would venture to ask.. what's wrong with "pushbutton" libraries? And why are they only useful to people who do not know/understand much about ML/statistics?

Say you have a dataset and you believe ,say, a Random Forest would be well suited to predicting some response. Let's assume you have a very good understanding of Random Forests. Why would you not want a push-button library? Why would you WANT to recode the thing yourself? It's not as if your Random Forest will be any better or "more correct" than the one on CRAN or whatever language's repository you are using. I would argue it's more likely to have mistakes (coding something like this from scratch is no small task and the ones on CRAN have often gone through many iterations, improvements, and reviews from many experts in the field). And let's say you have some domain knowledge or some informed belief about how the Random Forest needs to be adapted to your particular problem (a different loss function or something) - you can easily edit the R package source code to do that - no need to rebuild the car just to give it a new paint job...

I guess everyone has left the building, leaving a reply in case you see it.

>I disagree with you premise that R is not popular among machine learners

I know what you are saying, and I agree, that is why I chose my words carefully.

    I am not so sure about the *scope* of the 'bleeding edge' part

The word I wanted to highlight is "scope". The population that identifies themselves with any of these labels: machine learners, statistician, data scientist, data modeler, actuarial scientist, prediction consultant and all other variations...is huge. So whether R is popular or not depends on whom you want to include and whom you want to exclude.

There is a quite a lot of variation here. In one subset, if you know the ins and outs of R you will be taken for a wizard, in another subset just listing R/MATLAB as one of your major skill will actually work against you.

> what's wrong with "pushbutton" libraries?

Nothing at all and I did not claim its wrong either. In fact I said that is one of R's strengths. For people who arent into the theory of statistics or ML then R can be a heaven sent. Especially if all they want to do is try out a catalog of algorithms.

The analogy I use is that of an automobile industry. If your goal is to be a run off the mill automobile driver/chauffeur R is your competitive advantage. If your goal is to be an automotive engineer (design new models and algorithms) R is an inferior tool. In fact if you want to be a F1 racer, you have to look elsewhere too. Like every tool it has its sweet spot, as long you don't venture outside its golden.

R is bad building material, but if you dont want to build something novel in the first place (in a statistical or machine learning sense), just call a canned solution on your (medium sized data) data, then R is the boss. If you try to do anything meaty or clever then you have to be careful with R's gotchas. It helps commoditize data analysis and popularize techniques amongst its large base. If you want statisticians to be aware of some cool technique, you have to release an R package, because if it isnt in CRAN it does not exist.

Reply on news.ycombinator.com