R is hot [pdf] | Hacker News Reader

R is hot [pdf]

revolutionanalytics.com

50 pointssonabinu54 comments

R has some spectacular libraries and a huge userbase among statisticians, so it is definitely a big actor. On the downside, the core team who manage it appear to be somewhat averse to making the language progress in meaningful ways. R is very slow, has difficulty handling large data, and is a source of many "wtf" moments (e.g. see "the R inferno"), even for advanced users. I think the best thing that could happen would be a mass migration of R users towards Clojure, Julia or (more realistically) Python, and an attempt to reproduce in that language the key packages that are R's main appeal at the moment (ggplot2 mostly).

https://github.com/ContinuumIO/Bokeh is an attempt to provide a ggplot equivalent in python.

what advantages does this offer in comparison to matplotlib?

Are the points they describe (easy faceting, bulk application of aesthetic and visual parameters across categorical variables, pleasing default color palettes for categorical data, etc.) really that much of a pain point with matplotlib?

I used it for two or three projects but not really extensively and especially not for complex statistical plotting, thats why I ask.

> what advantages does this offer in comparison to matplotlib?

"Although it is a Python library, its primary output backend is HTML5 Canvas."

(Bokeh developer here)

The core goals of the project are to offer: interactive, easy-to-specify, powerful novel graphics, with good support for statistical plots, in a web browser. That's quite a handful of things to balance, but we are picking up the best ideas from already existing projects: matplotlib, chaco, ggplot, protovis, stencil. The project is still nacent but we have promising results so far. (For instance, here is an ipython notebook export with interactive ggplot-style faceted plots of a Pandas DataFrame: http://htmlpreview.github.com/?https://github.com/ContinuumI...)

This kind of composite plot is a one-liner in the grammar of graphics. It would be much more with Matplotlib or Chaco. There are also marvelous novel graphics that are possible with Protovis-style Marks & Glyphs: http://mbostock.github.com/protovis/ex/ Although these are technically possible with existing libraries, the amount of programming skill required to create them is outside the capability of most data analysts. Our goal with Bokeh is to make these all accessible to that larger audience.

To add to pwang's comment, to me the problem with matplotlib is that it makes an assumption that I believe to be untrue: the majority of data analysis tasks only need a pre-set list of plots. In my experience, most data analysis tasks need some novel combination of existing components, that may not have a specific name (e.g. pie chart, bar chart).

The R Inferno: http://www.burns-stat.com/pages/Tutor/R_inferno.pdf

Abstract: If you are using R and you think you're in hell, this is a map for you.

Agreed. Ideally, a new language/api will emerge that learns from the successes and failures of R. Something that is designed to be used as it is used presently, that incorporates Hadley Wickham's ideas, that has a less quirky OOP system, and that is more readable.

Perhaps currying/function compositions could be encouraged to avoid(hard, to(read(nested, functions))). However, being unexperienced with currying/function compositions, I have no idea if that would complicate the language...

You can already implement Curry in R: https://github.com/hadley/pryr/blob/master/R/curry.r :)

I think the best thing that could happen would be the development of a competing R implementation. In my mind this would include the definition of a minimal R language implemented in something like Riposte (http://www.justintalbot.com/wp-content/uploads/2012/10/pact0...), and the majority of base code written either in R, or the sort of C++ that Rcpp makes very natural.

The best thing that can happen to R would be to rewrite the parser in Haskell and make R a domain specific language. Leave the stats/R language to statisticians who originally designed R, and bring in the Comp Sci crowd to actually make a meaningful speedup.

That's an unusual suggestion - I haven't heard anyone complain about R's parser before. It's also hard to argue that R is a DSL (at least in the spirit of http://en.wikipedia.org/wiki/Domain-specific_language). There are many features of R, features that make it particularly well suited to statistics and data analysis, that would be very difficult to reproduce in an embedded DSL (missing values are one particularly important example)

Julia is meant to be a competitor to R. The Julia developer community have added dataframes to compete with R Datasets and packages.

See: http://docs.julialang.org/en/latest/packages/packagelist/?hi... and: http://docs.julialang.org/en/latest/packages/packagelist/?hi...

Interesting. First time I have heard of Riposte. It dousn't sound like a new language though, but rather a new runtime. Am I understanding incorrectly?

agree that the change is slow, but R is increasingly finding a place in the world of large datasets. Check out these http://www.oracle.com/technetwork/database/options/advanced-...

http://www.revolutionanalytics.com/why-revolution-r/whitepap...

The biggest problem R faces is the corrosive attitude the language developers and maintainers have towards the R community. The R core team sometimes seems openly hostile to users; instead of Benevolent-Dictator-for-Life, it's more of a military junta that suppresses the opposition.

Where is this "R community" hiding? The IRC channel is essentially dead.

You'll find the most active public R communities at stackoverflow and on the R-help mailing list.

You'll find a lot of them at http://www.r-bloggers.com/

that is a great place to look for help

I like R, it's more fun to write in than, say, Matlab. But it is unbelievably slow and has very little support for web-based projects. I'm reading "Python for Data Analysis" right now, and I'm pretty sure the book is converting me. Pandas' DataFrame includes R's data.frame's functionality as a strict subset, and that's enough for me to switch. I still might need to use rpy2 for awhile for certain packages though.

I started using data.table [1] as a replacement for data.frame recently and it's hugely improved my productivity in R. On the web front, the Shiny [2] project has a emerged as a possibility for quickly building web frontends for analysis projects in R.

1. http://datatable.r-forge.r-project.org/

2. http://www.rstudio.com/shiny/

If you like data.table, but use python, take a look a pytables.

Ugh, should've put a [pdf] warning in the title. Other than that, R can indeed be hot, but maybe just in the special 'flavor' Revolution Analytics developed, otherwise it's one of those languages you dread using but are forced to because so much scientific code/packages etc has been written on it, some with no equivalent in other languages.

just did put in pdf to the title

however you managed to skip past the auto-scribd-autodetection, congratulations! scribd is a mistake.

It looks like URL parameters after .pdf extension confused the parser that probably expects URL to _end_ with '.pdf'

It's also from 2010 (according to the copyright statement in the PDF).

Expect R is dominate as the SAS legacy begins to die. R is spreading across a lot on analytics departments in Universities, from Stats to Finance. The holdouts are the CS departments, who have rebranded much of statistics into data mining/machine learning and, smartly, use python.

Another important metric is the use of R in clinical studies. The FDA now accepts much research in R whereas before they did not. That is a slow moving change, and shows R will be around, at least in use by Pharma, for a while.

I dunno if R is that hot, but I think RStudio's Shiny (http://www.rstudio.com/shiny/) is on its way to making it hot. Best thing to come out in the R world in a long time. Besides being a great visualization tool for people who already use R, it might be something that makes programming easier to learn for kiddos (they can even learn some HTML + JS at the same time...) This almost brings R to a level where all the material taught in a freshman COMP101 class could be implemented in R (... as long as you don't even mention OOP)

I think R is just as terrible as Matlab. Personally I prefer Mathematica, but I even stopped using that, it's just so painful to use with everything being lists.

Could you elaborate your dislike for mathematica and matlab? In my university departement a lot of people are using matlab while I somehow prefer C/C++ as I still cannot get my head around it.

My use of Mathematica is usually reading some data (generated from a simulation, say) from a CSV file and then doing something with it and plotting it. What I dislike is that simple things like aggregation, slicing and dicing the data is kind of pain when everything is a list and using Table[], Map[] etc. It often occurs to me that it'd be much quicker to express what I want with an SQL query. Also, with Mathematica's lists header information is lost, so I always have to remember which column in the list of lists is what. It's kind of a pain. Basically, whereas in SQL I can tell it what I want (declarative), in Mathematica I actually have to implement it (functional).

Then, in the plotting phase I'm constantly looking up Mathematica's plot options because I don't use it often enough to remember them. By default almost everything looks like crap in Mathematica, and you have to put in 10+ options to try to make it look good, but even so I'm usually dissatisfied with the result. Often I want something and then I find someone on a message board saying that it doesn't support that, and then he shares his 100 line program which gets Mathematica to do that. Then I usually give up. Getting programs from the Internet to work with Mathematica is a pain because Wolfram pushes out a new version every year and they keep changing small things which break backwards compatibility. Compare to the (much less powerful) Google Charts library, where everything looks much better and easier to comprehend by default.

One of my startup ideas is to write something like Mathematica, much simpler, runs on the web as a service, you can program it in Javascript, has embedded SQL, and it uses something like Google Chart for simple plotting, or you can pull in your favorite plotting library. Let's you share your work by sending around a link.

PS: I played around with R, but it seemed even more cryptic than Mathematica. Admittedly I didn't spend much time in it.

Have you seen Afterquery?

http://afterquery.appspot.com/help

Thanks, this looks interesting.

Actually I'm developing/maintaining something like this at my job, as part of the Data Services team. Similar to this, it uses Google Charts and is a node.js app. It will be open-sources in a couple of months. The goal is to avoid using Gooddata for as much stuff as possible, GD is too sluggish.

Matlab as a language is awful and here are a few examples why:

- You can't chain indexing with function calls. So you can't do myfun()[1]. This gets obnoxious really quickly. I hope you like useless intermediate variables. - You can't have a N-D array with singleton trailing dimension. So if you want dynamically created array sizes you will have to create special cases for that. - One external function per file.

"How Did a Statistical Programming Language Invented in New Zealand Become a Global Sensation?". Hmm, slightly condescending title—though I had no idea that it was invented by two academics at the University of Auckland (not "Aukland"!)

Check this out http://www.nytimes.com/2009/01/07/technology/business-comput...

Is "invented" really the correct word, given that R started its life as a straight up Open Source re-implementation of the S language.

Indeed. Is the language the second most famous scientific 'R' from NZ?

http://en.wikipedia.org/wiki/Ernest_Rutherford

So, let's say you're interested in playing with R, where's the best place to start? I've got basic statistics background and some datasets I could play with.

http://www.codeschool.com/courses/try-r

Codeschool recently launched a free tutorial class for R thats interactive. They do a great job with intro level learning.

A great place to start learning the basics of R - coursera has a couple of classes using R - Computing for data analysis https://www.coursera.org/course/compdata and Data analysis https://www.coursera.org/course/dataanalysis

go to the R website - http://cran.r-project.org/

A great place to take a look and see some things that you can do with this program is this:

http://gallery.r-enthusiasts.com/

Examples of graphics with code, and some are very smart

If you are looking for a book, "The Art of R Programming" is (by far) my favorite R book.

It explains a lot of the general principles behind the language, so you can figure out details for yourself (in comparison to, for instance, the R Cookbook, which has hundreds more pages of details and code samples for specific use cases).

Has anyone explored the possibility of making a nice clean language that compiles to R?

Python has some nice tools to interface with R when needed (rpy2 for example). On top of that, scipy/numpy/matplotlib provides a lot of scientific routines.

Personally, pandas was the library that converted me from R to python for general data work.

This would suggest that "R is not" http://lang-index.sourceforge.net/#grid

The article is suggesting R has become popular for data analysis projects, not as a general purpose programming language.

R is essentially for statistics, it is not a general purpose programming language

Please add a [pdf] tag in the title!

just did, didn't realize the confusion it was causing

Reply on news.ycombinator.com