Comment by roenxi | Hacker News Reader

Comment by roenxi | Hacker News Reader

He is glossing over the major issue with base R, which is its tendancy to switch data types in a way that appears random to new users. As an undergrad I spent nights literally on the verge of tears debugging R code where the types had been mucked up by R's bizarre semantics. Claiming that data[foo] is the same as filter(data, foo) is not correct - the [] and [[]] operators have a lot of strange side effects depending on what argument is passed in. It is like saying map() and a for loop are the same because everything a map can do a for loop can do. That is true, but the single-mindedness of map is a major selling point because you know it is only going to do certain specific things. The issues here will disproportionately affect beginners who are likely to accidentally pass in the wrong thing - it needs to fail with an error or in a predictable way to best help them. Hadley's code fails sensibly.

One of the major innovations of the tibble package is that it puts the types of each column into the headings when it prints the table out. Dr. Matloff is grossly underestimating how painful it is for beginners to, eg, figure out when they are dealing with a column of strings vs a column of factors. If a beginner copies code to someone else for help, the other person can actually tell what types are involved! It is like magic compared to base R.

Given how ad-hoc the R community is and the paucity of people with programmer backgrounds, these minor usability improvements on controlling types are important. And anybody who cares about efficiency is using the wrong language.

EDIT Also, this comment that RStudio is bypassing an Open Source project's leadership - that is one fo the major features of open source. It is usually a healthy sign that the project leadership are lost in the weeds. I recall a situation with broad parallels in the history of GCC for example; it didn't kill the project although I believe a bad codebase died.

Another underestimated advantage is the guarantee that dplyr functions that manipulates a data frame, will always return a data frame.

> "Dr. Matloff is grossly underestimating how painful it is for beginners to, eg, figure out when they are dealing with a column of strings vs a column of factors."

This is just sapply(df, class)... Is this not found in R 101 material that beginners are likely to find?

Pay no attention to the myriad of other apply functions hiding behind the curtain and the difficulty in remembering what each of them does and why you can't just use map, eh? :)

So to answer your question, no, its not just simply that.

Looking at the documentation; "sapply" is itself a wrapper for "lapply", and will do a number of different things depending on what arguments are passed into the function or not.

There is nothing to remember to solve that problem besides the wrote memorization of sapply(df, class).

I use a simply device to keep these straight:

- (l)apply: List apply always returns a list

- (s)apply: simplify apply tries to return simplified result

- (v)apply: verify apply checks the return type conforms to user supplied example

- (m)apply: multiple apply applies FUN to multiple vectors

- (r)apply: recursive apply is essentially a flatmap

- apply : no device here, only use on matrices, never data.frames

Love it or hate it, it's pretty clear these fns had very little api design thought go into them.

As much as I think people can lean too much on the concept, single responsibility principle would've gone a long way here.

After reading your explanation I still, as has been the case for years, don't understand what sapply or rapply does, and vapply sounds weird.

That isn't going to change, because I'm just not going to use them or 'invest' the time in finding out what some statistician-of-yore's interpretation of a map is. Instead I'll stick to tidyverse map - returns a list. Or tidyverse map_[int/chr/dbl/etc, etc] if I want a vector of [int/chr/dbl/etc, etc].

That and the data frame manipulation verbs covers the most useful 80% of cases where *apply would otherwise be needed. If the base R team were implementing functions that way in the base, stats professors wouldn't need to complain about mass exoduses from base R.

Yeah, that's a valid point. And it took me a couple years to internalize these differences -- mostly by being burned on numerous occasions. Base R has a lot of idiosyncrasies and they have to be memorized, unfortunately.

I develop R packages for my colleagues and so I stick to Base R whenever possible. I don't want my packages depending on the tidyverse at all. But for EDA, I am agnostic about what my colleagues do. They should use the tools that stay out of the way and let them get their hands around the dataset intuitively. For me that's Base R, for others it's data.table or tidyverse.

> After reading your explanation I still, as has been the case for years, don't understand what sapply or rapply does,

There is nothing complicated about what sapply does... It simply means loop over the elements of the first argument (which must be a list; btw a dataframe, df, is internally the same as a list) and apply some function. lapply does this and returns a list, sapply does this and can optionally "simplify" the results into a vector, etc.

So:

lapply(df, class) = loop over the elements of df and tell me the class, return this in the form of a list

sapply(df, class) = loop over the elements of df and tell me the class, return this as a character vector

This is basically lapply:

  res = NULL
  for(i in 1:length(df)){
  res = append(res, class(df[i]))
  }
  return(res)

Or simply str(df) or summary(df)

Reply on news.ycombinator.com