Hacker News Reader

Comment by civilized | Hacker News Reader

Base R only? tapply? Ick, no thanks. New R programmers should learn tidyverse https://www.tidyverse.org/learn/, not just base R.

There are a lot of R programmers (including the author of the OP, apparently https://github.com/matloff/TidyverseSkeptic) who are used to the old way of doing things and allergic to tidyverse. But the base-R only, anti-Tidyverse attitude is going the way of COBOL.

I have worked full time in R for many years and it is no contest.

I agree with the author that one must learn vectors and subscripts, but that is not an excuse to shove dplyr under the rug. It is an easier way to do lots of things.

View on HN

I agree that new R programmers should start with the tidyverse, though actually the first element of the tidyverse I'd teach is the pipe and that's now part of base R; the second element I'd teach is using the readr stuff mainly to not have to worry about stringsAsFactors and stringsAsFactors is now default off.

Still, I think ggplot is a better way of thinking about plotting than base R's multiple and not very coherent plotting systems, dplyr beats the pants off any kind of base tools for manipulations, a lot of the tibble/pillar display stuff is great, and personally I disagree with Norm and think functional programming is as accessible as loops are to novice programmers.

I've taught a lot of people programming, and frequently work with novices. Functional concepts are nearly universally a sticking point for learners. For loops haven't caused any more trouble than the other basic concepts in my experience. It's an order of magnitude more difficult for learners.

Loops are very easy to debug in R. When they break and they're not inside a function, your REPL state is set to exactly what the environment was before the loop broke.

Can’t you do the same thing by dropping a browser() into a function you are applying to each element in a list?

It's not as convenient because now you have to rerun the code and get in and out of a debugging environment (which can be slow).

I think FP is good for more concisely expressing simple stuff that's unlikely to break. Don't really believe in using it as a complete replacement for loops.

How about a conditional breakpoint? Though this doesn’t solve the “issue” of rerunning through the whole list.

Or - once you’ve found what element in your list is causing the issue- extract it and feed it to your function with your debugger on.

Admittedly I have often had a more challenging time fixing FP code than if I had used a for loop. This might be because I am not as familiar/comfortable with FP than procedural flow.

There has to be a for loop eventually. But crawling through a for loop within a procedure can also be painful if there are many different case scenarios. You could counter that good code wouldn’t suffer such a problem, but if the code was good it might not have been broken!

> I've taught a lot of people programming, and frequently work with novices. Functional concepts are nearly universally a sticking point for learners. For loops haven't caused any more trouble than the other basic concepts in my experience. It's an order of magnitude more difficult for learners.

The problem is that is teaches people the "wrong" * way to use R. I agree for loops are easier to understand, but when nearly the entire R universe operates on vectors, you end up with students who will do R the hard way their entire career.

If you surveyed most R users (verses R programmers) I would say the majority would be unaware that the base type is a vector.

* I know there is no wrong way to do anything, but I have seen plenty of times for loops simply applying an already vectorized function to each element of the vector.

I think R users should know how to use vectors, vectorized functions, loops, and core tidyverse (e.g. basic dplyr and ggplot). FP stuff like purrr::map isn't essential.

Those interested in R with tidyverse may find "R for Data Science" (https://r4ds.had.co.nz) to be a valuable resource.

You’re being downvoted but it’s true - R would be dead today if not for the tidyverse (both because of ease-of-use and fostered community).

The experience between using the two is night-and-day. I would rather use the cli than base R at this point, and am convinced that the only proponents of base R over the tidyverse haven’t tried the latter for themselves.

Thank you. In reading the author's tidyverse critique, I think he has some valid points. It is wonderful that you can do so much with just vectors in base R, and this may be better for true beginners with no coding background. I do not agree with hiding vectors and subscripts and attempting to indoctrinate students with very pure and abstract coding approaches - as it seems to be alleged that some tidyverse advocates do. But "dplyr is bad" is just an awful take, and "dplyr is bad for beginners" is no better. These are just about the worst takes you could have about modern R programming.

My impression was that the last time I was using R intensively there was a movement back from tidyverse to base R. But that's a few years ago, so you say that's reversed again?

I'm not against it at all, in fact I was doing nearly all of my work in dplyr, reshape, ggplot,... and I second that it's much more convenient and intuitive than the various apply's,...

My impression was that it was quite political and Hadley Wickham had antagonized some people in the core R community (and tbf, it was a bit annoying with him introducing redundant packages, reinventing and relabelling multiple ways of doing the same things, and presenting each new iteration as the end-all-be-all).

I'd say there was been a backlash against tidyverse, but tidyverse is beloved by most new R programmers, and fighting it is a one-way ticket to irrelevance.

Re: antagonistic behavior, I would just recommend that all participants consider the old glass houses proverb as applicable to themselves, and try to do better going forward.

In contrast, in my experience students relying heavily on tidyverse love it - but are also poorly positioned for doing anything that's not contained in an easily found template.

I don't think that is the tidyverse's fault. Learning the tidyverse doesn't corrupt your brain and make it impossible to learn about vectors and subscripts.

I strongly recommend data.table R. Tidyverse is an improvement on base R, no question. Data.table has less intuitive syntax and can be harder to learn, but is lightning fast and memory efficient. If you're working with more than 1M rows, you should be using data.table.

Here are some benchmarks: https://h2oai.github.io/db-benchmark/

I think it's great people like data.table's performance but I don't think performance is a compelling reason to be prescriptive about data.table. It's a reason data.table may be appealing to some. People can try them and decide for themselves if the performance is worth the trouble. In many cases the performance difference with dplyr is hardly noticeable and IMO readability of dplyr becomes the more meaningful advantage.

I have plenty of projects with tables with millions of rows, and I rarely find dplyr to be slow enough to get frustrated about. And even then I'd rather endure, or maybe use dtplyr, or maybe use data.table for a small performance-critical part of the code, than make a wholesale switch.

Performance is probably the last reason I use datatable.

I just enjoy having to read much less code when compared to tidyverse and SQL.

Plus the syntax difference between tidyverse and dt are overstated.

https://atrebas.github.io/post/2019-03-03-datatable-dplyr/

I find data.table code as obscure as base R if not more so. Purported brevity (always illustrated only on very simple problems) doesn't compensate me enough for the obscurity.

I could get used to it, I'm sure. I could also get used to doing everything in base R, pandas, or Clojure. Doesn't mean it's a good use of my time.

I get the sense from how some people blog that basic "filter + group by + summarize" type pipelines on giant datasets are all they really do. data.table may be a great choice for that kind of thing. But I use dplyr as a tool of thought. I can solve complex, novel data processing and analytics problems in dplyr more precisely and efficiently than I can explain in words. Instead of having an idea and then having to think about how to code it, the ideas translate into dplyr in an obvious and automatic way.

All that said, data.table's authors have worked much harder to provide advanced functionality, most notably non-equijoins, that it looks like the tidyverse may never catch up to. So if you need that kind of thing a lot, you need data.table or maybe DuckDB.

What if you are working with 1000s of rows? Still worthwhile?

Reply on news.ycombinator.com