Hacker News Reader

Comment by kgwgk | Hacker News Reader

This also looks painful: https://community.rstudio.com/t/magrittr-inside-a-package/20...

That's an interesting caveat with pipes/magrittr (I didn't know of it), although it's out of scope of this debate since this discussion is just about how tidyverse uses pipes.

In my experience if there's a package that's required in a script/R Notebook outside of tidyverse, I just import the entire library at the start of the file Python-style (to make the dependency obvious), which would avoid this issue.

If you claim that something like

  the_data <- read.csv('/path/to/data/file.csv')
  the_data <- subset(the_data, variable_a > x)
  the_data <- transform(the_data, variable_c = variable_a/variable_b)
  the_data <- head(the_data, 100)

is a pain to read compared to

  the_data <-
    read.csv('/path/to/data/file.csv') %>%
    subset(variable_a > x) %>%
    transform(variable_c = variable_a/variable_b) %>%
    head(100)

I thinks it’s fair to mention how by making things more complicated by using pipes you expose yourself to other issues.

(In my opinion the first variant is not less readable, and it has the advantage of allowing any of the operations to be removed by commenting the corresponding line, while in the second case to remove the last operation you also need to remove the piping operator in the preceding line.)

[Edit: I’m not sure I understand your “out of scope” remark. You don’t mean that magrittr is outside of tidyverse, do you?]

They are roughly equivalent to read; but the reason for that is that the first code sample is clearly someone unwinding something that they are conceptualising as a piped statement. If you aren't conceptualising the code as an unwound pipe, you'd have to actually inspect the code and think about what each function argument means rather than being confident it is a set of chained transforms as you can be by glancing and seeing a %>%.

I'm a bit suspicious about that coincidence - after I met the pipe operator, a lot of my code looks like that. Before I met the pipe operator, not very much of my code looked like that.

As long as people are thinking in pipes then they can choose if they value making commenting code cheap vs making reading/typing code cheap. But it is quite important that beginners are introduced to pipes as a concept as they learn R. The semantics are critical, using the syntax is optional.

One could also say that the second code sample is someone unwinding something that they are conceptualising as a set of nested operations:

  the_data <- head( transform( subset( read.csv('/path/to/data/file.csv'),
                                       variable_a > x),
                               variable_c = variable_a/variable_b),
                    100)

All of them are equivalent, each representation has its advantages and disadvantages.

Well, yeah, if anyone wants to write their pipes that way that is also an option. They aren't going to though, that isn't really human readable when there is a sensible option like using a pipe operator.

If you find that the example above is not "human readable" then the following code will make your head explode because it cannot be transformed into a pipe representation (or at least it's not so straightforward):

  the_data <- transform( subset( rbind( read.csv('/path/to/data/file1.csv'),
                                        read.csv('/path/to/data/file2.csv')),
                                 variable_a > x),
                         variable_c = variable_a/variable_b)

Maybe consider:

  the_data <- read.csv('/path/to/data/file1.csv') %>%
            rbind(read.csv('/path/to/data/file2.csv')) %>%
            filter(variable_a > x) %>%
            mutate(variable_c = variable_a/variable_b)

You've thrown out a few variations on a theme here, but I'm not sure where you are trying to angle towards. There are a lot of ways to format code, and I can't tell you what works best for you.

But this isn't shaking the paradigm of having an initial block of data (here separated a little awkwardly into two files) that is being subjected to a series of functional transformations, which is sorta pipe-like. Being able to talk about this as a 'pipe' is hugely useful, and having a pipe operator really standardises how people can talk about it. In my experience, selling this as a concept to a beginner is a lot easier when it has syntax support so they know they are doing the 'right thing' and get error messages/uncomfortable looking code when they try to cheat. I don't really care how you nest the brackets, if you want to write something that isn't a pipe go with (edit obviously a non-buggy version of...):

  for i in 1:nrow(df){
   if(df[i,'variable a'] > df[i,'x']) continue;
   df[i,'variable ac'] = df[i,'variable_a']/df[i,'variable_b']
  }

If someone shows me that and I can tell them to 'go rewrite it using pipe operators, it will be better' they will naturally start seeing that that horrible block of mixed ideas can be separated out into a few basic composable operations, and that is funnily enough easier for a beginner to grasp in my experience teaching people to use R. Plus, they can tell that a statement is going to be a series of composes on top of raw data if they see a pipe operator. This is a useful thing that makes it easier to read the code.

For the record, the original example is from the homepage of magrittr (I found it here https://www.fromthebottomoftheheap.net/2015/06/03/my-aversio...) and I'm not trying to angle towards anywhere. I only wanted to point out that the version with a sequence of assignments is not so painful to read (to me) and even the nested version is readable and has some advantages. Not that anyone cares, but I think I would have written something like:

  df <- rbind(read.csv('/path/to/data/file1.csv'),
              read.csv('/path/to/data/file2.csv'))
  df <- df[, df$variable_a > x]
  df$variable_c <- df$variable_a / df$variable_b

or maybe:

  df <- subset(rbind(read.csv('/path/to/data/file1.csv'),
                     read.csv('/path/to/data/file2.csv')),
               variable_a > x)
  df[,”variable_c”] <- df[,”variable_a”] / df[,”variable_b”]

Whether you think these are badly written pipes or not pipes at all, I don't care. It's good enough for me.

My point is that sometimes you don't have a single "thread" of calculation. Putting rbind into a pipe like you did is somewhat artificial (broken symmetry) and doesn't work so well if there is some pre-processing before the merge and some post-processing after the merge (or if we have two or more essentially different arguments in a function that need some preprocessing). You may say that having multiple pipes, one merge operation (or whatever the function with multiple inputs is), and then a downstream pipe is the "human readable" way to do that. I'm not sure if that makes programmers able to handle some nesting superhuman or subhuman :-)

Teaching a "paradigm" can be too limiting. Looking at some random tutorial on the web:

"To demonstrate the above advantages of the pipe operator, consider the following example.

  round(cos(exp(sin(log10(sqrt(25))))), 2)
  # -0.33

"The code above looks messy and it is cumbersome to step through all the different functions and also keep track of the brackets when writing the code. "The method below uses magrittr‘s pipe (%>%) and makes the function calls easier to understand.

  sqrt(25) %>%
    log10() %>%
    sin() %>%
    exp() %>%
    cos() %>%
    round(2)
  # -0.33

Really? Do we want to teach people that the code below is so much better than the code above?

What do we expect them to do if they find something like

  100*exp(cumsum(0.6*diff(log(STOCKS))+0.4*diff(log(BONDS))))

which is a perfectly readable way of calculating the evolution of the value of a 60/40 portfolio of stocks and bonds from the value of each component at each rebalancing date?

http://thatdatatho.com/2019/03/13/tutorial-about-magrittrs-p...

I don’t think anyone is arguing that the pipe should be the -only- form of composition. Just that it’s a useful form when you have a linear sequence of transformations. Sometimes it’s useful to force something close to being linear into a linear form for consistency, but typically you would switch to an alternate form of composition, typically assigning to intermediate variables.

It’s easy to find examples of using the pipe in ways that I would consider suboptimal. But that doesn’t affect my thesis that, on average, the use of the pipe leads to more readable code.

I was also initially quite sceptical of the pipe, since it is a fundamentally new syntax for R (although obviously used in many other languages). I think the uptake by a wide variety of people across the R community does suggest there’s something there.

I was actually excited about pipes when they were introduced but in the end I'm pretty happy writing and debugging "unreadable" code.

I had a similar experience with Lisp syntax: Clojure's threading macros seem a neat idea but I do actually prefer old-style nesting of function calls.

Maybe my R journey is a bit atypical. I started learning R around the time you created reshape and ggplot and used them extensively. But as the "tidyverse" thing has evolved I have found myself more attracted to "base" R as I've become more familiar with its data structures and functionalities.

A lot depends on what you’re doing. Most of the time, when I’m not doing data analysis, I don’t find pipes to be that useful so I don’t use them.

Because the pipes are at the end of the line, you can comment out a line and it will still work fine unless you're removing the last line of the chain.

That’s precisely what I said.

Reply on news.ycombinator.com