TC39 Pipeline Operator – Hack vs. F# | Hacker News Reader

TC39 Pipeline Operator – Hack vs. F#

benlesh.com

50 pointsjashkenas75 comments

The thing about the pipeline operator that strikes me is how toylike all the examples are. Look at this article for example - it’s a bunch of examples of adding and exponentiating numbers, as if you couldn’t do that already in a single line of code.

My suspicion is that once you move away from toy examples, most pipeline operator use cases are already covered by chaining class methods on classes. I suppose you could argue that the pipeline operator allows you to operate across multiple different data types, but I further suspect that handling many different data types with a single function is an annoying thing to get right and this won’t come up in practice all too often.

To be fair, the case of pulling in only the methods you need on rxjs seems somewhat useful… but do we really need a whole new operator just for that? And isn’t that the point of ::?

It's a mindset thing. The pipeline operator encourages you to think about repeatedly transforming static dumb data. Dot-operators encourage you to think about repeatedly asking some smart blob of state to mutate itself. So the pipeline lends itself well to a purer mindset.

F# itself much prefers the pipeline, because of how its type inference algorithm works. F#'s type inference finds it much harder to work out the types when you use a dot operator, because so many possible types T could have `(x : T).Foo(bar)`, all those types are unrelated to each other, and notions of "has a member `Foo`" are quite hard to get right in the type system. By contrast, if you call `x |> List.map bar`, the types are almost totally fixed in an easy-to-understand way.

"Handling many different data types with a single function is an annoying thing to get right" - no, this is very common, and generics even make it easy.

Dot operators vs plain functions are just two different ways to call a function, binding the lexical context `this` to a different value, possibly `undefined` in a strict context (which is default in es6 modules). The two are not uncompatible and libraries like jQuery make a wonderful use of the first version (dot call). The fact is that everything is mutable in JavaScript and one of its main API (the DOM) is actually built around this concept.

I really fail to see how those kind of changes bring anything of value to the language, apart from creating complexity where there is much already: some codebases will try and use this before it's ironed out with different / changing / competing transpiler implementation while it is only stage 2. What happened with decorators (see babel-legacy-decorators) should be a warning against enthusiasm for these kind of things and remind me of the pitfalls of macros in the Lisp world...

I am not saying that you should stick with an idiom till the end of time but changes should try and lower that complexity while (in the case of EcmaScript) being backward compatible so as to not break existing stuff.

Btw I love RxJS and its variants, Ben Lesh's talks (as well as those by Matthew Podwysocki, Andre Staltz etc...) from around 2015 and then on were one of the reasons I got into JS as a professional.

I merely explain where the pipeline operator came from and why it's preferred in F#: because it plays nicely with immutable data and the Hindley-Milner type inference algorithm. I make absolutely no comment about the sanity of adding anything to JS (or, indeed, of using JS at all).

Chained function calls only work if the types in question have immutable functions that support chaining. This is the case for arrays with `map`, `filter` and for promises with then/catch.

But it's not the case for the majority of libraries, including many of the builtin types.

A pipeline operator is generic and can be used everywhere, without requiring the API to be built around chaining. This can often lead to much nicer, more readable code.

I think this is something you only start to really appreciate once you've used it a bit, like in Elixir or F#.

Obviously this is much more natural in a functional language, where everything is immutable anyway.

The pipe operator/function is useful and appropriate when all you are doing is stringing along some data through a series of transformations and where introducing a named variable would be more confusing and take away from the context of what is actually important.

Methods are nice but suffer from the fact that adding a custom method to a new object is more involved and dangerous than just creating a new function. However, "f(g(h(x)))" is not very readable (at least for English speakers) since while we read left to right, the invocation order is right to left. "x | h | g | f" is arguably more readable since the evaluation order follows reading order (plus you don't have to deal with so many parenthesis).

I want to reemphasize though that introducing a variable is often better practice and that the pipe operator only makes sense when there is no suitable variable name.

> However, "f(g(h(x)))" is not very readable (at least for English speakers) since while we read left to right, the invocation order is right to left. "x | h | g | f" is arguably more readable since the evaluation order follows reading order (plus you don't have to deal with so many parenthesis).

There's also a "hierarchy" thing at play. With "f(g(h(x)))", x is "inside" h which is "inside" g which is "inside" f. With "x | h | g | f" all are on the same level. For me at least, it's easier to imagine your data "flow" through functions when they are on the same level. It sound a bit weird when I put it that way, but I don't think I'm alone in feeling that. Fluent interfaces are considered easy to read, and help flatten the code. Same thing with promise chaining compared to the callback > of doom.

> Dot-operators encourage you to think about repeatedly asking some smart blob of state to mutate itself.

Smart Blobs don't have to mutate themselves anymore than procedures have to have side effects.

Of course they don't have to. As I said, they encourage you to think in the object-orientation "encapsulate state and teach it how to mutate itself on demand" way, but you don't have to use them that way.

I'm being somewhat obtuse here, but I've never understood that.

Why would I be more likely to mutate state in sending a message to an object than I would be applying a procedure to a struct? (or a function to a record, depending on your terminology)

Immutable objects are a very old and mainstream idea, I believe they're even talked about in Effective C++.

Long history and baggage of object orientation, I suppose. There's nothing inherently mutable about dot-notation (indeed, F# uses it for accessors on immutable records as well as for "proper" object-orientation), but lots of object-oriented languages use the dot for accessing objects, and lots of object-oriented idioms involve mutation. If you're going to send a message to an object, you do kind of expect something to happen, and that's often a mutation somewhere.

Having said that, I actually don't know the history of the dot operator in any depth (https://cs.stackexchange.com/questions/89031/what-is-the-ori... suggests it arose in Simula).

> Why would I be more likely to mutate state in sending a message to an object than I would be applying a procedure to a struct?

I think most people don't think about objects in terms of sending a message to them, but in terms of calling a method on them. OOP in JS, currently, revolves a lot around mutable objects. JS itself revolves a lot around mutability. Some people are trying to change that, at least by getting proper support for the alternative. The pipeline operator is part of that.

If you don't have state what do you need an object for? Would it make more sense to have a "free" function instead?

I think one of the use cases is that it's easier to have a series of functions on immutable data compared to chaining classes. Considering JavaScript may get immutable data structures at some point, this would work well with them.

There is also the fact that JavaScript supports different types of programming, functional being one of them. Having a pipe operator helps you to make very clean and readable functional code.

Edit: the article also mentionned that methods can't be tree shaken. That's another improvement.

Tuples and records are proposed immutable data structures and currently in stage 2[1] (same as the pipe operator).

[1] https://github.com/tc39/proposal-record-tuple

> once you move away from toy examples, most pipeline operator use cases are already covered by chaining class methods on classes

But what if we don't want to use classes? What if we want to be cool and compose functions out of functions, Ramda-style, or Elm-style?

Unpopular opinion: we could just add, say, `pipe` and `pipeAsync` into the standard library, appart from aesthetic, I don't see the pipe operator to be very useful, at least the F# one doesn't because it's literally just another syntax for function invocation.

Standard library? In javascript? You mean a keyword like `async` or `yield`, or a built-in object like Date or Math?

When you look at a Ramda-based JS code base, all there is to look at is function calls. Such code eschews most statements.

Piping is a common operation, though, and having dedicated syntax makes the code more readable. For those code bases that use curried, unary functions, the F# proposal is perfect (this is the original motivation behind the operator).

The hack-style pipe syntax that has been chosen is much worse in that regard, which is disappointing for the very folks that requested its addition.

It is useful as an operator due to the fact that you avoid unnecessary function calls/lambda creation. "pipe(x, f, g, y => h(42, y))" is not going to be as easy to optimize as "h(42, g(f(x))" yet "x |> f |> g |> h(42, ^)" has the ordering of the pipe function while being equivalent to normal function invocation for the purposes of optimization.

I sort of agree though about the F# proposal version. It doesn't make sense to introduce if it is basically equivalent to having a pipe/pipeAsync function. I suspect this is a big reason why the Hack version was favored.

> Chaining methods on classes

But defining these is typically left to library authors in many languages. Writing your own Fluent/chaining interface isn't usually worth the effort. Where in these languages because any function is "chainable" and gets it for free a lot of programming ends up being chained pipelines. Anything where there is a logical workflow (a lot of programming) ends up being nice to read when expressed with pipes (i.e. do this, then this, then that, and finally return this). The pipe operator goes hand in hand with currying as well since you can complete some of the arguments before wiring the partial completed function into a generic workflow; a pattern that typically isn't be done with Fluent/Chaining methods on classes.

The primary usecase (which happens to me fairly frequently in Javascript) is when you need to operate on a native or library data structure with some function other than its built-in methods.

Some people also find it easier to program in an FP style (separating data and functions) in their app code. It seems to be a matter of neurodiversity.

I've wanted a pipeline operator in JavaScript for a while now. Working with Elm and Elixir has made me think of problems in terms of data transformation, and the pipeline operator is the biggest reason why. "Take X then do Y, Z, J, K, L and finally send it over to the client" makes what would be complicated code be only as complicated as the reader wants, as they can inspect the functions they care about (assuming they even want to).

If you're inventing a special shorthand for functions like foo(42, ^), then why only in the context of a pipe? Make up some punctuation to introduce or surround this kind of expression, then use it anywhere. Less to learn, more useful.

and why stop there? Introduce lazy evaluation, type classes, custom operator overloading and what not...

I'm jesting of course, I find JavaScript already too complicated, it's like some people want to make it look like Scala... just no.

I'm mainly puzzled why they want to introduce additional syntax specific for it, when higher order function are perfectly possible in the current language already and integrate well with the F#-style pipe.

Very reasonable, and that's why I phrased it like "if we do this".

A TC39 member on this topic of restraint and priorities: https://erights.medium.com/the-tragedy-of-the-common-lisp-wh...

> I find JavaScript already too complicated

I get that you don't want it *to become* complicated, but as it currently is, JavaScript is still a rather simple language.

This is sort-of how Scala does it. _ in the context of a lambda means "the next argument", so it is general:

    val plus: (Int, Int) => Int = _ + _

It looks like Hack doesn't support first-class functions, which would explain why they don't do this. But I agree that it would be a neat short notation for lambdas.

There's a proposal for this: https://github.com/tc39/proposal-partial-application

That's more specialized: it can express

    foo(42, ?)

but not what you'd write within this pipe syntax as

    foo(42, ^) + 1

The tradeoff is that it doesn't need some extra delimiter since the function call is what delimits it. Perhaps that's a better tradeoff, I'm not sure; but for sure we shouldn't have both.

I don't see how any of this is more readable than:

    let v = first();
    v = second(v, 10);
    v = third(x, y, v);
    return v;

Why do we need a new operator for this?

    return first() | second(_, 10) | third(x, y, _)

I somewhat agree.

Further, it has to be |> to disambiguate from bitwise OR. So yet another symbol to confuse readers.

And the placeholder can't be _ as that's a valid identifier. The first issue on the tracker is the bike shedding thread about what the placeholder symbol should be, and it's neither _ nor ^, but instead probably something like ^label.

I think it's kind of ridiculous to say you can't use _ because it is a valid identifier. You could just have the pipe _ shadow the identifier _ if said id is present. I suspect the bigger reason is because of underscore/lodash being in wide use.

Suppose you had "const _ = 42; /* more code / return x | f | g(11, _);". This would be semantically equivalent to "const _ = 42; / more code */ return ((_) => g(11, _))(f(x))". Since it is a new function, shadowing is not forbidden.

I'm not the one saying it, check #92 I believe it is. The bike shedding thread. They don't want to shadow.

Agree what? You can't overload bitwise with another semantic

In your example you're mutating v, which is something people try to avoid these days, as immutable data is easier to work with. There's also a good chance you can't find a good name for v that fits what it is at every step. A good part of the pipeline operator is that you don't have to name every step. Sometimes there's just no good name.

> as immutable data is easier to work with

Source? This hasn't been my experience. Going on 10 years of heavy Node.js ecosystem involvement, immutable data enthusiasts are exceedingly rare.

Plus, this is const vs let, which is a reach to call "immutable data". We're not talking about object properties.

State in React should be immutable. It’s not rare at all!

React != JavaScript.

Yeah, of course. But vast majority of React is written in JavaScript and large numbers of JavaScript devs are writing React. It shows FP and immutability have significant base of practitioners in JS world - although many might not realise it.

I would argue most JavaScript developers do not use React. Perhaps in your own bubble, but there are way more people writing other things in JavaScript.

> Source? This hasn't been my experience. Going on 10 years of heavy Node.js ecosystem involvement, immutable data enthusiasts are exceedingly rare.

Not an authoritative source, but since you cite your own experience, mine has been the opposite. Especially so when TypeScript is also in the mix.

> Plus, this is const vs let, which is a reach to call "immutable data". We're not talking about object properties.

As I mentioned in a sibling comment, const is often used to signal the intent to treat an assignment as immutable. This isn’t universal, but I understand it to be idiomatic usage.

> Source? This hasn't been my experience. Going on 10 years of heavy Node.js ecosystem involvement, immutable data enthusiasts are exceedingly rare.

Immutable.js is relatively popular and immutable records and tuples are a stage 2 TC39 proposal. From what I've seen, immutability is mostly popular on the frontend, probably because functional programming had a huge impact there. It's also popular in other languages (FP or FP inspired generally, so OCaml, Haskell, Scala, Elm, Elixir, Rust) that are relatively close to the web ecosystem.

> Plus, this is const vs let, which is a reach to call "immutable data". We're not talking about object properties.

I'm not sure what you mean here.

A variable being mutable (let vs const) is not the same as the data referenced by that variable being mutable. There are languages that favor immutability (Elixir, for instance) that permit rebinding of local variables.

- One thing not discussed in the article is type safety. But TC39 increasingly considers the impact of proposals on typed extensions to JS like TypeScript/Flow. In your example, v must be a union type of the return types of each of the called functions.

- The pipeline operator can be used to assign to a const, to prevent further assignments. While const isn’t necessarily immutable, it is often idiomatically used to signal that the assigned value is intended to be final. The same can’t be achieved with your example without wrapping it in an IIFE, which is (IMO) much harder to follow than the pipeline operator.

- More subjectively, it (in the F# variant) encourages better interfaces. For most use cases where piping would be appropriate, it’s generally a good idea to make the data you’re processing the last parameter. This also, as the article notes, works well with partial application. Which in turn also encourages good practices like composition.

I think your first point is the only compelling reason (for me) yet I also hold the stance that TC39 shouldn't cater to corporate projects such as Typescript as they are not maintained by the same committee.

Those are my opinions, however.

They aren’t catering specifically to TypeScript (or Flow), but rather to increased interest in type safety across the community generally.

It’s worth noting though that the TypeScript team does participate in the standards process. And Microsoft has two members on the committee.

I do agree that they shouldn't cater to TypeScript or Flow or anything specifically, but I appreciate that they think about the impact of new changes on typing JavaScript. I find in general that code that can easily be statically typed is easier to understand and maintain.

Exactly this! And the inverse—where JavaScript devs complain that TypeScript types are overly complex and verbose—is just a symptom of the same principle. Those types are complicated because your interface is complicated! If you had a clear data flow/model the types will usually end up being much simpler too.

This is somewhat less true in some frontend projects, where the underlying libraries/frameworks come with infectious built-in interface complexity (side-eyes React/JSX even though I quite like JSX otherwise).

> And the inverse—where JavaScript devs complain that TypeScript types are overly complex and verbose—is just a symptom of the same principle. Those types are complicated because your interface is complicated!

Part of it may also be due to the limitations of TypeScript. Structural typing is not always the best choice. ADT support is also not the best.

I definitely have wants for the TS type system, particularly in the area of nominal typing. But it’s usually not this IME. It’s usually that untyped JS often is written in an overly permissive style where anything goes, or where state and data flow is a big ball of mud, or both.

The advantage is that you don't have to name things. "v" is a bad name in a larger context, and it quikly becomes unwieldy if you use a full name.

    let transformedData = transformFoo(data);
    transformedData = transformBar(transformedData);
    transformedData = transformBaz(transformedData);

 vs.

    const transformedData = data
      |> transformFoo
      |> transformBar
      |> transformBaz

If you want have const's the first becomes even more unwieldy, and in general I'd recommend using const by default.

Just like lambda syntax makes it much easier and readable to write small functions, the pipe operator make it much easier and readable to write chains.

> Just like lambda syntax makes it much easier and readable to write small functions, the pipe operator make it much easier and readable to write chains.

succinct =/= more readable.

Readability here is really in the eye of the beholder...

It's certainly quicker to type, but that's not a good argument enough to make such a drastic change in javascript.

> It's certainly quicker to type, but that's not a good argument enough to make such a drastic change in javascript.

I am so glad this argument has not been winning out in the JavaScript language design community. There are so many wonderful constructs that have been added in the last 20 years that are just sorter ways to write things that could mostly already be done in the language. Many of these features focused on allowing developers to write what they intended to do without having to explicitly construction the intermediate steps to express the mechanics of how that should be carried out using basic language features.

After all why do we need arrow functions when we could just sprinkle our code with `var self = this;` declarations for anonymous functions to bind to, the way it was done in the late 90's and early 00's.

I'd much rather JavaScript adopt succinct ways to write common code patterns and let the developers and the community decide if those constructs are the correct ones to use on a case by case basis.

> I am so glad this argument has not been winning out in the JavaScript language design community. There are so many wonderful constructs that have been added in the last 20 years that are just sorter ways to write things that could mostly already be done in the language.

Beware of survival bias, though. There have been many proposals that aren't part of JS today.

I find |> annoying to type. How about making it so unary functions can be invoked like this?

  arg function_name

Then instead of

  arg |> func1 |> func2 |> func3

you could simply write

  arg func1 func2 func3

It might get a bit ugly when used with arrow functions instead of named functions, and there might need to be a special case when arg is a function, but if those problems could be handled not too unreasonably this seems a fine approach for a programming language.

I would love the pipeline operator in Ruby, but unfortunately the unstable feature [0] was removed:

    1.. |> take 10 |> map {|e| e*2} |> (x)

> After experiments, |> have caused more confusion and controversy far more than I expected. I still value the chaining operator, but drawbacks are bigger than the benefit. So I just give up the idea now. Maybe we would revisit the idea in the future (with different operator appearance).

[0]: https://bugs.ruby-lang.org/issues/15799

> little hard on the eyes

Understatement of the year, that's the worst token I've seen in all my years of working with javascript and I hope it doesn't go through.

I don't get one thing.

if pipe() works for RxJs, and hackpipe is not better, then why don't you simply stick with pipe()?

Just because one operator is there, doesn't mean you have to use it.

JS classes are notoriously under powered, so much that both react and vue prefer functions over class components

Besides, it is possible that RxJs (and other libraries) may modify how they work to better utilise hack pipe

I had never seen Hack's pipeline before, which uses an explicit marker ^ to indicate which argument the piped object goes to. This seems like a really cool idea--you can wire functions together any way you want without having to create lambdas. Do any other languages use anything like this?

Mathematica has `f[a,#,b]&`, for example, which is syntactic sugar for `Function[{x}, f[a,x,b]]`, i.e. the function that takes one argument and calls `f` with that argument as the middle parameter and `a, b` as the first and last parameters. It's not quite what you're referring to, but it's similarly an extremely lightweight syntax.

Tidyr/dplyr (R) can do something very close to this but I don't know how it is achieved. There it is a dot

  y %>% f(x, ., z)  ===  f(x, y, z)

R lets you do macro-ish things at at runtime [1], and `.` is a valid variable name [2]. so %>% can just evaluate the AST of its right argument in an environment with `. = a`.

(it's probably a bit more involved because %>% also supports

  a %>% f(b)  ===  f(a, b)

in which case %>% has to do some AST surgery to splice another argument in front of `b`.)

---

[1] https://en.wikipedia.org/wiki/Fexpr

[2] think `_` in python etc. R uses dots instead of underscores

Clojure has the as-> threading macro (probably borrowed from another lisp) where you can specify the placeholder:

  (as-> [:foo :bar] v
    (map name v)
    (first v)
    (.substring v 1))

FYI, you can achieve the same thing with F# pipelines + partial application proposal [1]

  input |> func(?, 2)
  input |> func(1, ?)

While hack's placeholders work exclusively in pipelines, the partial application proposal can be used with any function

  "123"
    .split("")
    .map(parseInt(?)) // make sure radix param is not set

[1] https://github.com/tc39/proposal-partial-application

scala, with its magical underscore. Although it likely desugars to lambdas (or even anonymous classes lol, given how java works)

Actualy this whole debate should be called scala vs F# because they are more corresponding, being both functional language on a OOP runtime.

There is pros/cons to both approaches. The Pipe operator in F# works in conjunction with the HM type inference in the language and static functions often assisting the type checker. i.e. features that work together in a language vs tacked on. F# idiomatic code avoids a lot of virtual dispatch seen in Java/C#, etc as compile time polymorphism is typically how re-use occurs vs objects. I find F# favors compile time abstractions a little more than Scala historically although I note that could be changing.

For most of these proposals it is just syntactic sugar around invoking the function with a placeholder. The pipe in F# is slightly different in that most of the time it has no intermediate lambdas and is compiled in a more performant form. The downside is that unless you allocate that lambda yourself it has to be the last arg. On the plus side like most things in F# it is concise but shows all behavior explicitly - its usually a warning sign if you have to do that and helps reason about performance. Sometimes its a sign that the original function isn't written right or some other inline wrapper could be used for other usages of the function. You can inline a template to rearrange the arg's (inline) to avoid the lambda allocation as well.

This allows some level of re-use and performance benefits avoiding virtual table dispatch and a lambda allocation for many common methods (e.g. map, fold, iter, etc)

> The pipe in F# is slightly different in that most of the time it has no intermediate lambdas and is compiled in a more performant form

I believe the point made in the discussion is that the F# proposal leads to intermediate lambda if the function does not have the right arity. JS also don't do HM so that point is moot.

The people supporting the Hack proposal made the point that their expressions do not desugar to lambda and has no runtime cost, no idea how it works though.

cons miss the most important downside of F# proposal which is the runtime optimization aspect of requiring extra lambda functions.

I went through the github issue the other day, syntax aside, the two proposal boil down to functions vs expressions (no extra functions)

Reply on news.ycombinator.com