A generically typed pipe function in TypeScript | Hacker News Reader

A generically typed pipe function in TypeScript

github.com

137 pointsupzylon51 comments

When programming in a functional style, quite of often I find myself wanting to rewrite nested function calls as a chain. Hopefully at some point the proposed pipe operator will make it into JavaScript and TypeScript but for now defining a pipe function will have to do. The function should allow rewriting something like

  double(square(half(2)))

  pipe(half, square, double)(2)

For a while now I've struggled to implement the type definition for that function, so that every passed-in function can only accept the return type of the previous function as its parameter and the resulting function will take the arguments of the first function as its parameters and return the type of the last function.

The main problem with this is that trying to define a type for pipe's arguments up-front would require typing them as an infinitely-recursive type that TypeScript cannot handle. A common workaround for this is to define pipe's type separately for every number of arguments it can take. This is for example how RXJS defines its pipe function: https://github.com/ReactiveX/rxjs/blob/f174d38554d404f21f98a...

Other common solutions are to make concessions like requiring all functions to have the same return type or letting pipe only take one function at a time and returning an object with methods to add another function and invoke the chain. None of these solutions are satisfying in my opinion.

I think I've finally found an implementation that fulfills all these criteria. The argument and return types of passed in functions are correctly enforced (no matter the number of passed-in functions) and the pipe function returns a function that accepts the arguments of the first function, invokes all functions in turn with the result of the last, and correctly returns the type of the last function of the chain. Including asynchronous functions in the chain works to: if a function returns a promise that promise is resolved before being passed into the next function and the function returned from pipe will return a promise as its type.

There is one disadvantage to the implementation that I'm aware of: When passing in anonymous functions, the types of their arguments can not be inferred if they aren't annotated. That means that

  pipe(() => 10, n => n.toString())

would return the type

  () => any

but I think that's an acceptable tradeoff because when annotated

  pipe(() => 10, (n: number) => n.toString())

it will return the correct type

  () => string

Thought the implementation might be worth sharing here in case it's useful to someone else and because it's an interesting problem to solve in TypeScript's type system.

If you have any suggestions on how to improve the function's typing or know of any better implementations, I'd appreciate it if you would let me know!

Edit: I meant to link to the pipe part of the readme, but I see the link is just to the repo, that's unfortunate.

Here is the relevant section of the readme: https://github.com/MathisBullinger/froebel#pipe

and here the implementation: https://github.com/MathisBullinger/froebel/blob/main/pipe.ts...

An easier alternative is to wrap the value into an array, then use `.map()` for each function in the chain, and finally escape the value with `[0]`

I made a similar data structure[1] to allow adding side effect (no return value) as part of the chained function.

[1] https://github.com/beenotung/tslib/blob/9f9a9274c1e13be7ba83...

This works great for small datasets, but often you’ll have to eventually use streams rather than arrays. I’ve used pull-stream to do this in the past.

.use is a nice quality of life feature!

If you extract this `Chain` class to a separate package, I would gladly reuse it in my applications. I've been thinking about creating a `Vavr`[0] clone in typescript, as I really like the syntax used in that library, especially the `Try` construct.

[0] https://www.vavr.io/

This is a very elegant solution that some languages support (including the one I'm working on :) ): https://en.wikipedia.org/wiki/Uniform_Function_Call_Syntax

One downside can be namespace pollution, but imo it's worth it a lot of the time

You might find pipe in IxJS worth a look:

https://github.com/ReactiveX/IxJS/blob/master/src/asyncitera...

https://github.com/ReactiveX/IxJS/blob/master/src/iterable/i...

This is quite elegant!

Direct link to the source of OP's pipe function and recursive type definitions: https://github.com/MathisBullinger/froebel/blob/main/pipe.ts...

On a related note, I've been frustrated by the slow progress on https://github.com/tc39/proposal-pipeline-operator - specifically the thread in https://github.com/tc39/proposal-pipeline-operator/issues/91 which has 668 comments over 4+ years and shows no meaningful sign of consensus.

The TC39 group is (justifiably) very concerned about backwards and future compatibility, and Typescript has a policy of not introducing syntax that is in scope for Javascript itself until the syntax has formally reached a stable state (see: https://github.com/Microsoft/TypeScript/issues/2103#issuecom...) - so we're far from having a pure operator for this.

But `pipe((n: number) => n.toString(), (a: string) => a+' ')(3)` is as clean as I've ever seen it get.

And to use it in an ad-hoc way for left-to-right readability (and for code that will be maintained by those who think curry is just a tasty dish), it's trivial to implement an applyPipe on top:

    applyPipe('foo', strip, title, (s: string) => `${s}: bar`)

OP - it would be great to have that, or something named slightly better, out of the box!

in the Python library `returns`, `applyPipe` is called `flow`, and I agree that it's a very handy utility to have on top of a pipe.

Thanks for linking the code, I meant to link to the pipe section of the README but obviously botched that and can't change it now :)

I was really exited about the pipe proposal and actively followed the discussion like 4 years ago. But with every passing year of no progress my excitement is slowly dying.

An `applyPipe` function is an interesting idea. My only concern with it is that the first function in the pipe could have more (or less) than one argument. And in that case how would you know where the arguments stop and the functions begin? Maybe the first parameter should be an array of the arguments. Or applyPipe could be of the form applyPipe(1, 2)(add, square, whatever). What do you think? If there is interest in this I'll add it.

If I'm reaching for this form, I'm likely thinking about a pipeline of unary functions, and the TC39 proposal seems to take the same approach. Plus, doesn't every other function in the pipeline need to be unary after the first?

As a half-baked thought, perhaps callPipe would take exactly one non-function argument to start the pipeline (which would appeal to most use cases), and applyPipe could take an array of args, mimicking the call/apply duality in native JS?

Yes, every function except the first one needs to have exactly one argument.

Others have also said that `pipe` is not an ideal name for the function. So perhaps renaming it to `compose` and have `pipe` be of the form `pipe(arg, funA, funB, ...)` (like you suggested for `applyPipe`) might be the solution. Will need to think about it a bit more.

Note ‘compose’ conventionally connotes a right-to-left order, so that (f ∘ g ∘ h)(x) = f(g(h(x))).

Is there an implementation where ‘then’ is added to Object.prototype as:

  function(f, ...preargs) { return f(...preargs, this) }

And then you can use it to pipeline sync and async things together? It’s still kinda bad and seems pretty crazy though. Some obvious issues are:

- the preargs thing presumably doesn’t work for promises so you’d need to use bind.

- there’s no great way to specify ‘the function which is property foo of the object being operated on’ to e.g. map an array that is in a promise you can’t write .then(.map, …). But I think this is also a problem with other pipe implementations.

I think `Object.prototype.then` would break every source file that uses `typeof o?.then === 'function'` as a check for PromiseLike behavior.

Trying to understand this: is the lambda symbol defined somewhere globally? How can they just use it Willy nilly as a generic Stand-in for a spread parameter type?

It's just an identifier like any other defined in types.ts: https://github.com/MathisBullinger/froebel/blob/main/types.t...

It's defined as

  type λ<TA extends any[] = any[], TR = any> = (...args: TA) => TR;

meaning λ is just any function and λ<[number], string> would be the type of a function that takes a number as its only argument and returns a string.

Ok that’s what I thought, thanks!

JS (and TS thereby) allows arbitrary unicode in identifiers (variable names, class names, [TS] type names). The lambda symbol is just acting as the name of a type parameter instead of something more ASCII usual like `T` or `F`.

There's a lot of interesting discussions around it as some math libraries like to use math symbols and greek letters and sometimes even emoji as variable/class names in JS and other developers have a gut reaction that they do not like it.

applyPipe is added now: https://github.com/MathisBullinger/froebel#applypipe

It's included in version 0.21.0

Only complaint here is on the function signature. Normally a pipe operator takes in arguments first and then sequential functions. This pipe function fixes the reverse ordering of nested function calls, but the starting inputs will always appear at the end.

You've gone from:

    fourth(third(second(first)))

    pipe(second, third, fourth)(first)

when you could have done

    pipe(first, second, third, fourth)

Yes, more akin to what you’d normally call ‘compose’. Fun stuff nonetheless!

I agree. Calling it pipe adds confusion to FL understanding for newcomers.

pipe is a left to right compose. this is standard nomenclature in many similar libraries.

`compose` would actually imply

  compose(fourth, third, second)(first)

this pattern is standard in my experience for function composition. You're creating a function first, which can later be used to derive a result from an input.

I agree. This is what fp-ts's pipe[1] function does. It also has flow[2], which is basically a reversed compose.

[1]: https://gcanti.github.io/fp-ts/modules/function.ts.html#pipe

[2]: https://gcanti.github.io/fp-ts/modules/function.ts.html#flow

This is great, thanks for sharing.

I'm currently using @arrows/composition https://caderek.github.io/arrows/packages/composition/#pipe because of the light way library approach, which you seem to share.

I also try fp-ts https://github.com/gcanti/fp-ts but is a bit an academic style, so difficult to introduce in everyday work. Implementation of pipe: https://gcanti.github.io/fp-ts/modules/function.ts.html#pipe

I still need to wrap my head on your use of generics, but yours looks more flexible than the static type approach that other libraries (include RxJS) implements, does your pipe support types for any length of arguments? Does require a specific version of TypeScript?

Nice work.

Thanks! Yes, it should work with any number of arguments. Or at least in theory it does, it seems that around 47 chained functions TypeScript will give up and stops computing the type. It's written in TypeScript v4.7.2 but I didn't use any recently introduced features, so it should also work a few version back. How far back exactly, I'd need to check.

My issue is for things like select you would want to take you path first then your data last, this let's you build functions like

const selectFoo= select(['foo']);

selectFoo({foo:'bar'})

But also then allows these functions to be used in pipes

const upperFoo = pipe( selectFoo, toUpper )

This is how the ramda library does it[0]

[0] https://github.com/ramda/ramda

Remeda (https://remedajs.com/) has this, I've been using it for a long time. What's the advantage of your library over such competitors?

Um, Ramda has pipe and types: https://www.npmjs.com/package/@types/ramda. Although I liked my own implementation of a pipeP (pipe that automatically unwraps promises) that let you specify the type of the input and output of the pipeline: https://github.com/chughes87/ramdaP-ts/blob/main/index.ts#L1....

Looks like they define pipe separately for every number of arguments: https://github.com/DefinitelyTyped/DefinitelyTyped/blob/mast...

What do people think of fp-ts?

I really like it, but TypeScript syntax is a bit unreadable with it. I'm used to Scala which looks a lot cleaner with extension methods and collections being immutable by default.

That looks interesting, I hadn't heard of the project. Will definitely have a look at the library.

But it defines its pipe function the same way RXJS does, separately for any number of arguments:

  function pipe<A>(a: A): A
  function pipe<A, B>(a: A, ab: (a: A) => B): B
  function pipe<A, B, C>(a: A, ab: (a: A) => B, bc: (b: B) => C): C
  function pipe<A, B, C, D>(a: A, ab: (a: A) => B, bc: (b: B) => C, cd: (c: C) => D): D
  ...

https://github.com/gcanti/fp-ts/blob/master/src/function.ts#...

> But it defines its pipe function the same way RXJS does, separately for any number of arguments

What’s the issue? This has been a common way to do this sort of thing since C++ template meta programming. Have you honestly ever broke the bounds?

There isn't necessarily any issue with it besides maybe not being very maintainable. I just tried to find a generic solution for the fun of it

I fully recommend this. The fp-ts is quite readable for JavaScript. Combine it with io-ts and you have a robust way of handling user inputs:

    pipe(
      decoder.decode(req.body)
      , E.mapLeft(_ex => new InputValidationError(decoder.decode(req.body)))
      , RTE.fromEither
      , RTE.chain(handleInput)
      , RTE.match(
        error => {
          res.status(500);
          res.send(error)
        },
        result =>  {
          res.send(result)
        }
      )
    )(reader)

I had built an "async" version of this - https://github.com/rocky-jaiswal/async-utils/blob/main/src/p...

Basically can solve a lot of problems by passing state to a list of "piped" functions which modify the state and eventually generate an output.

Is there an easy way to debug Typescript functional constructs like this? I tried implementing this the other day and it was really hard to debug errors in my logic with Sublime's Typescript package

Impressive work, congratulations! Though overloaded definitions would be easier to maintain in the long run

Hmm, I didn't think it was actually possible to write a general and generic type for a function like pipe/compose. It's something that many statically typed languages struggle with unless they bring in the 500lb gorilla that is a full dependent type system.

I don't know the definition of "dependent type system" with enough precision to know whether or not TypeScript technically counts as one, but it has lots of powerful capabilities that allow driving the type system via provided values

I think it is technically a dependent type system, but not in the same way as say Coq or Idris.

Is this what modern idiomatic TS looks like?

Reading through the pipe implementation, as someone who primary works in the relatively "boring" world of backend languages, this looks like the kind of code that would get shot down review for being too clever and not very readable, regardless of how "elegant" it may be.

Using triple assignments (a = b = somevalue)? Using λ as a type name instead of writing out lambda with symbols that are actually easy to type? Overloading/reusing variables that makes it hard to mentally trace how data flows through?

The alternative is like 20 different type overloads and even then, it wouldn't be fully general.

Why does it matter how complicated it is if you’re not the one maintaining it? Your head would probably spin if you saw how complicated the database code you use is, as a backend dev.

Some domains require horribly complicated code that I’m perfectly happy admitting that I’ll never understand it. Chaining function calls is not one of those domains.

I suppose I don’t really care, it’s not my library and I’ll never use it, it’s just interesting to see how differently styled it is.

Chaining functions _does_ need this though, in current Typescript. The pipe function has to enforce some really complex constraints (inner types between siblings/fn args). There's really no other "correct" way to write this without calling it a day and just doing a generic for different amount of args (<A>, <A, B>, <A, B, C>, etc).

I agree it's complex, but necessary for a utility like this if you want correctness for arbitrary N args

Chaining assignments and using non-ascii characters as variable names is definitely not the norm in most TypeScript projects I've seen and worked on.

When I started this project it was just for my own personal use and kind of a creative / intellectual outlet for me where I didn't have to make any compromises and adhere to anyone else's style. So to this day some of the code might be a bit... exotic. It's definitely not how I'd write it in my day job.

That's the JS part. The type-definitions I'm afraid are just unreadable because it's TypeScript. I feel they're like regexes in that regard: write once and try not to touch them again.

Reply on news.ycombinator.com