Hacker News Reader

Comment by mbostock | Hacker News Reader

You don’t need lodash or a new operator for the example provided:

  departments
    .flatMap(d => d.employees)
    .map(e => e.salary)
    .reduce((p, v) => Math.max(p, v), 0)

That said, the pipe operator is nice because it allows similar chaining patterns on subjects that are not arrays, as shown in the proposal’s README: https://github.com/tc39/proposal-pipeline-operator/blob/mast...

View on HN

That's all well and good ( increasing readability ) but the problem remains that each step has to finish before the next step can begin... sometimes the entire dataset won't fit into memory/machine/whatever...

More useful, IMHO, would be a way to EASILY compose a true pipeline:

  const
    _pipe = (a, b) => (arg) => b(a(arg)),
    pipe = (...ops) => ops.reduce(_pipe)

...but have the behavior work like unix pipes ( a stream ), nodeJS supports this concept at it's most basic level using the pipe() abstraction, although you have to supply methods which handle being pipe'd to, and from... an example:

  const crypto = require('crypto');
  // ...
  fs.createReadStream(file)
    .pipe(zlib.createGzip())
    .pipe(crypto.createCipher('aes192', 'a_secret'))
    .pipe(reportProgress)
    .pipe(fs.createWriteStream(file + '.zz'))
    .on('finish', () => console.log('Done'));

*ripped from: [source](https://medium.freecodecamp.org/node-js-streams-everything-y...)

Imagine reading a 100gb json file line-by-line via ajax on the client, and feeding into the pipeline of transformative methods -- iteratively introduce data in one end of the pipe, and gathering the results at the other end, and creating some visualization like a graph or whatever... without ever having to have the entire thing in memory at once...

Have you seen https://github.com/labs42io/itiriri? It does lazy queries on iterables, like IEnumerable from C#.

    import { query } from 'itiriri';

    function* fibonacci() {
       let [a, b] = [0, 1];

      while (true) {
        yield a;
        [a, b] = [b, a + b];
      }
    }

    // Finding first 3 Fibonacci numbers that contain 42
    const result = query(fibonacci())
      .filter(x => x.toString().indexOf('42') !== -1)
      .take(3);

    for (const e of result) {
      console.log(e);
    }

    // outputs: 514229, 267914296, 7778742049

I wrote a lib that does this too. It's been a while, but using generators tended to be way slower than just using arrays, except in the most obvious cases (array of 1000000 elements, only take 5, no sorting involved, etc). Maybe that's changed. It's been a while since I've checked.

Realistically, there's always a few functions I want to call on my data that aren't part of the prototype of the thing I'm transforming. And then this pipe operator is called for.

Good point. Is flatMap already supported by browsers or is it a proposal?

Both flat[1] and flatMap[2] are unsupported in Edge, IE and Node.

1. https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe...

2. https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe...

Still a proposal and experimental, stage 3.

Reply on news.ycombinator.com