Hacker News Reader

Comment by kubb | Hacker News Reader

Go people will do this and they'll be content:

  a := []int{1,2,3,4}
  it := slices.All(a)
  it = slices.Reverse(it)
  it = slices.Map(it)
  it = slices.Filter(it, func(i int) bool { return i % 2 == 0 })
  slices.ForEach(it, func(i int) { fmt.Println(i) })

I don't judge the Go enjoyers, but I prefer writing TypeScript to Go which says it all.

Type-inferred arrow lambda for function arguments would go such a long way in making this code nicer... And not make compilation slower at all.

  it = slices.Filter(it, i => i % 2 == 0)
  slices.ForEach(it, i => fmt.Println(i))

View on HN

No, Go programmers would write

    a := []int{1, 2, 3, 4}
    out := []int{}

    for idx := range a {
        val := a[len(a)-idx-1]
        if mappedVal := Map(val); mappedVal % 2 == 0 {
            out = append(out, mappedVal)
            fmt.Println(mappedVal)
        }
    }

In modern Go I might write a reverse iterator, that index is a bit hairy, which would cut out the 'val :=' line, but as there is not yet a standard library option for that I'll leave it out.

Many Go programmers would claim this is cleaner and more understandable, unfortunately I'm not one of them.

The bigger problem is that it's not efficiently composable. If your caller needs to additionally filter `out`, that's another loop with a copy.

The proper term is deforestation https://en.wikipedia.org/wiki/Deforestation_(computer_scienc..., and I have seen Go libraries which do this

ah, that's what it's called! I did this in my compile-to-Go language, but I didn't know the name for it: https://github.com/lukechampine/ply?tab=readme-ov-file#suppo...

In most imperative languages, writing .Map().Map().Filter().Map() is another full copy for each call anyhow. As a sibling notes, it is possible to "fix" that, but the fix is not even remotely free. It is quite complicated to do it generically. (Although there are other benefits; the most natural approach to that largely fixes my grumblings about refactoring I mentioned in another post.)

Plus, in a for loop approach, it is not true that the caller may need another loop with a copy. They may just loop over the result and skip over the things they don't need. They only need a copy if they are going to pass that on to something else.

A drum I can not stop banging on is that you can not just take a neat technique out of one language and slam it into another without examining the end result to make sure that you haven't murdered the cost/benefits tradeoff. You can slam together all the maps and filters and reduces you want in Haskell, and applicatives and monads and all the fun, due to a combination of the laziness and various safe optimizations like loop fusion. In an eager context that lacks loop fusion, going .Map().Map().Map().Map() has radically different performance implications. For instance, "take 10 $ map f list" in Haskell will only call "f" 10 times. .Map().Take(10) in most implementations will create the full array, however large it is, and slice ten off the end after that.

In imperative languages, contrary to frequent claims from the functional programming crowd, for loops are actually often better in practice. The solution to their pathologies is to be aware of them and not do them. But it is far, far easier to get good performance out of a for loop in an imperative language than to contort one's code into a pale parody of functional programming.

> In most imperative languages, writing .Map().Map().Filter().Map() is another full copy for each call anyhow.

Which imperative language? In Java and rust, two languages I know, all these operations are lazy until the final collect. So no copy is made

Nor in Python. Nor in C++20's std::views. The very point of iterators us to proceed by one element, avoiding intermediate copies of collections.

One case where the for-loop would be much more efficient is a simple transformation like incrementing each element of an array, or adding two arrays element-wise. A serious C compiler could unroll such a loop into several vector instructions which work on several elements at once. Maybe even LLVM can recognize something like that in Go code.

That's why I specified the code. If you're writing .Map().Map().Map().Map(), you are usually getting a lot of intermediate arrays.

If you have a .Collect() call, you're in the deforestation branch. This has its own issues with stressing the inliner (turning simple, direct for loops over large collections into traversals that include several indirect method calls per item in addition to the payload is pretty easy), but that's still generally better than stressing the RAM.

Rust's map doesn't operate on arrays at all from what I can see but operates on iterators directly. This is good and generally more correct. However there's a lot of languages that don't support that. Rust is generally also going to be more reliable about compiling it all away than a lot of other languages where it will be really easy to spill over what the inliner can handle. Those long compile times in Rust do have their benefits.

There's also languages that sort of split the difference, e.g., it is not that difficult in Python to use itertools and generators to correctly write something that will not generate a lot of intermediate arrays, but it is also easy to write a series of calls and list comprehensions to write otherwise equivalent code that will create a lot of intermediate arrays.

I expect as we continue to build new languages over time they're going to all look like Rust here. It's pretty obvious that conceiving of loops as iteration over some sequence is the way to go. However, that is a result that we got to precisely because of our experiences with a lot of languages that don't support it as well, or support it inconsistently, or as is actually quite common, the language nominally supports it but the ecosystem tends to assume concrete values a lot more often than it should, and all these languages are still around.

Writing in this style correctly in imperative code is more difficult than a lot of people jumping up and down about how we should rewrite all our loops as maps and filters tend to account for. It can be done, but it's often harder than it looks, in at least one of the writing and the performance if not both, and the harder it is, the more the costs stack up on the costs side, and the harder the costs/benefits analysis becomes. And I still don't like how it refactors in most cases.

> In most imperative languages, writing .Map().Map().Filter().Map() is another full copy for each call anyhow

This is incorrect. In fact, the only such mainstream language that I can think of is JavaScript. Python, Java, C#, even C++ all have proper abstractions for lazy sequences, which would get used in such circumstances - and their whole point is that they are composable.

> Plus, in a for loop approach, it is not true that the caller may need another loop with a copy. They may just loop over the result and skip over the things they don't need. They only need a copy if they are going to pass that on to something else.

The point is that your function doesn't know what the caller needs. Yet by making it an eager loop with a copy, you are making that decision for the caller.

The original version relied on no fewer than five different imported functions from a package that, if you're not already familiar with them, you would have to go read to see exactly what they do.

The updated version has just one such function.

A function gets less readable as you have to go context switch to more and more different functions to understand what it's doing.

Yes, the `slices` functions are named such that you can make a decent educated guess as to what they're doing, but if there's a problem you're trying to debug that doesn't save you from having to dive into each one.

> A function gets less readable as you have to go context switch to more and more different functions to understand what it's doing.

This is true.

A function also gets less readable every time you have to mentally translate from low-level implementation code, to a high-/human-level description of "what is this actually doing?" (e.g. `val := a[len(a)-idx-1]` - "ohhh, ok, that's iterating in reverse"). Extracting out common patterns to methods with well-known names like `Filter` or `Map` or `ForEach` short-circuits that repeated parsing.

> that doesn't save you from having to dive into each one.

99% of the time, it really does. If you can't trust something as basic and well-defined as `Map` to do what you expect it to do, you've chosen a poor library on which to take a dependency.

> 99% of the time, it really does. If you can't trust something as basic and well-defined as `Map` to do what you expect it to do, you've chosen a poor library on which to take a dependency.

The original comment said:

     it = slices.Map(it)

I understand Map taking an array and a function, as in the article. I don't understand Map taking simply an array.

Then I went to look it up, and turns out that slices.Map does not in fact exist. https://pkg.go.dev/slices

Personally I feel like this exchange is a vivid illustration of exactly my original point.

The facts that GoLang is missing such a fundamental tool from its standard library, and that it's easy to make typos when coding not in an IDE, hardly support your point.

In fact, the very fact that you originally thought of the correct interpretation despite the fact that it had been misrepresented is a great example of why common shorthands are useful.

I suppose there are different sorts of programmers who prefer different levels of guardrails pre-made for them.

Some people consider `Map` to be a fundamental tool. Like the original commenter mentioned, some people also prefer Typescript to Go.

If I was interviewing someone for my (primarily golang) company and they mentioned that not having `Map` to be a downside of Go that hindered its readability, that would be a strong mark against them. I would not want to try to dig through the N levels of abstraction they would create in the codebase to hide what was happening with all its bugs.

Other companies don't mind this, and use other languages, like Javascript or the latest versions of Python.

Python in particular is great for what you mention, as nearly every slightly common programming idiom has its own keyword with its own special usage, so you can always use that instead of writing the fundamentals yourself.

Personally I hate Python because I prefer not to have to learn a dozen new obscure tools and keywords with each minor language release. I dislike trying to read packages written in Python, because they use whatever 20% subset of the language the author was aware of. I like Go because I can look at the source code for nearly any function in nearly any library and understand it immediately. Nobody would import "leftpad" in Go.

Different languages for different people.

I really like how you've presented this (despite disagreeing as a point of personal preference on almost every point you make :) ). I'm baffled by the level of repetition and verbosity that Gophers seem to prefer - but I also can't deny that they've created some truly astonishing tools and projects, despite having to reimplement core concepts over and over again! Clearly, it works for them.

As you say, different languages for different people. Best of luck to you, and thank you for an insightful and civil discussion :)

Go programming style is like programming in 2005 before functional programming because widespread (Google Guava, Clojure). If you showed functional programming to programmers in 2005 many of them would find it difficult to read.

I think the killer case for this is not hiding the computational complexity of mapping/reversing/etc.

I write a lot of typescript and golang and I often notice in typescript my chains end up iterating the same array N times when it couldbe been done in one smarter iteration.

I don't think I care though -- both styles are way more than performant enough. You can be more careful about it in cases it is not.

Iterating an array 10 times doing 1 operation on each element each time is equivalent to iterating it 1 time while doing 10 operations on each element.

This isn't true outside the theoretical best case. Definitely not true in most interpreted languages. Consider the performance difference of .reverse() (in-place or copy) vs using a descending for loop. Generally with interpreter chaining there are more "hidden" allocations/operations.

You're comparing apples and oranges. Of course modifying an array or copying it takes way more time than looping. The comparison was not about that, it's about one loop with 10 instructions VS 10 loops each with a single instruction, for the exact same instructions.

There's no way to do this (a single instruction loop vs 10 loops with a single instruction) in JS or Golang.

I don't mean assembly-level instructions, just general instructions in the language:

  for i := range arr {
    foo1(arr[i])
    ...
    foo10(arr[i])
  }

  for i := range arr {
    foo1(arr[i])
  }
  ... 
  for i := range arr {
    foo10(arr[i])
  }

Found the theoretical informatician... Nope, not in real world software engineering. Especially not in JIT runtimes like V8. Try it!

I tried it, code: https://pastebin.com/cA8YkE8R

Result:

   process1: 2s 251.327833ms
   process2: 1s 537.721625ms

It's a little bit faster (on my machine at least) if you combine the filter and map into a flatMap (it's still not as performant as the imperative solution though).

  function process3(input) {
    return input
      .flatMap((n) => (n % 2 === 0 ? n * 2 : []))
      .reduce((a, b) => a + b, 0)
  }

If it needs to be fast use oldskool for() loops.

  function processfor(input){
    let sum = 0
    for (let i = 0; i < input.length; i++){
      if (input[i] % 2 !== 0){ continue }
      sum += input[i] * 2
    }
    return sum
  }

https://jsfiddle.net/gaby_de_wilde/y7a39r15/5/

Yeah the comment I was originally responding to included a for loop (in the Pastebin link). My point is that if you're set on going down the functional route you don't need separate map and filter steps, you can just use flatMap which is effectively a filter and map combined (returning an empty array filters out the current value, since an empty array gets flattened into nothing).

Of course, if you want the most performant solution an imperative for loop is faster (which is what I said in my last comment).

input.reduce((a, b) => b % 2 !== 0 ? a : a + (b * 2), 0)

or if you want it really silly.

input.reduce((a, b) => a + (b % 2 && b * 2), 0)

dont ask me why but for(a of b) is slower than for(i=0;i<b.length;i++)

Array.prototype.reduce can basically do almost anything a for loop can do, since it gives you access to state from one iteration to the next. The only reason I didn't remove the flatMap in my original example and convert it all to reduce, is because there's no longer any method chaining which was the point of the original comparison between the Go and JS examples.

> dont ask me why but for(a of b) is slower than for(i=0;i<b.length;i++)

Probably because for of loops use the iterator protocol. So I'm assuming under the hood the JS engine is actually invoking the Symbol.iterator method (which is slower).

  #! /usr/bin/env node --experimental-strip-types
  
  function processIterator(input: number[]) {
    let sum = 0
    for (let i = input[Symbol.iterator](), r; (r = i.next()); ) {
      if (r.done) return sum
      if (r.value % 2 === 0) sum += r.value * 2
    }
  }
  
  function processfor(input: number[]) {
    let sum = 0
    for (let i = 0; i < input.length; i++) {
      const value = input[i]
      if (value % 2 === 0) sum += value * 2
    }
    return sum
  }
  
  
  const input = Array.from({ length: 1_000_000 }, (_, i) => i)
  
  console.time('normal for loop')
  console.log(processfor(input))
  console.timeEnd('normal for loop')
  
  console.time('iterator for loop')
  console.log(processIterator(input))
  console.timeEnd('iterator for loop')

But this is apples and oranges. Process 1 creates and calls lambdas, etc.

A proper comparison could be:

  # Case 1.
  for n in range(len(data)):
    data[n] = transform1(data[n])
    data[n] = transform2(data[n])

  # Case 2.
  for n in range(len(data)):
    data[n] = transform1(data[n])
  for n in range(len(data)):
    data[n] = transform2(data[n])

It illustrates how this kind of code is used in the wild - array functions vs a for loop. What you did is not even a fruit, if I go along with your metaphor.

Funny enough, the iterator version is much faster in rust because the compiler can more easily optimize iterator chains than custom for loops:

https://play.rust-lang.org/?version=stable&mode=release&edit...

(I'd recommend running it on your own machine as the rust playground limits memory and will likely kill this program)

Output from my machine:

    $ cargo run --release
        Finished `release` profile [optimized] target(s) in 0.05s
         Running `target/release/iterator`
    Process 1 returned 18270843109002848788 and took 64.58175ms
    Process 2 returned 18270843109002848788 and took 308.969083ms

In an ideal machine you are absolutely correct! But when executed by a real physical CPU, if the array doesn't fit I the cache, iteration strategies matters!

Not in this way, not significantly. At worse, for each new loop through the array you have an extra cache miss. If the array is large enough, 10 extra cache misses will barely even be measurable.

It's only for small arrays actually that the cost of the loop infra and the extra cache miss per loop matters.

https://en.m.wikipedia.org/wiki/Loop_fission_and_fusion

Until the array no longer fits in your cache :)

By that logic, calling any function with a loop or recursion should be forbidden, as it hides computational complexity at the call site. I.e. should we get rid of `slices.Sort` (etc.) because it hides computational complexity?

They might indeed. Or

  a := []int{1,2,3,4}
  it       := slices.All(a)
  itRev    := slices.Reverse(it)
  itMap    := slices.Map(it)
  itFilter := slices.Filter(it, func(i int) bool { return i % 2 == 0 })
  slices.ForEach(it, func(i int) { fmt.Println(i) })

No fun to write. Is it better to read? Yes, if you want to know exactly what's going on. Not really, if you are skimming through a lot of code, you're used to cruising through chains of func calls, and the above feels like speed bumps.

> I don't judge the Go enjoyers, but I prefer writing TypeScript to Go which says it all.

The functional style is fine for simple filter/map algorithms, but once you get into zipping multiple iterators together, classic for-loops are often easier.

My company's hiring coding task had this "trap", at one point you needed to write an algorithm to merge consecutive "empty" cells in a data structure that is used to represent a simple table. This is dead easy with a regular for-loop, but writing it in a "functional" style could easily become extremely hairy and/or have O(N^2) complexity.

You can use loops in typescript if you need to.

Sure. I'm just saying that the functional style is not always the best.

> it = slices.Map(it)

There is no such function as "slices.Map()".

https://pkg.go.dev/slices@go1.23.2

Nit - I think you've missed the `func` argument in the `slices.Map` line.

(But - yes, bravo, correct, you're right and you should say it)

I did, but I believe some people understood what I meant anyway.

Reply on news.ycombinator.com