Hacker News Reader

Comment by insulanian | Hacker News Reader

My code is full of map, filter, reduce/fold and similar generic reusable functions.

How do people deal with such things in Go? Do they really make copies of such functions for every type they're working with? (And by type I don't mean just int/string, but all model entities/classes.)

View on HN

map and filter are often just syntactic sugar for a loop, so I just write a loop.

I work on a ~1M LOC codebase in Go and it's really not a problem. map and filter would not make my life significantly easier. They're solving easy problems.

Sure, I have some of this style code:

    var names []string
    for _, m := range machines {
        names = append(names, m.Name)
    }

But, really, is that so much worse than this?

    names = [m.Name for m in machines]

Sure, it's spread across a few more lines, but line returns don't cost extra money... and if you decide you want to later do something more inside the loop for each machine (default the name, cache the ID, etc), you can do that trivially by adding lines inside the loop.

This code is not hard code to write. If you're used to just being able to slap out map/filter etc in a single line, I could see how it could be annoying... but it's easy, just write it. There are far more difficult things to figure out in our jobs, why worry about the easy stuff?

>> But, really, is that so much worse than this? names = [m.Name for m in machines]

You can take that code and interpret that as database query, like C# LINQ.

you frist example is 'how' vs 'what' of second example. Once you stop telling the computer how to do things and just tell it what you want all kinds of things become possible.

But then you have no idea what the computer is actually doing. There's a big performance difference between an in-memory loop and querying a database. This is one of the things I like about Go... what the computer actually does in response to any random line of code is pretty obvious (except for function calls, which of course can do anything). When you hide away the loop inside a map statement, you get people doing dumb things like this:

    names = [m.Name for m in machines]
    ids = [m.ID for m in machines]
    addrs = [m.Address for m in machines]

So now we're iterating of over the list of machines 3 times... or making 3 database queries or whatever.

>But then you have no idea what the computer is actually doing.

You trust it the same way you trust go compiler to do the right thing when you give it code to compile.

That's the nice thing about Go code... I almost always know exactly what it'll make the computer do. I know how much memory will get allocated, what the likely CPU usage is going to be, etc. The abstraction between the code and the computer is low, which helps lets a lot in understanding why your code is slow, or why it's producing a lot of garbage. "oh hey, here's a loop in a loop... oops, N^2 time".

What the computer actually does is dramatically different between reading a register and reading a block of a mapped file, but I value concisely expressing the intent rather than elaborately repeating mechanics that might change (e.g., I could optimize machines.__iter__ without rewriting all the statements).

It's worth noting you'd probably want to write this out in Go as follows:

  names := make([]string, len(machines))
  for i, m := range machines {
      names[i] = m.Name
  }

append() is pretty costly, and should be avoided when possible for performance. This is actually relevant to your how vs what concern-- on the one hand, we could trust the compiler and the builtins to always take the most efficient approach, and only program 'what' we want. That could avoid something like using append in a loop where you know the length of your intended output slice. On the other hand, if there are edge cases which the built-ins take into account that we don't care about and want to ignore, being able to and having the intention of programming 'how' can have performance benefits.

I like writing 'how' because I get to benchmark and test different approaches to solve the same problem, whereas 'what' might take a usually-fast approach that isn't a good solution in our specific case.

Yeah, true. And in production, that's what I do. Append is for when you don't know how long a slice will end up being.

It's not about how hard or easy the code is to write, at all.

Any line of code you don't write, is a bug you didn't write either. I worked on a significantly size go codebase and the number of stupid bugs that popped up, like bad loops instead of a simple `filter` function, were just silly.

Having all this memory safety and other stupid bugs ruled out but not map, filter, or reduce just felt lazy on the Go authors part.

Of course, now that a member of the go team has said it, we'll see a huge shift in what the go team thinks about generics. I'm glad I don't deal with the go community anymore.

Yes, that's so much worse. At least to my eyes.

It's a death by a thousand cuts. That's 3 dense lines that you need every time you map across a collection.

Which obscures whatever else is happening in the rest of the method/function, so now you've got that much more cognitive load to figure out what it's actual core purpose is.

Which may be subtly modified by the writer purposefully, but you miss it because you assume it's the same "it's just map idiom".

Which may be modified by the writer accidentally, but you miss it because you've trained yourself to parse it as a lump and take its correctness for granted.

Actually readability at the point of use is very reader-friendly:

    machines := db.GetAllMachines()
    sort.Sort(byName(machines))

For those not familiar with Go, the sort.Sort line is converting the list of machines into a type that matches the interface that sort.Sort expects. The list of machines is sorted in-place

The only part that can be subtly changed is the Less function, which determines the sort order. And as I've said elsewhere... that definition of sort order is something you have to define in any language (for non-trivial types).

What does the implementation of `byName` look like? I'm not sure I understand what its return type would be.

Either sort.Interface or a type that implements sort.Interface

byName is a type. It's a named type based (likely) on a slice of machines. The byName type implements the functions necessary to support the interface that the sort.Sort function requires:

    type byName []Machine
    
    func (b byName) Len() int { return len(b) }
    func (b byName) Swap(i, j int) { b[j], b[i] = b[i], [b[j] }
    func (b byName) Less(i, j int) { return b[i].Name < b[j].Name }

sort.Sort takes an interface type that has the methods Len, Swap, and Less, as defined in the signatures above, and uses them in a sorting algorithm which sorts the values in-place.

Does all of that have to be implemented every time you want to sort by a new predicate?

If so, that seems like quite a lot of boilerplate, no?

Mostly, yes (you can finagle your way out of rewriting Len and Swap if all you need to do is change the Less function, but it's probably not worth it from a code clarity point of view).

In a million lines of code, this costs us approximately 114 extra lines of code beyond the minimum necessary for any language where you need to specify a sort order (assuming you need at least one line of code per type/sort algo to tell the computer how to sort your random list of objects).

So, it depends on what you mean by a "lot".

And also you don't have to remember all the different functions, only a for loop to learn.

Reply on news.ycombinator.com