Go Chainable: .map().filter().reduce() in Go
github.com
github.com
As one who works mostly in Rust and JavaScript and is passingly familiar with Go (I used it for a couple of projects eight and nine years ago), these seem some pretty severe limitations.
Rust’s trait-based iterator system is delightful, so that you can map, filter, reduce, &c. on any iterator, lazily, and even define an extension trait to define your own methods on any iterator, thereafter accessible by just importing that trait.
In the end, I think the current scope of generics and interfaces won’t be enough to produce any feeling other than “shoehorned” for this functional style in Go. It’s just not a style that works well in all types of programming languages.
I've placed some thoughts in the Readme [1] so I don't forget. Hopefully at least some of the restrictions will be dropped.
I predict that production go code will continue to use for loops for this kind of thing, even once generics are widely established.
I've only used it in passing, but everytime I see examples they're verbose and look clunky.
For example, the chainable methods are nice, but comparing to JS looks more like ES5 than modern code.
Do you find Go preferable to other languages for solo projects?
I don't prefer Go because it is verbose, error as values without constructs to manage them is a pain in the ass.
I'd use go over language X,Y or Z because of its standard library and ease of deployment. The language itself has great things but is verbosity, mainly when it comes to dealing with error C style, its absolutely not a feature.
Now that we have generics, things will get more interesting.
> Nothing is implicit and that makes good easy to read.
I mean this is a language with garbage collection, that thing certainly is implicit.
Coming from languages with exceptions, where you have no idea if a function call is going to explode without praying that the library has its exceptions documented or reading the source code, having error values that you cannot (easily) ignore is a blessing. It's not as good as something like Rust's Result sum type, but it's pretty close. It makes you explicitly handle, or bubble up, every error in your program which in my experience is a huge cause of unexpected exits in other languages.
"People who prefer Java use it _because_ it is verbose. Nothing is implicit and that makes good easy to read."
The irony.
It could be less verbose without loosing readability. My two top picks would be to add a ternary expression and a real while loop so that you could write better iterators.
while next := some.Next(); next != nil { // ... }
Such a while loop would be consistent with the if statement support for an assignment and following check.
Projects like OP's just tell me that OP doesn't understand Go, and why Go was designed the way it was. They're trying to make this cool thing from another language (that they understand and know how to use) work in Go. But if they really understood the philosophy behind Go they wouldn't be doing this (and their life as a Go programmer would be a lot easier).
But then, I had the same experience coding in Rails. I hated all the magic, the "if you do this then Rails will automagically do that". I want my program to do something because I have specifically written it to do that thing. No surprises, no hidden layers of abstraction. Nothing implicit. I spent a lot of time fighting the magic and trying to write simple, verbose code in Rails. It wasn't fun.
but let's not assume the intentions of the OP here. It's perfectly okay for someone to try and implement collections API using generics in go just to see if they could
Other than Go, I have varying levels of experience in C, C#, Common Lisp, Python, (embedded) assembler, Fortran, Tcl, PHP, and maybe a few other languages I screwed around with for fun.
At first I really really liked Go, and I grew to loathe it. I still think Go probably excels in large software development shops where some of its design decisions make a little bit more sense. For a solo project, unless you're trying to get hired to write Go professionally, I would avoid it.
Some complaints in no particular order:
- An unused import is a compile error, which means every time you comment out a variable while debugging, you also have to go fuck with your imports
- SemVer is baked right into the language, but Go simply cannot handle major versions > 1.x in any sane way. The documentation[0] actually recommends that you copy-paste your entire codebase into a separate v2/ subdirectory and then maintain that.
- No function overloading
- Annoying type/interface system
- Smug, snotty community (mostly #go-nuts on freenode/libera)
That's about all off the top of my head but it was enough to turn me away from go for good. If I was writing an httpd or something similar, as part of a large team, I might choose Go. For just about any other use case I would not, because it sucks for those use cases.
One of the main reasons we went with go for a particular project was the ease of distribution and being able to build it for all platforms. The "all platforms" part isn't actually easy at all. It's a relatively simple app but does use GTK which is a nightmare to get working correctly on Windows, and a bigger nightmare to get working with our CI system that expects builds to be dockerized. In my opinion, it's not worth it.
At least it's fast when it does run, which isn't 100% of the time since sometimes it was just exiting with no output until we set up Sentry inside the app.
I feel like it's similar to Java, where the "write once, run anywhere" thing doesn't actually apply in practice. Combine with how GTK wants root stuff handled (TL;DR you have the app that runs in the background do the actual work that requires elevated privs and run the GUI as a separate app then do IPC / expose an API) and how different platforms have different ways of running a persistent daemon + how that complicated distribution (we're at the point now where we have an MSI for InTune for Windows and a JAMF package for Mac + the binary itself just works on Linux and we can push that with Salt) and...if it had been my decision, I would have just used python or node.
As a bonus, none of the libs for accessing our $BIGCO internal services are ported to go, so for some things we can use GRPC but other stuff just sucks to integrate with. It's so so so so so much easier to just use the most-supported or at least a well-supported language within a $BIGCO for the network effect.
I've always stayed away from node because the ecosystem looks like such a shitshow from the outside, but the speed of V8 appeals to me.
Python is great for quick stuff that doesn't need to be performant. I use magic-wormhole and pgAdmin4 probably every day and they're great, but I shy away from Python for anything that needs threading.
One thing I will say in go's defense is that I found goroutines to be a fairly pleasant and straightforward experience (at least as far as multithreading can be straightforward and pleasant)
In my experience, non-type-specific function/operator overloads ultimately end up being a pain point. They’re generally wonderful —- until they’re abused, for which there’s a tendency. That’s certainly true of C++, and likely true in other languages which support overloading to varying degrees. With generics, the potential value of overloading in Go should now almost always be (hopefully) close to zero, so I think this criticism may be less notable than it was before.
> Smug, snotty community (mostly #go-nuts on freenode/libera)
I wonder if that’s not specific to those communities rather than the Go community as a whole. My general pulse on things is that the community Go isn’t less inclusive than others, but that may be a biased viewpoint.
Maybe. My experience with IRC communities, especially tech-focused ones, is that they tend to be pretty welcoming and friendly. For example #lisp and its sibling channels, on libera, are some of the nicest places around.
By contrast everyone I interacted with on #go-nuts seemed to have a huge chip on their shoulder and a quick snarky answer always at the ready for why Go is always better than $language in every way, no matter what.
That's not the documentation though. Just a blog post. The documentation [0] recommends a branch be created. Creating a directory is not even mentioned.
A great example is this .map().filter().reduce() joke we're looking at today.
Well-written Go is absolutely sublime.
Sure, you can still write gross Go. But I'll take gross Go over gross C++, gross Haskell, or gross Python any day.
To me it feels like C with some extra features, kind of like Objective-C. If you need to do some C-like things but can afford the small latency etc overhead introduced by Go, what Go provides (GC, large standard library, green threads, package management, etc) feels like absolute luxury compared to using C. C# is another GC language with the ability to go lower level, but the syntax can also be pretty verbose.
After 15 some-off years of writing code, to me, the most key component to a language is the ability to grok a new codebase written in it quickly and be able to contribute with as little fuss as possible. The potential network effects to this are huge, and Go’s level of simplicity is central to that: there’s generally only a small handful of ways to implement a solution in Go, and the built-in testing, benchmarking (and now fuzzing) are really the icing on the cake for me. That the standard library is mostly comprehensive doesn’t hurt either.
TypeScript is probably a close second for me, but there’s a fair amount of complexity involved in testing and in choosing libraries to fill certain gaps. Feels like that’ll get more and more solved over time, but with Go there are really few arguments about it.
I'd be unlikely to use a library like this in my Go code.
In all my current projects I default to starting with golang. Some things annoy me, but on the whole the benefits outweigh the annoyance.
I am very much looking forward to the pending 1.18 release - not for the generics, but for the fuzz-testing support. I've been fuzz-testing my applications/libraries for the past few years, and I think this is extraordinarily useful and I hope with a wider audience using such techniques we'll all benefit.
(golang developers already have a good culture of writing test-cases, much like Perl did back in the day with the TAP format. But fuzz testing is magical and often catches things developers didn't think about.)
return String(this.path).split(".").reduce((obj, attr) => obj[attr], this.object);
let attrs = String(this.path).split(".");
let obj = this.object;
for (let i = 0; i < attrs.length; i++) {
obj = obj[attr[i]];
}
return obj;
The other day, there were some complaints about Celery; the OP noted that just starting the application took 280MB, after which it grows to 14.5GB. I've inherited two Python web apps, and I understand the pain. My Python apps grow to about 4GB each. The Go app I've written powers two other web apps, one of which has a higher usage than the Python apps, and it hasn't grown beyond 40MB, and barely uses CPU, on a low-powered virtual server.
I don't think go is a good language. It's not enjoyable to write. It's annoying to read. It doesn't bring me happiness.
Compared to rust, it has a ton of severe flaws. It doesn't have mutex guards, so manually releasing mutexes is the norm. Every error is returned as the 'error' interface, so your compiler can't tell you what types to check for (not that there are sum types anyway), nor can the function signature, and error handling in general is a total mess as a result. It's easy to mess up ownership with a few things, especially channels. It has its share of footguns and warts. It made the million dollar null mistake. It feels like writing code while hobbled with its intentional avoidance of macros, type-level abstractions, and a number of other "features".
But, compared to rust, it has a vibrant ecosystem of fairly mature libraries. If you want to, for example, speak some slightly obscure protocol, or interact with a somewhat unusual API, you'll usually find a pure go library that some pour soul has hacked out, one line at a time. In rust, you might find a half-completed nom parser some college kid wrote started and abandoned, if that.
There are other languages with this property (of libraries already existing for anything you might want to talk to), but these other languages are javascript and python, so if you want some semblance of speed and some semblance of a static type-system, Go's a good compromise between library support and static typing.
It's because of this reason that I begrudgingly use Go, even though as a language, I find rust, (modern) C++, and haskell to all be generally more pleasant to write and read.
I have a great deal of experience (decades) with assembly, C, Lisp, ML-style derivatives (Standard ML, OCaml, Haskell), shell script. Since Go is already 12 years old now and I've been doing it since day one, I guess it also belongs in this category.
I am very fluent in Rust, Prolog, Coq, Agda, Idris, Lean, Ada, Erlang.
I even have some professional experience with C++, Java, C#, Python, Objective C, Swift, APL, R, Julia, Matlab, Fortran.
And yes, Go is my language of choice in the domains where that makes sense, like servers and command-line tools.
> For example, the chainable methods are nice, but comparing to JS looks more like ES5 than modern code.
I don't know anything about JavaScript (except that I don't want anything to do with it), but the code in question is a far cry from idiomatic Go code.
> Do you find Go preferable to other languages for solo projects?
Absolutely, when that makes sense. For example, Go is a lousy language for writing a compiler in, I'd prefer to use OCaml for that. That being said, I wrote a couple of compiler targets for Go in Go. Even though ML-derivatives are a better fit for writing compilers, there's a much greater value in having the Go compiler itself written in Go.
However, I will definitly advocate its use against C for userspace applications.
There are some drawbacks too but i guess more experienced you are in any language more issues you will discover with it but you also learn to workaround them.
I used to love Go, but that my "wide variety" of language experience was mostly all dynamic languages.
My current opinion of Go in short is:
> Gives micro-level simplicity at the cost of macro-level complexity
From what I've seen of Go, this makes sense to me.
However I'm surprised that many python fans like Go. It seems like just enough static typing to make the patterns that you are used to painful, without enough to actually describe your program.
[1]: https://www.python.org/dev/peps/pep-0020/#the-zen-of-python
If people want to write functional code they should use functional languages rather than writing non-idiomatic code in other languages.
Thankfully this style is unlikely to gain popularity in Go because they make the syntax for doing this verbose enough and ugly enough that most people aren't gonna bother with it except a top level map or filter.
I do heavily prefer Optionals however when dealing with Values that can be null. Dealing with nullity checks is annoying.
Not every program needs to be golfed down to one line.
In addition, the method-based approach scopes the variables to each individual call so there's no risk of, say, transforming the loop variable but then accidentally filtering based on the original instead of the transformed.
"I hate this. I don't know why people insist on forcing this programming style into every language. Goto and labels exist for a reason. They are simple, they work, and nine times out of ten, they are faster than this crap."
As for performance, that is going to vary language to language, but in general iterator processing pipelines are not slower and can even be faster than raw loops. Whether this is the case in Go, I have no idea.
With all of that being said, I am not sure if I would use this style of programming in Go since it doesn't feel idiomatic (yet?).
Or, a gateway drug for ECMAscripters who can't drop their `.map(x => y).filter(x => y).reduce(x => y)` habit.
Still, if you know you’ll have a small working set, and you don't what the extra deependency, there's lots of cases where bet readability, composability, and maintainability wins over efficiency.
Because being explicit about intent is important. Map means "transforming each element of the collection", filter means "keeping only some elements of the collection". With both, you only have to write what happens to one element, and it will happen to the whole collection. This makes code easier to read. You don't have to spend energy to understand if the loop is only a transformation, or also filtering. Since you chain them, you can easily isolate each transformation, which makes code easier to read and easier to test.
> Reduce itself is awful to begin with.
I don't really understand what's supposed to be awful about reduce. It's just another form of for loop. I don't see people arguing that for loops are awful. The thread you linked is just someone that's biased against reduce for some reason that I don't quite understand. Reduce makes explicit that when using a for loop and accumulating a result, you need both a base case, and what to do when going from n to n+1. Again, this allows you to isolate the "going from n to n+1" part easily. Recursion itself is great for some problems.
You mention the only worthwhile exception: sums and other 3-character operation.
If you’re using reduce to construct objects you’re just better off with a for loop. Having a random awkward named function that only makes sense when plugged into reduce is not any better than just have a function that includes the loop itself:
- newArray = array.reduce(operation, [])
+ newArray = operation(array)
The problem is that reduce can do anything and most people use it to do anything. It has no advantage over regular for loops unless you’re proficient with real functional programming (if you use forEach() in JavaScript, you’re not)
> With both, you only have to write what happens to one element, and it will happen to the whole collection. This makes code easier to read.
Filter and map are ok. Filter, map and reduce are not, at least not in JavaScript. Try writing the equivalent for loop and you’ll find that it’s just as simple to follow.