The first reply to the thread deems "map" and "filter" borderline-useful, but "reduce" not worth it - that type of misunderstading just shows that the author (and the 60+ people who +1'd it) didn't spend enough time understading functional constructs yet.
There is nothing preventing people from writing their own reduce() in their own utilities package, and I expect a thriving ecosystem of "even-more-functional-constructs" packages to spring up as soon as generics hit stable.
The discussion in this proposal is not about this. It's only and entirely about what the stdlib should provide, and that is much more a question of practical and idiomatic usage than it is about one's understanding of a theoretical CS syllabus.
I'm not arguing what's good and bad, nor what should or should not be in the Go language.
I do not understand the people who think that functional-esque programming is the only and best way to do things who are apparently just sitting and waiting for Go to be able to do it. Are you forced to use Go or something? This is an honest question, btw, and I am legitimately interested in answers. If the only reason or the biggest reason you're not using Go is that you can't import map-centric programming into it, why not use any of the many languages that support that rather than griping about a language where it's still ultimately going to be bodged on? Again, honest question, because there's no shortage of such languages.
My perception is that Go isn't that popular that there's a lot of people forced to use it against their will because it's just The Language.
Anyhow, even after generics drop, it still isn't going to be any fun. I expect to use a map here or there often enough that it'll be nice to have it in the standard library but there's no realistic way that it's going to become "the best way to program in Go". If you're waiting to try it out after the generics drop, I would suggest trying something else instead; Elixir, Nim, Crystal, Pony.
I am happy using Go right now when they don't have map.. but including map and not including reduce/flatten is just odd
Have you actually taken the time to write out what it looks like, as I have in that blog post above, though? It's not pretty. Any sane Go programmer is not going to want to use this for very much. (I write that carefully. I anticipate using it a non-zero amount myself. But it's not going to be a lot.)
I understand liking map/filter/reduce, even if it is the beginning of functional programming rather than the end. I don't understand absolutely demanding it in languages where it doesn't fit well and insisting on using it no matter the cost.
for go, a language that idiomizes variable names like `f`, you think that people would be more on board with fewer characters. There are many other areas where go has chosen to avoid tidying up the syntax/making lives easier in the name of simplicity, though I digress.
As for your earlier question of "why don't you pick another language," I think this is a silly question to ask. The vast majority of people working for hire are not picking their language. If you have a few hundred kloc written in go, I doubt very much people are in a position to casually introduce new languages, especially if they're trying to improve the current system.
It's not. It's more. A lot more sometimes, plus the indentation for trying to chain maps together is ugly.
I think more people who think this is going to be great need to clock some time on the Go Generics playground [1], implement some Map and Filter and MapError functions, and convert some real code to this style in real Go. It's awful. I'm probably spoiled by Haskell and how nice it makes all this, but it's awful even against normal Go code. Try it.
The problem is, the thing that makes "functional-style" programming in Go difficult wasn't "missing generics". It was, missing generics, somewhat heavy-weight anonymous function syntax, indentation rules not optimized for deeply-nested functions, a compiler that almost certainly won't optimize through very many layers of these constructs, the inability to put methods on core types and the fact the core types are missing the useful methods/interfaces like "Traversable", no higher kinded types so no monads or anything else like that, a variety of little syntax issues that make this all slick, and probably some things I'm missing. Just getting some generics (and not having generic methods that can add new generic types hurts this style too) just isn't enough to make it slick. Go would need a suite of changes to make this slick and efficient, not just these generics.
I’m writing a complex aggregator in go and the abstraction is buffered channels. There is no map or filter in sight. It is not needed.
I mean, yes, most companies have a relatively small list of supported languages, whether explicitly or practically. If someone at our company wants to write a program that does some binary data munging that's even more frustrating in Java, and needs to do it faster than Python allows, they have to use Go. We will not let them use C or C++, because most of the team will not understand it and it won't be able to integrate with any of our standardized libraries or CI infrastructure.
A program structured as a sequence of short, tight loops over vectors/slices as described is more or less hitting the performance sweet spot of modern microarchitectures.
The general form of a short loop over a vector plays to the strengths of branch prediction and the instruction cache while the temporal and spatial locality of the memory accesses are conducive to caching and pipelining.
Moreover, there's usually no need to allocate or copy the array in that sort of data flow. I mean, unless you're chasing worst case performance to make a point or something. Slice the buffers out of a pool and allow ownership of the data to follow the execution context, then you're free to modify it in place.
The above points cover 3-4 orders of magnitude real world performance.
It is true that there are some issues with the optimizer working in this pattern but I have only seen performance severely degraded in this pattern by compile time type ambiguity -- some but not all interface{} parameters, anonymous functions / closures in scopes with dynamic type. Interface types with a pointer receiver do not typically encounter these specific issues, and I believe the empty interface type has much better performance since ~1.12 or so as well.
Sure, but even faster is not to loop over intermediate arrays at all, by virtue of never constructing them in the first place when they aren't necessary.
"Moreover, there's usually no need to allocate or copy the array in that sort of data flow. I mean, unless you're chasing worst case performance to make a point or something. Slice the buffers out of a pool and allow ownership of the data to follow the execution context, then you're free to modify it in place."
That starts getting into "I'm sure someone can come up with some solution that meets some of these goals". Whatever map you're talking about here isn't one that is defined as creating a new slice based on mapping a function over the old slice. I mean, it kinda sounds like you're saying "well, if you just write a conventional for loop you can do all this in one pass" to me? Which is my point? I'm not the one pitching for lots of array creation, it's people who insist on using maps and filters in a language that, of all the major languages, just isn't going to put in the optimization time to convert them back into loops under the hood.
The sooner we, as engineers, can collectively acknowledge that we're doing things so wrong it's costing us approaching 4 orders of magnitude performance, and agree to stop dismissing knowledge of the hardware as forbidden knowledge, and abusing acceptance of reality as pointless micro-optimization... then I'm sure we'll make progress towards accepting one of the many patterns people have been advocating for upwards of a decade which do solve these problems.
One of the patterns which happens to be able to reclaim most of those missing 4 orders of magnitude performance is functional data flow, which is by unfortunate coincidence essentially the pattern you're denouncing here for its performance. It actually maps almost directly to the ideal implementation because it's ... literally expressing problems in terms of vector operations with no data dependence, which maps perfectly to a parallelized pipeline of the instruction types that achieve near optimal throughput and realized IPC.
I am not trying to be a dick but I am very passionate about this topic and I disagree very strongly with your assessments of the performance here.
> Sure, but even faster is not to loop over intermediate arrays at all, by virtue of never constructing them in the first place when they aren't necessary.
In those examples of this type of pipeline, the vectors you start with are slices of hardware device descriptor queues which reference a DMA region, and by progressive transformation of the receive buffers and composition with additional buffer regions through intermediate decoded states you produce reply buffers.
The hardware is programmed to sample only the packets of interest to this descriptor queue, and copies the packets matching this filter directly to the DMA region referenced in the descriptor it places in the queue.
With sufficiently sophisticated hardware it is possible to offload decode of increasingly large fragments of protocol logic or even entire applications.
Description of the operations in a language of common composable operations over the type of vectors of buffers allows the amount of any given application which is mapped to e.g. general purpose CPU, GPU or other FPGA/ASIC offload feature instructions to vary continuously over time or API surface at runtime pretty neatly, it creates breakpoints in the logic flow that more or less always map exactly to functional hardware boundaries, because everything's pretty much just vectors of buffers all the way down really when you think about it. Your process's entire runtime is just another vector of buffers to the kernel.
Go has a common style across most code bases I've seen. Partly that's because Go is opinionated about style, and doesn't really support doing things any other way. I don't see this as a bad thing, though that may well be because Go's style happens to be similar to my own. I have friends who learned to code on Ruby and JS and who don't like Go at all, because it doesn't really support the way they like to write code.
Adding the ability to write functional-ish code to Go is going to fragment the style. There will be code bases written in functional-ish Go that are difficult to understand for people used to a more imperative style. And vice versa (though I suspect imperative is easier to understand). I don't see this as a good thing. It's kinda "if you want to write functional-ish code, why not pick a language that you can do that in, instead of demanding that Go supports it?". But I realise this is probably a lost cause, and we're going to see that wave of "even-more-functional-constructs" become so common that we end up with JS-like library churn.
Probably I'm looking at it the wrong way, but it looks to me like it's a relatively harmless language extension with the added bonus of correctly typing stuff normally cast as interface{}.
The problem with generics is they're a hammer, and everything is going to look like a nail. Instead of being a rarity, and everyone basically writing idiomatic Go as normal and only pulling out the generics when really needed, this functional-ish style that relies on generics (as in TFA) will become common.
Devs who are used to writing array.map code in JS will start on Go and immediately find the slices package in the standard lib, and create a generic for every single slice they use, because that's what they're used to and how they think it's meant to be used. For...range loops will look as old-skool and primitive as for loops look in modern JS (despite all their advantages).
We already have the "learning path" of newbie Gophers, whose first question is always "which framework should I use?", and it's an uphill battle getting them to accept that they should just use the standard lib. Trying to get them to accept that they basically never need generics is going to be harder.
Old crusty Gophers like me, who are used to and like the simplicity of idiomatic Go, will be left shouting into the void. Again.
edit: typing stuff as interface{} is also one of those things that you learn to get past. I'm working on a ~80Kloc Go code base at the moment, and have not had to resort to interface{} once yet. Interfaces are way more powerful than they appear at first. I'm not saying interface{} doesn't have its place, and it's used heavily in the standard library, but it's not as common as you'd think at first glance.
The two big places this shows up in golang, imo, is the lack of high quality data structure libraries, and the lack of widely used standard library for routine channel management tasks.
The former is a huge opportunity cost. If I'm coding on .net, jvm, c, or c++ I have a variety of state of the art data structures available. Some of these structures are extremely difficult to develop, so the "just write your own boilerplate" approach is completely insufficient. Point to code generation templates all you want, the reality is that this problem is preventing golang from improving much past using mutex protected hash maps and google's interface{} typed btree. Meanwhile, if we look at say .net, you have a very high quality lock free concurrent hash table available that scales to massive heap sizes. This is no niche thing: the bw-tree / cosmosdb storage engine is entirely designed around the capabilities of this lock free map. It's easily responsible for 100's of millions of dollars of value to software in that ecosystem now.
The lack of simple, clear, and useful channel lifetime management functions is another rough edge that bites many go programmers daily. The many abuses of Context make this problem quite clear.
I understand the sentiment of not wanting golang to turn into the Stephanov style C++ generics. But I think that's an imaginary fear. The golang generics design is quite different and far more "gopher" for lack of a better term. The problems golang is facing due to poor type flexibility in this area are not imaginary: they are endemic.
And I get that Go lacks in some areas. That's fine. All languages lack in some areas. Go wasn't designed to be all things to all people. It's very good at writing servers in. And there's a few projects written in Go that have proven to be pretty popular and scaled well ;)
It's not really YAGNI, which Go has been accused of (a lot), with some justification. It's more that simplicity is valuable, but it has a cost. Keeping Go simple is good, but the price of that simplicity is sometimes high. As you say, there is an opportunity cost.
I still think that cost is worth paying, and I'm not convinced (yet - I may be in time) that the reduction in simplicity that comes with generics is a price worth paying for the flexibility it brings.
I really hope this doesn't happen because then you have random utility libraries that "infect" projects for what is essentially basic flow control. I would say stuff like this is best supported in the standard lib and not home rolled.
It’s not very charitable to assume that people who disagree with you must not understand your position. In particular, one person could like reduce() because it’s very abstract and can be used for lots of problems, but critics could dislike it because very high levels of abstraction impede readability. Both parties may understand each other, but they are optimizing for different things (abstraction for its own sake and readability, respectively).
I do think this is about understanding, as it's not about syntax, or what feels nicer. It's about building blocks and what comes before what. Given the nature of the proposed library, to give the users an arbitrarily less abstract construct is a design mistake as I mentioned in another comment in this thread.
For example, I remember when I needed to flatten a bidimensional array in JavaScript, perhaps back in 2005, and the internet told me that I needed to flatten the array. So I learned then that array.flatMap() served that purpose: to flatten a bidimensional array. This is of course incorrect, but it took me many years until I found the concept again and understood what were the building blocks around flatMap, and what kinds of other abstractions it allowed.
Per my earlier comment, it's only "a design mistake" if you assume that it's always better to trade readability for abstraction. This isn't always the case, and in professional software engineering it's often better to optimize for readability--rarely are high degrees of abstraction beneficial (of course, someone will respond to this with some anecdotes of the instances in which high degrees of abstraction are helpful, which is just an enumeration fallacy).
I don't think the aim of the package is to move Go towards a more functional style. Rather, it's to provide a set of utility functions that mesh with the style of existing Go code. For example, it would be useful to have a utility to 'pop' an element from a slice, even though that's not a particularly functional idiom.
You can see this clearly in job listings looking for Go coders. Almost nobody is looking for junior level Go coders with less than a year of experience. Companies are looking for Go coders with at 2, 3, 5+ years of experience instead.
A simple syntax may provide benefits, but it doesn't seem to be the benefits most companies want in practice after all.
I have never seen a job posting asking for less than a year of experience with anything.
Despite our wishes around that, functional constructs do have a hierarchy of abstraction vs power, and skipping a layer is the type of design that backfires in the end, like with Java doubling down on the collection APIs without introducing monads explicitly for users.
I'm not necessarily disagreeing with you, but how has that backfired for Java? It seems as popular as ever, and I don't hear many people lamenting the lack of monads in Java.
I meant specifically the lack of flexibility in some regards because of the missing of monads and friends.
For example, we have a few newer languages, TypeScript and Kotlin come to mind, implementing the "null operator":
thing?.fieldA?.fieldB
If the language designers had chosen to take one step back and implemented it in a more generic way (in some sense, in the same as my other comment in this thread about implementing "map" without "reduce"), the feature could've still been implemented in the language, but without the limitations.
I'm not sure what you meant with this point:
> people lamenting the lack of monads in Java.
Do you mean that, because you don't see people lamenting, it's not a useful feature?
But monads don't compose with each other, do they? I thought you needed something more (monad transformers, algebraic effects, etc.) to get things to compose. Or maybe you meant something else by composition.
> I can't use this for my own abstractions
Yep. But Java doesn't even let you overload addition afaik, so being able to overload ? is pretty much a pipe dream.
> Do you mean that, because you don't see people lamenting, it's not a useful feature?
No, it's just that if something "backfired" for Java, I'd expect to hear people complaining about it.
They do if they're[0] traversable (cf literally `Data.Traversable` in Haskell), which almost all specific monads are. You'd have something like:
newtype Comp f1 f2 a = Comp { unComp :: (f1 (f2 a)) }
instance (Monad f1,Monad f2,Traversable f2) => Monad (Comp f1 f2) where
(Comp a) >>= k = Comp $ a >>= (map join . traverse (unComp . k))
[0]: https://stackoverflow.com/questions/42284879/is-the-composit...
0: in vaguely the same way that Int isn't a legitimate Monoid because you get nonsense by switching back and forth between sum and product (and min, max, xor, and, or, etc)