Function types in Go | Hacker News Reader

Function types in Go

jordanorelli.tumblr.com

121 pointszemo29 comments

Go's support for first-class functions definitely sparked my interest in the language. But one thing that Javascript, Python and Ruby programmers would expect of higher order function support is the ability to use the general collection methods such as map, reduce, filter (or inject, collect, fold - whatever they are called in your language). So far as I can tell, generalized functions of these types are not really possible. You could write some that were typed interface{} and then did explicit casts in the closures you passed, but that would not be nearly as elegant as what other languages provide. I believe in a static type system without generics this is not really possible, and this is really (to me) the most painful cost from the lack of generic support.

Go is an opinionated language. Go people would just tell you to write the loop; that's what it's there for.

To be clear, generics seems to be one thing that Go isn't actually opinionated on. As I understand it, they are open to including generics in future versions of the language, but only if the implementation is elegant.

Well, as far as I understand, that implementation would need to overcome the Generic Dilemma as described by Russ Cox: http://research.swtch.com/generic

So far, the Go team decided to go for approach #1.

Option #3 is unfairly presented. There is no reason a language couldn't implicitly box every type, but automatically create efficient specializations for (some of) the types actually used. You can have your cake and eat it too.

Of course, it's not easy to write such static optimizations. But it is, in principle, possible, and the sort of thing you only need write once.

> But it is, in principle, possible, and the sort of thing you only need write once.

Well Go is open source so "someone" would just have to write, test and submit a CL / patch... ;-)

> So far, the Go team decided to go for approach #1.

Except where they personally really wanted parametric polymorphic collections so they baked them as special case into the runtime, of course (map, chan, …)

In a sense, they went for #3 (auto-boxing) without the syntactic sugar. Casting int to interface {} has a similar overhead to boxing in Java.

The loop was a trivial example. Less trivial examples are not possible either. For instance, I'd like to write a function that takes in a channel and returns a (directional) channel that has been turned asynchronous, by running a goroutine that manages a queue and doing some other things above and beyond what a merely buffered channel does (emulating Erlang's way of managing async messages with backoff timeouts), but I can't do that correctly and safely. I haven't completely written this function in fact but it's certainly at least 20 lines.

I'm making a list of the things that I tried to do in the language, then couldn't; I'm up to four right now, each of which are not trivial examples, but are instances of code that I must repeat because I can not refactor it. This is not a plus. (My list is at home and I'm not, and I'm sort of saving it for a blog post anyhow, but this is representative; not killer, but the issues compound as the program gets large.)

Mind you, I know the language is young, etc etc, and I support the idea of doing it right rather than doing it sooner, nor am I attached to "generics" as the only possible solution. However, there is a real problem here for refactorability, and I would not want the Go community to build too much of a metaphorical callous around this issue and get to the point where they just reflexively respond with annoyance. There is a problem here.

>For instance, I'd like to write a function that takes in a channel and returns a channel that has been turned asynchronous, by running a goroutine that manages a queue and doing some other things above and beyond what a merely buffered channel does, but I can't do that correctly and safely.

see here: http://gowithconfidence.tumblr.com/post/31426832143/stacked-...

the trick is to make a channel of buffer size 1, then when adding to it, simultaneously try to read and write from that channel using a select statement. Only one will ever be ready.

You could also do something similar really horribly with interface{} but that's almost always a code smell: https://gist.github.com/jordanorelli/3668150

I probably wouldn't do that in production code.

Either way, generics in Go remains an open question.

That's simpler that what Erlang does (you need some code for the time backoff, and while I think Go can do it it is inherently the sort of code that ends up a complicated mess of cases that, if I could lock it behind a function, would be very well abstracted away for the user), but it is the basic idea. The problem is, you have to copy & paste those lines of code for every type you want to do that with; you can't vary that code on "Thing".

In this case the fact I'd have to use interface{} isn't a code smell, it's a language smell. Looking in two different "directions" from Go, that's trivial to implement as a single Thing-agnostic function in the weaker-type system Python (gevent queues have similar enough semantics for this to be a fair comparison), and it's trivial in stronger-type system Haskell (also similar enough semantics with TChan to be fair). Also I note you did not return the chan for the user to do with as they like, abstracting out the mechanics from the task, but have a hard-coded "join" function in there. You can easily take in a function to "do something", but you won't be able to do anything with the resulting value, not even return it, unless via inteface{}. (Which makes me cringe every time I type it.)

Again, to be clear, I'm not trying to dump on Go here; there's a lot of things to like about it, especially its infrastructure. But this is by far my biggest pain point with it personally. YMMV.

You can abstract the channel operations behind interfaces and regain type safety. Inspired by the sort package, I wrote http://thegoods.biz/broadcast for doing broadcast type patterns safely. To use it you implement two trivial methods on a channel type and away you go, single producer multiple consumer channels.

I think there are clever solutions to these types of problems that have yet to be explored because most of the focus is on what Go lacks from other languages.

It concerns me that Go has apparently built up such a community of people who insist that there aren't any problems here and you just need to learn the Go way to do it, which nobody quite knows yet, and doesn't quite meet the bill when demonstrated, and seems to require an awful lot of copy & paste boilerplate even so. I don't think that solution addresses my concern, which is that I want to be able to take a channel, and return a modified channel. You can't do that by "wrapping behind an interface", because once you return anything other than another channel you've lost, because you can't "select" on anything other than a channel. You can't say "Oh, I'll wrap the select behind something", because there's no way to compose two selects together; if I want to select on this channel and that channel I must have them both in hand, as channels. So if I want to factor out a particular strategy for managing input on channels, it's just impossible as it stands now; the type system is sufficiently limiting that we can see that that is impossible to write such functionality.

(I haven't gotten around to coding this yet, since I've been taking a break, but the actual solution I'm planning is like this:

    in the library;
        func DoSomethingGeneric (interface{}) (interface{}) {
            blah blah blah lots of reflect probably
            return newThingy
        }

    in the module using the code:
        func DoSomethingSpecific (Thing) (Thing) {
            return DoSomethingGeneric(Thing).(Thing)
        }
    }

I'd make it correct by construction, but that's annoying. Annoying is bad at scale, it means people do the lazy thing instead of the right thing.)

Further, I don't see any way to abstract out the pattern that you do have in hand; you wrote it, and if you want to stick a new type into it, you'll have to write quite a bit of code again.

It's too soon for the community to be this dogmatic about how Go has an answer to everything, when it plainly doesn't. And let me underline this: I'm not complaining that it doesn't have an answer to everything; it's young. I'm complaining that it's too early to be so dogmatic. This is the biggest turn off to me about the developing community.

I keep thinking maybe I'm just spoiled by Haskell or something, then I remember; I can do this in every other language I know in one way or another. In the dynamic languages, it's not any more type safe than anything else, but it works. In the stronger languages, I can get this done. Only Go has has a type system that is strongly typed, but forces me to bypass the system to get it done. (Perhaps C, though that's contingent on so many things it's hard to construct a fair comparison. One must first build a fair amount of structure to even approximate the problem I'm having with Go here.) And it's not like I'm a stranger to programming under very strong restrictions. I like Haskell.

(Also, I am aware that "the community" who is saying this is not everybody; those who are not should probably work on telling those people to tone it down a bit. And broadly speaking, I actually do like Go; this conversation is focusing on my two biggest pain points and is not representative of my full opinion.)

Others just move to D or Rust.

Look, with Go's interface{} system and its out-of-box "pseudo-generic" typed arrays, slices, maps and channels -- there really isn't an awful lot missing in day-to-day pragmatic programming. Pragmatic as in "old-school imperative line-by-line programming with a few modern ideas mixed in as feasible" -- not necessarily software engineering concoctions of patterns of factories etc.

Again -- the interface{}s and boxing/unboxing, typed maps/slices/arrays/channels -- they all represent stuff where (A) the compiler can infer type knowledge safely and (B) very fast compile-and-link times are not compromised.

Like any language, Go isn't designed to cater to every use-case or programming style. You wouldn't even consider C for your above issues -- right? -- well the safe bet is to always view Go more in the realm of a modern, saner C rather than comparing it to Haskell, or Python, or Scala, or ......

The topic of generics comes up about 2x a week on the golang mailing list / Google Group, often (well sometimes) discussed in-depth with good arguments and quality of discourse. So... either the core gophers are just too clueless and dogmatic to be adding them to the language, or their reasons are still as valid as last month or year with regard to Go's primary design goals and focus.

  > I'm making a list of the things that I tried to do in 
  > the language, then couldn't; I'm up to four right now

I'm curious to hear the remaining three. This is unrelated to Go, I'm just generally interested in real-world code examples that are challenging to express in any given language.

What you're after here is called parametric polymorphism (commonly known as "generics"). Go does not have it. Debates still rage on the ML about whether to have it or not, but nobody has yet come up with an implementation makes most happy.

Go does have polymorphism though, in the form of subtype polymorphism. In Go, it is achieved using interfaces.

Subtype polymorphism typically has more programmer overhead than parametric polymorphism, so it is not used for simpler things like a generic map, fold, etc. However, it can be used to reduce code duplication in other cases, like sorting (among many other things).

I almost missed your ML pun. To the uninitiated, ML was the first language supporting parametric polymorphism around 1976.

What is interesting about Go's subtype polymorphism is that it is structurally subtyped.. so A is a subtype of B if it fits the description of B, it does not need to explicitly declare that it "supports" or "implements" B.

If you need parametric polymorphism and higher order functions in a strong, statically-typed compiled language you may want to take a look at Haskell.

I feel fairly productive in Go, having initially decided "OK well let's just see how far I'll get without generics before giving up". Turns out the instances where I think I need generics are much rarer than I initially expected. Also, sometimes we just need a bit of template programming rather than full-blown generics. Templates let you express whatever you'd use generics for but also much more. So once you realize you won't need generics if you could just get some template programming support, the problem becomes trivial: there isn't any out-of-box support for templating but it's trivially easy to write the bit of string processing required for a pretty general reusable flexible fast poor-man's templating preprocessor, and even easier to hook that process in your build routine.

Some people use "gotgo", I rolled my own minimalist template preprocessor:

https://github.com/metaleap/go-buildrun

Now you'll see I have written some sizable amounts of Go code at https://github.com/metaleap/ and https://github.com/go3d/ yet I have only had to use that "templator" in a very few rare instances.

But then, I didn't come to Go with a functional-programming mind-set, which automatically means I somehow don't actually need to map-reduce-filter-collect-fold-etc. that often... ;-)

I really liked this treatment of first-class functions. It started off slow, with a lot of trivial, contrived examples that should be obvious to anyone who has ever seen first-class functions in any language. But then there were some very pragmatic patterns, with standard library examples, that used recursive function types and methods on function types to build some very neat APIs. Building a lexer out of a recursive function type actually sounds like a really cool thing to try, if I find some free time I want to try it without looking at Go's standard library `text/template` lexer and see how it compares.

> Building a lexer out of a recursive function type actually sounds like a really cool thing to try

Rob Pike did just that for Go's text/template package (http://golang.org/pkg/text/template/).

See the source lexer source code at http://golang.org/src/pkg/text/template/parse/lex.go and note that stateFn is defined recursively.

There's a video by Pike describing how he built the parser at http://www.youtube.com/watch?v=HxaD_trXwRE.

Probably my favourite thing about Go. The ability to create a DSL with a systems level language is kind of amazing.

"Function types in Go can also have methods."

Wow, blew my mind... this has never occurred to me but, of course, why not?

Not sure yet how I am going to abuse that feature, but already looking forward to it.

The HandlerFunc type in net/http (http://golang.org/pkg/net/http/#HandlerFunc) does this.

channel types can have methods, too ;)

http://gowithconfidence.tumblr.com/post/31426832143/stacked-...

I never really noticed how similar Go's function expressions are to JavaScript's before, but I'm now pretty certain they were one of the (subconscious) factors that got me into Go.

Anonymous functions as an expression do have their purpose, unlike the author has stated, especially when you want to isolate different blocks of code from each other by putting them each in a different scope.

Always fun to see examples of first-class functions; thank you for the read!

Reply on news.ycombinator.com