Hacker News Reader

Comment by mattlondon | Hacker News Reader

I can feel the pain on the Sort issue. I've personally found sorting annoying in Go - I had a bunch of structs representing data entities from a database that all had the same field and I wanted to be able to sort them by this field.

Seemed like a LOT of work (basically implementing the same sort that was 99% identical for every struct) or use weird reflection-workarounds to get this to happen. In Java I would not even given this a second thought and be back to coding up the important part of the code ages ago.

I am a new go-lang user so would love to know what the best approach to resolve this is without a) repeating the same thing for every struct, or b) relying on "unsafe" reflect techniques (since AppEngine rejects code that does that) - surely sorting structs is a super-common, basic thing for a systems language? I've seen someone just nonchalantly say "Use interfaces" but I'm not sure still.

I like the language generally but this is a real "WTF?" moment for me.

View on HN

I had the same feeling first. But practically in my code, I found that ok, you need to copy/paste a bit first but then if it works it stays there, you are not "sorting" new kind of "types" every day. The time spent on coding is way more "around" the algorithms than "within" them.

I suppose that we will see more and more code generators which will practically remove the need of generics. We already use them without complaining for serialization in JSON/Protocolbuffer/etc...

If you're copying and pasting then you're doing it wrong.

Not in go. You're allowed to in go. Because some googlers reckon that's cool.

>you are not "sorting" new kind of "types" every day

You'd be surprised.

I am writing a chemoinformatics database, so for my practical use, these are a lot of lines of codes with pretty involved algorithms and I am practically not annoyed by the lack of generics.

For the ones down-voting me, have you coded something in Go, big enough to be a real in production project, where at the end the lack of generics is a real issue (performance because using interfaces or maintenance because copy/paste to have the performance)?

> For the ones down-voting me, have you coded something in Go, big enough to be a real in production project, where at the end the lack of generics is a real issue (performance because using interfaces or maintenance because copy/paste to have the performance)?

Didn't downvote you, but yes I have.

I'm part of a project team that uses quite a lot of Go in production (for analytics work), and lack of generics was particularly painful.

>(performance because using interfaces or maintenance because copy/paste to have the performance)?

I don't like interfaces (namely, interface{}) for their lack of safety for generic work -- performance comes second to that.

And I don't like copy/paste like ever.

>For the ones down-voting me, have you coded something in Go, big enough to be a real in production project, where at the end the lack of generics is a real issue (performance because using interfaces or maintenance because copy/paste to have the performance)?

Isn't that a sure fire way to selection bias? The ones that ended up coding something significant in Go will usually be those that put up with the Generics issue (or don't even know what they are missing).

It's like asking C programmers if they mind missing GC, closures, etc.

It's not like the utility of Generics is some open question in PL either. It's a settled matter. Even Go uses them, but doesn't offer them to the programmer, or suggests generic but not type-safe solutions like interface{}.

I wrote some big things in Go and didn't find the lack of generics particularly problematic. Different languages are good for different things-- Go is good for building things that are relatively concrete. For something like a symbolic math package or a scripting language you might want a different language that makes different tradeoffs.

If code generation is used to make up for something missing in a language (be it generics, metaprogramming etc.) then that's a pretty clear sign something is wrong.

It's definitely acceptable to use a workaround for a missing feature once or twice in a language, because no language is perfect, and no language benefits from being burdened with all features imaginable.

But if a workaround becomes part of the day-to-day workflow, then you are likely using the wrong language.

Examples could be: using (textual) code generation for generics, or using type annotations throughout a dynamic language project.

Well put. Though I would add that though generics are convenient, they are not as needed nearly often enough to be a crucial missing feature in a language. Yes, it's uncanny to copy here and there, and certainly, the Go team should try and resolve this. But from where I'm standing this (i.e. generics) is one of the very few fair criticisms of Go which can be leveled from the Java, C++ or C# communities.

Code generation is not about a deficiency in the language, C++ has templating but I will often use code generation since you only need to run that once and templating bloats the compile time for ever.

>I will often use code generation since you only need to run that once and templating bloats the compile time for ever.

Don't you need to compile the generated code?

For reproducibility you have to regenerate, write, read, and parse it, which I can't imagine being faster than instantiating a template in memory.

Yes of course, but compiling the code is faster than generating the code and then compiling it. Templates are much slower than just compiling code straight.

With the exception of pathological metaprogramming examples -- and even those have largely been fixed -- there's no way you could even measure this, let alone justify such a strong, broad opinion. You're using incomplete information to justify sloppy engineering and promoting it to others.

It's compile times, those are very easily tested and measured.

Templatizing/de-templatizing enough code to see a difference would be a significant effort on any non-trivial code base. But I'll spare you the trouble: instantiating a template is less work than parsing a duplicated file. Some of the early C++ compilers had problems but it hasn't been an issue in 20+ years. If you look at both the G++ and Clang test suites you'll see they verify performance, memory usage and correctness with complicated templates by doing basically this exercise for you.

ok, thank

Everyone agrees something is wrong, even the Go team. The debate is whether it's meaningfully wrong, or just "someone's wrong on the Internet" wrong --- because the cost of righting this wrong will be high.

I don't think we will.

The Go community doesn't hate protocolbuffers, but it does tend to think that generics are evil and shouldn't exist.

I'm certain that generic generators will be shunned by the community at large, due to solving a problem they don't believe needs solving. Without the network effect to build up a user and developer-based, they'll languish in obscurity.

I sure would like to be wrong, though!

I haven't seen any indication that the community hates the idea of having generics. The contrary seems to be the case.

Seems like they talk a lot about how it's a problem but don't put the effort into solving it.

In other words, they aren't interested in getting generics. Not really.

Write a code generator. That's the best solution at this moment.

Or, y'know, just copy and paste the trivial code. It should take all of 30 seconds.

Not saying that it's pretty, but it's quick and easy.

And that's exactly why I am not convinced that Go is as maintainable as often claimed. 30 seconds for you, but how many hours for the poor souls who will come after you and ask: should I change this copy too or is it a separate case?

It reminds me of this: http://typicalprogrammer.com/abject-oriented/

This thread is about sort interface implementations. Their general form will never change, and the only specifics unique to any given copy are those specific to their type. It is obvious to any Go programmer what can and cannot be changed in this situation.

The thread was more about mattlondon's problem with sorting according to the same key in different structs. I used of modified version of this example[0] to illustrate what I think is his problem[1]: you can sort circles and planet by radius, and probably many other things in the original case, but you are required to copy-paste the definitions.

I can understand that instantiating templates by hand is not so bad. But once it is written, maintaining the code is not so obvious for someone not acquainted with it.

Did I introduce a bug or not in the code? Not easy to tell without context.

[0] http://stackoverflow.com/a/28999886/124319

[1] https://play.golang.org/p/Dq0AJjkhhr

Why not write an interface for getting the radius and a radius sorter for sorting on that interface? I made a modified version of your code to illustrate.

https://play.golang.org/p/Ya7tUhDnO2

*edited a typo

Thanks a lot for your example. I would certainly take this approach and I am happy that you took the time to write it.

I feel a little sorry for you because I put a trap for the parent poster in the original code.

I wrote "a[j] < a[i]".

You "fixed" it while refactoring, which would be the good thing to do if I made a typo. If however I really wanted to sort circles differently from planets, then you made an error while refactoring what looked like copy-pasta but wasn't.

My original point was that when you "have" to copy-paste, you cannot clearly see what is or isn't part of the copy and what is new.

You found a way to rewrite the code without much copy- pasting, and I am happy to see that. Still, in other places where code duplication arise, there could be similar problems from a maintenance point of view.

It's 3 lines of code the most newbie of newbies could understand... if this is the biggest problem I have in my day to day programming, I'll be happy.

Is it mildly annoying to type out those three lines? Sure. You know what's a lot more annoying? Basically everything else in programming.

The problem with code duplication is not about being lazy, or having to type. I love typing.

When you dive into a codebase full of vaguely similar yet different blocks, it starts being more than mildly annoying to understand the intent of the code and make the correct change.

nobutseriously...

    type userList []*User
    
    func (u userList) Len() int           { return len(u) }
    func (u userList) Swap(i, j int)      { u[i], u[j] = u[j], u[i] }
    func (u userList) Less(i, j int) bool { return u[i].Name() < u[j].Name() }

What change would you ever need to make to this code that would be at all difficult? The first two methods on userList are never ever ever going to change. The only thing you could possibly want to change is how to sort inside the Less method... and that code would be the exact same code you'd have to change no matter what language you're in. You still have to define the sort one way or another for non-trivial types.

So, yes, it's some extra typing... but saying that it makes maintenance harder is just plain wrong.

Not all duplication is bad in all cases. It is more a factor of quality, not a direct, causal relationship. See for example this review of publications about code duplication: http://eprints.eemcs.utwente.nl/15314/01/ewic_ea09_s4paper2.....

Yes, I agree completely. Take leftpad, for example....

And more seriously, there are times when factoring out "common" code makes the code significantly more complicated, and often turns your common code into a morass of special cases as your application progresses, and these cases that looked "the same" end up being "not quite the same".

commit c270f19d456e862aeb559098fa32d36fbc5329a0 Date: Wed Apr 13 04:57:46 2016 -0700

    fixing level selection again

commit dae018e4f76db0e9da810b4560d6c8e1c7029048 Date: Mon Apr 11 07:06:47 2016 -0700

    fixing level selection for hint case

commit a754bcd0d09ab1e357ae28c8f06baf65d879878f Date: Sat Apr 9 02:33:14 2016 -0700

    forgot to fix the level selection in the move case

Maintenance.

What if I'd done that 20 times around my code-base and then later I discover there's an off-by-one error in the original code that's maybe been inherited into all 20 copies?

If I could actually reuse code properly I'd only have to fix it once.

Without effective code reuse, I have to hunt down the copies, each of which may have slight modifications to make them better fit their use case (and might be hard to grep for as a result), figure out whether or not the bug exists in that copy (and whether it can actually be triggered), and fix it there.

The description of inheritance in http://typicalprogrammer.com/abject-oriented/ seems relevant here. (Edit: Derp, junke got there way before me!)

Then one day you find a bug in your code and now will have to find all copies of it and to make things worse someone else modified some of the copies so now not all of them are identical.

You might be right, this is the best solution in Go at this moment, but that's a local optimum. You don't have to solve this kind of problems in many other languages.

The generator given as an example in the official blog [0] is called "stringer". It is made of 639 lines [1] mixing ast parsing/traversing and printf statements. If this is what I am supposed to write then of course copy-pasting becomes a pragmatic alternative.

[0] https://blog.golang.org/generate

[1] https://raw.githubusercontent.com/golang/tools/master/cmd/st...

Or go the JS route and write a transpiler. Language is lacking? Make a new one.

Or just use a language that supports generics instead?

The first company to make a fork of Go that runs on the JVM and has compiler-specific extensions will make a ton of money, IMO.

This will only target a very small group of developers, who are comfortable with JVM, but not with Java or Scala. I don't see any money there.

JVM is a major drawback for any language. Many people don't even look at JVM languages.

No, it's the opposite. It would appeal to people who built large Go codebases and eventually realised that they were tied to a toolchain that was years behind the state of the art.

A Go for the JVM would immediately give Go developers much better optimising compilers, high quality cross platform IDE-integrated debugging and profiling, much stronger garbage collectors, ability to access the large quantity of Java libraries that are out there, ability to mix in other non-Java languages too (javascript, ruby, R ...), code hotswap, better monitoring tools, a vastly larger standard library, better code synthesis tools and so on.

The JVM is hardly a liability. It adds features and carries a cost, but Go is substantially similar to Java in many ways and getting closer with time. I do not see any credible plan from the Go developers to beat the JVM developers in terms of GC or compiler quality, for instance.

JVM cannot appeal to Go developers, because compiler is a significant factor why people choose Go in the first place.

There are quite a few native compilers for Java, the difference is that the best ones aren't free.

Go GC since 1.6 openly claims <=10ms STW pauses. Does any open source Java GC offers that? Also Go uses an order of magnitude less memory for running process compare to something similar in Java so I do not see how optimizing compilers in Java are doing any better job.

In my experience Java brings a mindset that there must be some complex way to solving a problem so lets find out that.

Go 1.6 GC is exactly what I mean. It's a design that gives tiny pauses by not being generational, or incremental, or compacting. Those other features weren't developed by GC researchers because it was more fun than Minesweeper. They were developed to solve actual problems real apps had.

By optimising for a single metric whilst ignoring all other criteria, Go's GC is setting its users up for problems. Just searching Google for [go 1.6 gc] shows the second result is about a company that can't upgrade past Go 1.4 because newer versions have way worse GC throughput: https://github.com/golang/go/issues/14161

Their recommended solution is, "give the Go app a lot more memory". Well, now they're back in the realm of GC tuning and trading off memory to increase throughput. Which is exactly what the JVM does (you can make the JVM use much less memory for any given app if you're willing to trade it off against CPU time, but if you have free RAM then by default the JVM will use it to go faster).

BTW the point of an optimising compiler is to make code run faster, not reduce memory usage. Go seems to impose something like a 3x overhead vs C, at least, that was the perf hit from converting the Go compiler itself from C to Go (which I read was done using some sort of transpiler?). The usual observed overheads of Java vs C are 0 to 0.5x overhead. The difference is presumably what the compilers can do. Go's compiler wasn't even using SSA form until recently, so it's likely missing a lot of advanced optimisations.

tl;dr - I have seen no evidence that the Go developers have any unique insight or solutions to the question of building managed language runtimes. They don't seem to be fundamentally smarter or better than the JVM or .NET teams. That's why I think eventually Go users will want to migrate, because those other teams have been doing it a lot longer than the Go guys have.

sievebrain - Did you read the full issue? This is an edge case - a program running on a 40 core machine that the developers were trying to keep to a 5MB heap. And yes, the answer was "use more RAM", but by "more" they mean "40MB". Not like gigabytes or anything.

There's always going to be edge cases in any GC/compiler/etc ... you just can't account for every case. I suppose with java's infinite knobs, you might be able to... but then you have to tune the GC. In Go, there's just one knob (a slider, really, more CPU vs. more RAM), and 98% of the time you'll never need to touch it. I had honestly forgotten it exists, and I work on a large Go project daily at work.

You don't have to tune the Java GC - I never have, and I use and write Java apps all the time.

People can and for big servers often do, to squeeze out more performance or better latency, but it is definitely not required.

In the presentation I linked to, GC tuning (after switching to G1) reduced tail latencies a bit, but otherwise did not radically change anything.

> Go GC since 1.6 openly claims <=10ms STW pauses. Does any open source Java GC offers that?

Yes. HotSpot has had configurable max pause times for years and years [1]. If you want less than 10ms, set MaxGCPauseMillis to 10ms. It also has a state-of-the-art generational GC, which is very important for throughput, as bump allocation in the nursery is essentially impossible to beat with a traditional malloc implementation.

[1]: https://docs.oracle.com/cd/E40972_01/doc.70/e40973/cnf_jvmgc...

From the link I see:

> The following example JVM settings are recommended for most production engine tier servers: -server -Xms24G -Xmx24G -XX:PermSize=512m -XX:+UseG1GC -XX:MaxGCPauseMillis=200 -XX:ParallelGCThreads=20 -XX:ConcGCThreads=5 -XX:InitiatingHeapOccupancyPercent=70

So Oracle mentions 200ms for most prod use. I am not sure how you are able to deduct ~10ms pause from that link.

And just because one can configure ~10ms does not mean at JVM will start respecting it. There is nothing in any official document by Oracle about max GC pause time. The results Google threw are mostly around ~150ms as min pause.

> It also has a state-of-the-art generational GC.

And it needs something like

http://www.amazon.com/Java-Performance-Charlie-Hunt/dp/01371...

to do what JVM can do in theory. In practice as a java users I am used to seeing ~20-30 sec pauses for Full GC.

The only effort in open for sub 10ms GC for large heaps is Project Shenandoah:

http://openjdk.java.net/jeps/189

and it is long way from availability.

You can ask for 10ms latency and you will get it. This is basic functionality of any incremental/concurrent GC. The throughput will suffer if you do that. But HotSpot's GC is far beyond that of Go in regards to throughput, for the simple fact that it's generational.

Nongenerational GC pretty much always without exception loses to generational in heavily GC'd languages like Java and Go. There is no silver bullet for GC; it requires lots of hard engineering work, and HotSpot is way ahead.

Go's unique proposition, IMHO, is compile times. If you have a codebase that's 10 million lines, with 10 or 100 developers working on it for 10 or 20 years, compile times really matter.

Can you build a language that runs on the JVM that compiles as fast as Go? Perhaps. Java sure ain't it, though.

Have you done a comparison? JavaC is extremely fast and compiles incrementally. Turnaround time from editing a file to seeing the change on screen is measured in a couple of seconds on my laptop. I don't think Go has any speed benefit in this regard.

I doubt it. I did Java for over a decade, but jumped at the opportunity to use Go with its statically compiled binaries. The JVM is great, but being tied to it is kind of a hassle. Being able to hand a small binary to someone and say "here, run this" with no worry about dependencies- it's a beautiful thing.

The downsides are enormous.

You can use the 'javapackager' tool in Java 8 to make a tarball that contains a JRE. It's not small, but that's a result of the age of the platform; it just has a lot of features (in Java 9 there is a tool that can make a bundled but stripped and optimised package).

Go binaries are getting larger and the compiler is getting slower over time, as they add features. They don't have any magical solution for bloat: eventually, they'll add the ability to break the runtime out into a dynamic library as it gets fatter and fatter.

Or of course they can just refuse to add features and doom their users to a mediocre, frustrating experience forever.

Go 1.7 (tip) binaries are smaller than Go 1.4 binaries.

Actually, the compiler is getting faster in 1.7, and binaries are getting smaller: http://dave.cheney.net/2016/04/02/go-1-7-toolchain-improveme...

I don't know. You'd have to switch ecosystems completely. When you need Go ecosystem and generics, only thing to do is go transpiler route.

There is also a route of a fork. Adding generics doesn't look very hard to me.

Wouldn't that end up being essentially equivalent to adding generics to go?

Couldn't you create a common interface that you could use for the sort?

No, the sort package requires you to define methods on the slice of what you want to sort. For instance, if you defined a struct S, you need to implement Less, Swap and Len on an alias type of []S (since you cannot implement methods on slice types).

No. There is no interface that "sortable" things implement, other than the empty interface, and no neat way to define one.

Numbers and strings support <, but there's no way to specify operators in interface, and no way to specify operators for other types anyway.

You could do

    type Comparable interface {
      Compare (x Comparable) int // -> -1, 0, 1
    }

    func Sort (x []Comparable): []Comparable {
      /// ...
    }

And whilst this Sort could work, how do you call it? []int isn't []Comparable, and can't be converted to one: you have to make a new array. Then, when you want an array of ints on the other end, you have to convert it back, which now involves run-time type assertions.

Even an array of something that implements Comparable isn't compatible - it can't be, because Go doesn't know Sort won't take Bar[] and put a Foo in it, if Foo and Bar both implement Comparable.

And, whilst you can define Compare for int, the other argument will be a Comparable, not a int, so you'll have to have a run-time type assertion for each comparison.

Conversely you might try

    func Sort (x []interface{}, f func(interface{}, interface{})int) []interface{} {
      /// ...
    }

You still can't sort []int, and your comparison function can't know it's receiving int, so it will have to type-assert both arguments at each comparison.

The whole problem is that there is a common interface, but it's a PITA to reimplement it every time (which means copy-pasting a little less than 10 lines) when every other language gives you sorting with a single, simple, 1-line function (or even a single argument of an already existing function).

I made this: https://github.com/imjasonh/tros

It's slower than doing it yourself but for normal-sized slices you shouldn't notice.

Also https://github.com/bradfitz/slice which is similar but faster and relies on more unsafe black magic.

Completely oblivious about Go, but can you not provide a key function to the sorter that reaches into the struct and pulls out the field you want to sort on? (And presumably explodes in some spectactular fashion if it doesn't exist.)

Or is that the "unsafe" reflection you're talking about?

For struct's at least, the answer would be to use an interface method which exposes the key field you want to use. This would be type-safe - you wouldn't be able to pass a non-conforming struct to the sorter.

EDIT: I'd argue what we need in Go is something like a type-class for interfaces, so we can match on fieldsets the struct contains, and not just methods.

I think matching on fieldsets would be the ticket here.

I guess right now I could add a pointless no-op "marker method" to make each struct match a "CanBeSorted" interface, then have my sort function work in terms of things implementing CanBeSorted, but that does not guarantee the fields I want to use are there.

Sigh.

I am hoping that the "no backwards incompatible changes" thing Go has wont prevent fixing things like this sorting nonsense. Right now, whichever way I approach this sort of thing just feels icky.

This is stupidly easy with http://dl.acm.org/citation.cfm?id=2738008 - I opened an issue for considering something like this for go https://github.com/golang/go/issues/15295 hopefully I didn't err in posting it :)

are you really posting a paid article on HN?

The article on Sci-Hub https://sci-hub.io/http://dl.acm.org/citation.cfm?doid=28138...

I agree with the Sort issue, I faced the same problem. However, I cannot see how generics would fit well into code that is supposed to be easier to understand and maintain.

Reply on news.ycombinator.com