Problems with the Golang runtime and toolchain
dtrace.org
dtrace.org
Different people come from different backgrounds and, based on that, find different tools useful. Ultimately you need to use the tools that let you get your job done most effectively, however that is defined in your particular case.
The Plan 9 toolchain has worked and continues to work well for Go. Most importantly, it has been something we understand completely and is quite easy to adapt as needed. The standard ABIs and toolchains are for C, and the decisions made there may be completely inappropriate for languages with actual runtimes.
For example, no standard ABIs and toolchains supported segmented stacks; we had to build that, so it was going to be incompatible from day one. If step one had been "learn the GCC or LLVM toolchains well enough to add segmented stacks", I'm not sure we'd have gotten to step two. (Later, for gccgo, Ian did have to add segmented stack support to the GNU gold linker, but Ian also wrote that linker, so he was uniquely prepared.)
As another example, Go 1.4 replaces segmented stacks with stacks that grow or shrink, as needed, by moving. And Go 1.4's garbage collector (GC) is also completely precise wrt stack references into the heap. These require precise information about which words on the stack are pointers and, for the GC, which words are initialized and live. Go 1.5 will use that same information to introduce a concurrent (bounded pause) garbage collector. Getting the information through to the runtime was non-trivial but possible for us to do with the Plan 9 toolchain. If we'd built on GCC, it would have required first threading all that stack map information through the GCC or LLVM backends and into a form that the runtime can use efficiently. Like before, I doubt we'd have gotten to step two. (I don't know how gccgo is going to tackle this. It's a big undertaking. I expect gccgo will have segmented stacks and imprecise stack scans for a while yet.)
The toolchain is quirky and flawed in many ways. Some people don't like typing shift so much in assembly files. Some people don't like the center dot hack (glad the author didn't find the Unicode slash!). Those are really cosmetic details, and if you stop there, you miss the point, which is that it's a small toolchain that we can keep in our heads and make arbitrary changes to, quickly and easily. Honestly, if we'd built on GCC or LLVM, we'd be moving so slowly I'd probably have left the project years ago.
The linker in particular is a piece of code in transition. The author is 100% right that it's not great code. But it does work well, and it is being put to production use at many companies as you read this. The code to produce ELF object files and dynamically linked binaries in particular is new, is my fault, and was quite difficult to fit into the original structure. The linker is overdue to be replaced. The new linker is in the Go tree in src/cmd/link, but it's unfinished.
Using a standard linker instead doesn't work well for a managed language. The Go runtime needs to know where all the pointers are in the data and bss segments, and a standard linker won't put that information together for you. The Go linker does that. The Go linker also writes out runtime information for unwinding stacks during GC and translating program counters to file:line for use in stack dumps. Again, this is all much easier to do with a toolchain that we control entirely than having to build into existing ones. (I am not aware of any way to get file:line information through an existing linker except in DWARF tables, which is overkill. And then there's OS X and Windows, where you _really_ have no control over the toolchain.)
The calls to throw everything away and start over are at best hyperbolic. Certainly there is room for improvement, but the custom toolchain is one of the key reasons we've accomplished so much in so little time. I think the author doesn't fully appreciate all the reasons that C tools don't work well for languages that do so much more than C. Most other managed languages are doing JIT compilation and don't go through standard linkers either.
It is a bit embarrassing that our self-moderation here did not cause your comment to rise a little higher toward the top.
"In the process of working on getting cgo to work on illumos, I learned that golang is trash. I should mention that I have no real opinion on the language itself; I’m not a languages person (C will do, thanks), and that’s not the part of the system I have experience with anyway. Rather, the runtime and toolchain are the parts on which I wish to comment."
For anyone not working in the internals of the golang compiler/runtime, as long as the language is good and the tools produce the proper output when given the proper input, who cares whether they offend someone who writes only in C (and keep in mind I say this as someone who wrote code in C/C++ for almost 15 years, and still has a very soft spot in my heart for C... but not C++, fuck C++).
There seems to be a separating line between people who love and hate Go (well, one of a few such lines) which involves the Go team's convention of making stuff that should be scary for most programmers look and feel scary (see: unsafe package, some of the stuff in cgo, etc). IMO, they take the right approach (keep the easy stuff easy, make the hard stuff possible, and warn of landmines [both explicitly and implicitly] early rather than later) but some subset of people seem to get offended by these choices and lose sight of everything else Go offers.
referring to anything Plan 9 as "colossal" is a new one, i admit :)
Plan 9 is not a product of Lucent. It is a product of Bell-Labs. Lucent killed Bell-Labs. Calling it a Lucent product is akin to the worst insult to us you could come up with, unless you said it was GNU-like.
Plan9 C is a product of Thompson, Pike & Ritchie, you can't get more C than that, whatever your standards committee thinks.
> Without the benefit of decades of lessons learned running Unix in production
I'm not sure what the author of the rant thinks the people who invented Unix were doing all those years.
Really?
That looks like an obvious contradiction. Call me a language descriptivist. If everyone else means something else by C, then the thing in question isn't C.
It's Go Backlash-Day on HN. The pedestal they've been putting it on must have got high enough.
as far as the technical arguments against the Plan 9-descendent tools are concerned, yes, they're outdated. but the lack of cruft accumulating over the last 20 years in that toolchain is also the reason why they went with it as bootstrap for Go -- it was simple enough then. this is changing, that's why most of the Go runtime isn't written in C anymore.
the author made the argument that Go's toolchain is hard to port. this is contrary to previous experience with Plan 9's toolchain. i'm hoping 4ad (who did the port to solaris/illumos) can weigh in with his experience, but from what i saw in the commits the process did not take altogether long or appeared arduous. cgo is a separate matter, as we clearly see :)
as far as the author's personal dislike for all things pike that's a different issue altogether. and quite amusing :)
What can I say :). I love the Plan 9 assembly. It's the same on every operating system (even though the calling conventions are different on different systems!). It has some higher level constructs that make the assembly more consistent between different hardware architectures. It's verifiable, to some degree, by go vet.
It's not great, there's nothing great to it; in many respects it's old fashioned and anachronistic (static register, really? In Go?), but it works just fine. Other assemblers work fine too. I never felt the need to complain about assemblers. They are such a tiny, trivial, implementation detail I am amazed to see them mentioned at all.
As for the Plan 9 C dialect, the Go toolchain recently removed the C compiles, so I don't see the point of discussing Plan 9 C at all, although I like Plan 9 C quite a lot too.
The linkers work the way they work because they originally supported only static binaries. I don't like the linkers at all, although I like what they do. Cross-compilation is a marvellous thing. The code sucks though, I hate it. But there are new linkers in the work. While I can rant away for days about the linkers (I hate the fucking code), I can't complain about the features. I love the features and how they work. I can just complain about the (inappropriate) code.
The Solaris port was definitely slowed down by the fact that on other Unix systems, Go encodes the system call table. I had to add ELF support for linking with shared libraries, in order to not encode the system call table on Solaris. But I only had to do that once. Say, if I port Go to Haiku (God forbid), the code will now exist, and it will work. If Go supported linking with shared libraries from the beginning, someone would have still written that code. ELF shared library support takes time to write and it's irrelevant (from my perspective) whether I had to do it or someone else. Someone has to write it at some point, the total engineering time spent is the same.
As for the portability of the toolchain, it's pretty portable, though porting to a new operating system is a too minor of a job for that too matter. Time is dominated by other effects. But you can see it if you port it to a new architecture. I am doing the arm64 port, and the Go compiler was very easy to port, much easier than say, gcc.
Is there any reason that you can't just encode the system call table on Solaris too?
The situation is exactly the same as on Windows. Windows Go binaries are dynamically linked agains Windows standard libraries, and Go binaries use the shared libraries instead of doing system calls themselves.
I'll save you the trouble : "The ANSII Forth standard does not describe Forth, but a language with the same name."
Cities changes with time and people who stayed in that city decades ago, when they visit recently, will get the similar impression. Any one who saw a phone in late 70's and saw it now, will get the similar impression. Even jobs also change where designation/title may remain same but underlying technologies,tools,working hours,team composition, challenges,expectations on delivery ...etc all may change.
In conclusion, this is natural phenomenon w.r.t time, where top remains same and change happens below.
I'm not sure what you think they're doing, but they were not running Unix in production.
I've read an interview with Ken Thompson taken around the nineties, and he mentioned they were using Windows desktop machines. And true to the words he had Windows on the monitor on his desk.
And of course they were doing academic and researchy stuff. Not running Unix in production.
That's the seventies. Or at best early eighties.
We're talking about the last 30 years.
Not to mention that whatever customers Bell Labs had, doesn't translate that Richie, Pike and co were running Unix in production or cared about those systems and those customers themselves...
They had moved on to other researchy stuff...
The last Unix release with people from the the Lab was UnixWare 7.1 (1999), the last from AT&T in 2001.
Anything else is not Unix.
We know Rob wrote Utah 2000 in that year.
That's 15 years ago.
http://www.thetimes.co.uk/tto/multimedia/archive/00221/97222...
the blog seems to be from a Sun old-timer who pretty typically suffers from ... err ... enjoys this superiority style complex of "we have already invented everything, why stupid word chooses to go its stupid way instead of that great way we think it should go". This complex (and, as a major result of it, refusal to accept reality) is one of the major things while Sun went down. (hi! to Sun guys. Have you seen the abomination what FB did to our nice offices in MPK!? :)
Also, later in that thread:
> It's not a proper assembler, only a way to generate the startup object files necessary to get the binary running, and is neither clean, complete, nor documented.
> None of that is excusable, only true.
Plan 9 C compilers, incidentally, are going away too as soon as they port the rest of Go into Go.
It's clear to me there was path-dependency--some folks who mostly worked on Plan 9 started a little skunkworks-y language project, and rather than deciding to learn the LLVM world and slot themselves into that à la Rust or similar, they brought along the toolchain they'd been working with since forever.
To me, that's more like any other case of sticking with legacy code than NIH--it's "I'm using these tools I've worked a lot with because that's a lot cheaper for me than switching" not "I'm going to build tools from scratch right now because [I think I can do better|it's fun|rabid badgers made me do it]."
I wouldn't mind using LLVM or some such to handle the lower levels at all; I hear it's great elsewhere and I specifically hear Go's ARM codegen is pretty wonky. But it's also expensive to do and a lot less important to most of us than the things the team's working on, like lowering GC pause times.
Finally, in case anyone read it as serious, "Instructions, registers, and assembler directives are always in UPPER CASE to remind you that assembly programming is a fraught endeavor" is a joke. This is the same language that has 'notwithstanding' as an (ignored) keyword; not everything deadpan is serious.
> Unsurprisingly, Plan9 is a colossal failure, and one look at the golang toolchain will satisfy even the casual observer as to why; it was clearly written by people who wasted the last 30 years. There are plenty of other problems with Mr. Pike and his language runtime, but I’ve covered those elsewhere.
Go is not C, and it's not going to be like C. It's not really reasonable to expect it to have the same toolchain conventions as C. I'm not saying the choices to Go authors have made are necessarily good, but I don't dislike them only because they're different from C.
And yet, it's not much above C -- and in some cases, it's less.
Most of us have seen the back and forth over generics in Go, including Rob Pike's own blog? post about why Go doesn't have them. With that said, though, I feel like I've also seen a lot of internals-related discussions with good discourse from all sides.
Whatever the existing design is or isn't, it doesn't seem fair to go off on such a rampage unless there's some sort of documented example that they are actively trying to avoid simplifying things and really are seeking to fulfill all of the things the author wrote. So... is there any evidence of that?
ECB is the simplest AES mode, you just encrypt each block of your message using the same key. Here is an image of a penguin encrypted with AES-ECB:http://upload.wikimedia.org/wikipedia/commons/f/f0/Tux_ecb.j...
Do you see the problem?
The Matasano Crypto Challenges I did way back when were in Go and the ones involving ECB were just done as for loops over the standard library AES Decrypt/Encrypt methods, something like this:
for i := 0; i < blockCount; i++ {
start := i * blockLength
end := start + blockLength
aes.Decrypt(decryptBuffer, input[start:end])
output = append(output, decryptBuffer)
}In any case, it's not surprising that much of the `C` toolchain was ditched. There are few things more frustrating than trying to compile and link complicated C programs across platforms (the only thing that comes to mind are C++ programs).
The things wesolows pointed out about the Go asm dialect do seem weird, though. And what's with the Unicode dot character?
Software is complicated business, and making new things that work similar to old things is a reasonable (and often successful) way to deal with the complexity. However, if it isn't close enough to the original, it might cause more problems, as it did for the author when he was using this not quite real assembly language.
I really hope that some knowledgeable HNers will weigh in on this, it is fascinating to read about.
The author identifies as someone who enjoys C programming (so I do not doubt their sincerity here), but for those of us using languages with memory safety, I think this may not apply. Particularly when I consider security, I cannot help but worry about any C/assembly I write.
I'm not interested in compilers, I hate writing code in low level languages, I cannot for the life of me understand assembly and Go works extremely well for what I want to do. I've used PHP/C#/Javascript my whole life and I'm more concerned about writing software than writing code. I don't want to nitpick on the tiny details of a running software with memory registers, cpu cycles, cache miss, etc unless there is an obvious bug or performance issue I need to fix. Those things should be abstracted away from me as a developer and go is a great compiled language that looks and feels like an interpreted language from a developer point of view.
The post itself is substantive [1], so we've unkilled it and attempted to give it a neutral title.
1. Edit: though dismayingly not free of the same drama-mongering. Hopefully the HN thread will stay substantive in response.
Actually, I think the term "linkbait" is thrown around far too freely here and should probably be banned.
It is a substantive article though, even if one happens to disagree with the author's conclusions. I'm glad it was resurrected from flagkilled state.
So what happened with those users that flagged a link without reading the article?
I also feel like there are some factual errors. He chides the Go developers for "arrogantly ignoring" existing tools for linking code. But the gccgo integration was done by Ian Lance Taylor, who is on the Go team at Google. I believe Rob Pike is on record as saying that he wanted multiple implementations of Go to increase the robustness of the language.
I would appreciate a good critical look at the Golang runtime and toolchain. How does its performance stack up, how fast does it compile, etc. But this ain't it. This is a guy complaining that he doesn't like the naming of registers in machine-generated binary files. I will literally never need to care about any of this. I don't even need to know this to make use of assembly language in my Go programs, since I can use the C library integration for that.
because they fucking forgot incremental compilation http://en.wikipedia.org/wiki/Incremental_compiler
This is what they should have spent their 20% on: https://gcc.gnu.org/wiki/IncrementalCompiler
Also, Go does have incremental compilation at the package level.
In practice compilation times are not a pain point for golang.
Is there any rule about also having to be tactful if they fulfill the first criterion?