Rob Pike interview
evrone.com
evrone.com
What exactly is misleading about generics?
The psychology of thought leaders is fascinating. The term generics is colloquially understood to mean parametric polymorphism and here's Pike redefining the term to draw some distinction that doesn't really exist just to remain self-consistent.
Come to think of it, this is a really common pattern. If you've made remarks that you must now backtrack then refine terms until your old writing doesn't seem to contradict your current stance because you can just say people were using the terms wrong and you were right.
Some results for searching "generics": https://docs.microsoft.com/en-us/dotnet/csharp/programming-g..., https://www.typescriptlang.org/docs/handbook/generics.html, https://docs.swift.org/swift-book/LanguageGuide/Generics.htm..., https://doc.rust-lang.org/rust-by-example/generics.html. If you search for "Wadler generics" then a book about generics in Java is the first result: https://www.amazon.com/Java-Generics-Collections-Development....
"Parametric polymorphism" was the "whole" feature, while "generics" meant specifically generic containers, usually wholly agnostic about their type parameters, like std::map or std::list. Since the rise of Java Generics (which are really somewhere in between, more than containers but less than full parametric polymorphism) I don't hear this distinction anymore.
> What about ALL the other components NOT written in Go?
Out of curiosity, which components are you thinking about?
I could start listing off OS kernels, and then I'll move on to device drivers and hypervisors.
"The Cloud" is enabled by the fact we can efficiently separate applications and users running on the same hardware. The software which ultimately enables this is not written in Go... or Rust... or C++... or any other "hyped" language, but it is still an essential part of any cloud service.
But, the joy of 2020-era design is that nothing is forcing you to use Go. Everything is coupled with network APIs these days, so you can generate your protocol buffers for whatever language you want and write your chunk in that. You don't have to look at the success of Go in this space, think "but I don't like it", and leave the field. You can do whatever you want. But, I do think it's accurate to say that Go has a lot of mindshare in this sphere of the Universe. It is what it is.
Also, by restricting to open source you dropped 2 biggest and successful players.
I think to get an idea of what mindshare looks like, you have to look at people that started from nothing today. These are the people that are making architecture decisions with an eye to the future, growth, and immediate productivity. When you're inside a huge org, you have to think about what is easiest to integrate with, and what skills you can get from other teams. At a company that's 80% Java, that's Java. It would be insane to switch, and likely to fail.
I think the common thread here, and what separates this from internal tools a la "Amazon is 90% Java" is that these tools are Open Source from the beginning and want to encourage community contribution as much as possible. I don't like using go myself, but even I can't deny that if you're running an Open Source project and would like to encourage contributions from people of different levels of involvement and general programming proficiency, go is one of your safest bets.
Those components are either being rewritten in go, or packaged as containers to run on infrastructure that is written in Go.
Normally I would be quite happy riffing with ideas in Ruby or Python, and just running my code and iterating trying to figure out what to do.
When I tried this in Go, every time I wanted to change the shape of my code to try doing things slightly differently I had to do a lot of work to get the code to compile again.
I feel like a professional coder wouldn’t hit that road block because they’d have a better idea what they were doing in the first place.
I have scurried back to Ruby with my amateur tail between my legs, but I hope to try Go again some day and find a style that’s a bit more forgiving of my not-Staff-SWE skills.
Sure, you can’t move quite as fast in go as you can in ruby, but you also get the benefit of not having to deal with errors like “undefined method foo for nil:NilClass” in production, which is nice.
We were....not fans. We were already in the habit of writing extremely clean and healthy code[1], and having a language hand-holding and hectoring us into doing things in its narrow, specific way was just too much of a hit to productivity. Now that I work in a C++ codebase full of very smart domain experts but light on engineering expertise, I can appreciate Go's value in certain circumstances, but if you're lucky enough to work with a talented and experienced engineering team, Go is a pain in the ass for anything but beginners.
[1] A habit I'm paradoxically trying to unlearn a little at my newer company, due to being on a different point on the short-term velocity/long-term health spectrum
A lot of the compiler errors really could be errors in pre-commit hooks (like, the one about unused variables), but the language designers made a trade-off to be Even More Opinionated (than, say, Rails) because they thought it was best from a pedagogical perspective.
Go is not the tool of choice for experienced professional programmers for exactly this reason.
The ability to quickly refactor code is very important and Go fails at it horribly for a number of reasons. 1) sheer verbosity. 2) multiple return makes altering function signatures tedious. 3) no generics mean code is written in hard to refactor styles out of the box 4) poor reflection support 5) poor tooling compared to professional grade languages (Java/C#/etc)
I could go on but long story short it's not you, it's the tool.
These issues can be overcome by remaining vigilant but for the most part they are part of the accepted "cost" of Go.
Which is to mean you pay them in return for a language that is very easy to learn (albeit hard to master because of the above problems), has a great runtime and has reasonable out of the box performance and memory footprint and stellar compile times.
These tradeoffs can actually be right for some teams, i.e larger ones with less experienced engineers that can afford to spend more cycles doing busy work refactorings.
However it's often a poor fit for small teams trying to work on software that is evolving quickly and can benefit from features of more powerful languages.
And one of the absolute dog's-breakfast worst to refactor regardless of editing environment. No statically typed language should be this hard to do basic changes in, and yet it exists.
Being writable in a "simple text editor" isn't a plus when the "simple text editors" in 2020 are IDEs with the name filed off.
As long as you never refactor anything.
> Rust is among the hardest due to its complexity and demanding static analysis
I have never wanted or needed to offload Rust's "demanding static analysis" to my text editor. Rust's linearity/lifetime type system is really quite simple - it's not like you have to keep track of long-distance lifetime relationships in your head.
Even a reasonably powerful type system with e.g. HKTs, arithmetic, etc. like Haskell's rarely benefits from external tooling, and any interactivity can often be just as easily done via a polling build tool like ghcid rather than integrating it into a text editor.
The one place where it strikes me as sane to have a "smart" text editor is if you're using a proof assistant-style language like Coq/Idris/Isabelle/Liquid Haskell/etc. where there's actually a non-trivial amount of interactivity required while writing the code.
The demanding static analysis and clear compiler errors actually make it easier and more robust to write unaided for me. Especially when it comes to refactoring -- it's essentially a process of resolving the compiler errors and you can be pretty confident you've arrived at a correct result.
Source: I write Rust everyday in a plain text editor without syntax highlighting.
[citation needed]
i.e Go is relatively well suited for a simplistic web service. But even if I don't expect to need to extend it I would still prefer to write it in something that is more extension friendly than Go.
Therein lies the problem. Go actually isn't better than other tools at most things so if you don't actually have a tool of choice than Go is rarely the best choice assuming you are comfortable with a large range of tools.
Building a database? C++/Rust (arguably JVM) are better choices. Building a correctness critical piece of software? Rust/Haskell are likely better choices if this is your main constraint. Building a big data pipeline? JVM language of choice is probably the clear winner because of sheer ecosystem momentum. Building a large web application? JVM/C# are miles ahead of the competition here. Building a bunch of microservices? This is probably one case where Go actually does reasonably well, if you work hard architecturally to constraint to complexity of each service it can do OK here. Unfortunately, just OK. JVM/C# probably still beat it out here because of better tooling. Building a kernel? C/C++/Rust/D/Zig. GC is a non-starter. Building a browser? See above.
Once again, I could go on. Go can be used for lots of these but all of the tools above are better for each of these domains. So if you are an experienced professional with a wide array of languages at your disposal what are you going to pick Go for?
Think about successful tools use Go - they would have been a nightmare in some aspect in most other languages. Can you imagine Kubernetes in Java? Docker in Python? It shines where it shines for a reason.
Yes, I can, that is how it was originally prototyped and due to pressure of Go advocates on the team it got rewritten in Go.
However given that Go type system isn't as advanced as Java, the result wasn't the best one.
"The clusterfuck hidden in the Kubernetes code base"
https://archive.fosdem.org/2019/schedule/event/kubernetesclu...
"Fixing the Kubernetes clusterfuck"
that's really funny, because I always thought of Go as the language that programmers who were aging out flocked to because it has
1) google-cred
2) familiar algol syntax
3) close enough to C++ to make a run for it
4) (systems-wise) safe enough to make it worth it
5) hip enough that you don't seem like an aging out programmer anymore
(I myself am an aging out programmer, that ironically just got started, so take that as you will)
But my observation is from 5 years ago and things may have changed.
1) I don't think Go is in the Java (pre 8) realm of verbosity, but I concede it's not as compact as say Python. 2) Technically you can write single return Go but idiomatic Go leans towards immutability so it's not recommended. 3) I've run into the lack of generics a few times, but personally haven't been too hampered by it. This is an entirely valid complaint, hopefully support will be added soon! 4) Some would argue this is a feature. ;) 5) This was more true a few years ago. Now with Go Modules for package management and VSCode + Go extension with full debugger freely available, tooling is much improved.
I cut my teeth in the 90s with C, and have professionally used PHP, Python, Java, C#, Javascript (both server and client side), and have been using Go as my main choice for the past 4 years. I've worked on teams as small as 1 up to teams in the hundreds, both as an individual contributor and tech lead. I say all that because I would consider myself an "experienced programmer". I am mostly writing APIs for web and mobile apps to communicate with, although I have written code in a wide variety of contexts. I say all of that because I can't imagine going back to Java or Python. I even migrated a bunch of my services from Typescript to Go because I found Go to be far more productive and easier to work with. Refactoring as always been easy. At my current job as a Chief Software Architect, I have over 25,000 lines of code in production running Go. I say all of that to provide background and context on my humble opinions.
1) Verbosity is among the leanest I have seen for a type-safe language. Compared to Java or C#, it is much more like Python. Perhaps the most verbose sections are a repetition of "if err != nil" blocks, but those are there for a reason and I explicitly prefer that over long "try catch" blocks.
2) Not sure what you are arguing here. If you are talking about functions returning multiple values, many languages do that. If you really don't want to, return a struct or a map. Or choose to not return multiple values. That is a choice of the programmer, if I understand you correctly. You could also use any of the major editors (I prefer VS Code) to refactor; the tooling is awesome (see #5)
3) Maybe it is due to the nature of what I do, but I have never once thought "man, I wish I had generics for this." I am sure there are some who genuinely need it, but I haven't run across that in my own experiences. I find that structs and interfaces have been everything I needed there, but again, my use case is on the server.
4) I haven't needed to use reflection either, so I have no comment here.
5) Hard disagree. GoLand (by Jetbrains) and VSCode are INCREDIBLY powerful. I will never go back to Eclipse or Visual Studio if I can avoid it. Go ships with tons of tools (formatters, linters, vetters, etc) and integrate with a ton of others (delve, etc). So your statement simply isn't true.
Your last statement is probably what drove my response. I was able to spin up a full API server incredibly quickly alone, and then with one other person, in a fresh startup with minimal issues. Rules and requirements were changing daily, so the code and business logic had to change. I can't imagine having to do that kind of work in Java (don't get me started on the deployment story here...). I would argue Go is a great fit for small teams. It is a small language with opinions, so programmers can focus on their use case.
Go isn't perfect, but it's my tool of choice. That's my two cents though.
1) This is hit and miss for me. I have to read Go code frequently: not every day or even week, and sometimes under incident pressure. It's less verbose than Java/C#/JavaScript, more verbose than Python/Ruby/Scala. But it's sufficiently verbose to require, for me, cognitive discipline to not skim (ie, "I've been skimming, need to go back up" awareness). So when I'm reading go I put my Java/C#/JavaScript hat on not my Python/Ruby/Scala hat. Not a big problem, but it was so, so close to being in the latter camp. For me then, its regularity claims are something of a wash such that I'd like to see some supporting empirical data or summation of anecdote that it does in fact optimise well here and make good engineering tradeoffs.
2) The problem (I think) is being being to skip by the returned values. It's better than return codes in C, but you're not absolutely required to handle them. I suspect trying to patch structured returns onto an algol-like is a dead end, and if we want structured returns, we want to use a language that really induces us to 'process it forward' eg via map/flatmap/case handling, rather than adding ceremony around if. I think Go made sensible and realistic choices here.
3) Lack of generics are a pain, but a pain for writers. Go does not optimise for that concern, except afaict for its core language creators who have escape valves. The conspiracy theorist in me says Go was wait and see on generics (and exceptions, and objects) as a function of when it was incepted, and may now reasonably conclude that generics are here to stay so the language will need to solve for them, but objects and exceptions were good holds. ie, I don't completely buy it required a decade to figure it out, but could buy it took a decade to observe generics are not going away.
4) In Go, this seems to be satisfied by codegen and templated YAML. I wonder if this is generational, in that the need for reflection might be lessened because of how work on cloud/virtualised systems happens (networked services handling data in and out, with regular deployments).
5) Agree. GoLand I find a great tool for reading and delving into programs, and VSCode is a fantastic 'every day carry' for code.
Edit: C# is actually quite expressive too, I have a hard time believing Go is less verbose than C#.
That said right now I get little opportunity to use it because it doesn't fit into the current ecosystems I am working in. Shame because it really is a joy to program in, tooling is excellent async programming in C# -just works- which isn't the case in Rust, Java (or Kotlin) or anything comparable.
So while I don't get to use it often I hold C# in high regard.
1) there are worse languages in that category. some are even more widely used.
2) that doesn't make sense. return arguments are part of the argument list signature. either changing any argument makes this tedious, or none does. you indicated that only return ones do for some reason.
3) can't argue there. in the rare occasion that type asserting interfaces is required because of lack of generics, things really do suck.
4) can you elaborate on that?
5) only thing i found lacking was absence of advanced refactoring tools. for everything else, the available tooling seems to be on part with what i've used with java (anecdotal)
- configurable sandboxing
- hot reloading
- AspectJ (cross-cutting code injection at link time or during classloading)
- JMX (extensible RPC API for metrics and administration)
- YourKit (interactive remote profiler wired in during classloading)
That's fine, in most cases, writing python code, you find out that you fucked up when you run the script. Compiler errors catch whole classes of errors that you ordinarily wouldn't find unless you ran the code.
I should add that it’s heartening Pike actually acknowledges this in the interview, regarding limited typing and how it helps playfulness in languages like Python and Ruby.
I mean, I don't typically prototype or iterate on ideas in Typescript, either; I do that in Javascript, and reserve strong typing for when I know what I need to build, and it's just a matter of doing so in a way that's reliable, maintainable, and secure.
The point is that flexibility and strictness both have value, and it's more worth knowing which is more useful at a given point in a project's lifespan than trying to be dogmatic about one or the other.
Ha. Try Rust if you like a challenge: at least you will get outstanding compiler diagnostics while refactoring, and far better performance as an end result.
It's difficult, but fair. I can not praise enough the software developers that realize proper errors are vastly superior to extensive docs.
It's so silly for example that math.Min/Max only works on float64, it won't work with float32 without casting, or integers (although for ints is highly discouraged, because casting to float could cause you to get a different number, it is encouraged to just do if statement yourself)
It is a horrible language, only popular because it came from Google. It's biggest benefit was that it supposed to be easy to pick up, but it also demonstrates that the simplicity doesn't help at all with a significant code base, you still have to spend the same amount of time to learn it as in any other language.
My experience with software engineering went from "the language (type system, grammar, syntax, etc) is super important!" to "the language is one of the least important considerations in choosing a language". The more important features are tooling and ecosystem.
I don't want to have to learn a new configuration language or imperative DSL just to add project dependencies or setup tests. I don't want to have to learn yet another documentation framework for generating documentation from code comments. I don't want to have to spend lots of time setting up CI/CD pipelines to build my source code and documentation packages and publish them to their respective repositories. I don't want to have to worry about whether my users or fellow developers have the right versions of dependencies or the runtime installed on their system. I don't want to have to make unmaintainable Dockerfiles because it's the only way to have performant builds in the happy path. I want to minimize time wasted during code review on comments about style or programming paradigms. Similarly I don't want to waste time waiting for code to compile.
If the language is type safe in 95% of cases, that's good enough--people still make lots of money writing lots of 100% untyped JS and Python and Clojure and so on--the extra 5% would be nice to have but I'm not going to trade tooling, ecosystem, and developer velocity for it.
Basically, as I become more experienced, I begin to understand the difference between programming and software engineering (my heart is in programming, but software engineering pays the bills). Go is a great language for software engineering, but most of its criticism seems to be directed at programming concerns.
Is it though? While tests in go are not terrible, to be a truly effective software engineer (vs just a programmer or dev) you want a very good test and documentation story, and there are languages out there with far, far, better tests and documentation primitives than Go.
Further, I contend that Go has excellent documentation primitives, notably that it doesn't have any sort of markup or documentation package management. I've never seen a better language in this regard.
Similarly, with documentation, I really like that the godoc toolchain has resisted all the markup and annotation syntaxes that you see in Java. And I like that Go's convention of using a public URL as the module import path helps enable sites like godoc.org.
I find the language itself slightly more than mediocre. But I really like the toolchain and the restraint of the maintainers.
Docs for elixir are pretty, too: here's a library that I wrote. The doc system is extensible enough that I could patch in docstrings from another language and also hyperlink (the </> buttons link to github) the GitHub code across languages (the `zig structs` section in the sidebar is autogenerated by parsing zig code) https://hexdocs.pm/zigler/Zigler.html
My joke is that you want to write documentation that's pretty and html-responsive enough that you can read it on the can between coding sprints.
I agree Go makes things more difficult to mock, but it's also the language I've used where it's easiest to avoid mocks and test the real thing. We have tests for REST and RPC services that... just make a service. There's relatively little boilerplate to make a gRPC or HTTP service with some basic expectations/fake data attached and just run your real client against that real fake server. ("Relatively" meaning, less than the equivalent mocks in Java, for a substantially more realistic test outcome.) And once you have this fake server, it's probably useful in multiple projects, while mock setup tends to be suite- or even case-specific.
Sometimes with the right approach (mapping PostgreSQL calls 1:1) and right tooling (uvloop, Cython) you can write code in Python and still squeeze better performance than in Go[1], but you need to know what you're doing.
If you want to explore types, these resources[2][3] were useful for me.
[1] https://github.com/MagicStack/asyncpg
The more I work in JS the more I realized that I'm dumb. I'm not worthy to hold the power that JS has, much more other more complicated languages.
I tried Go. I like it. It makes me feel safe and smart enough to write production grade app.
Small docs, small language, no need fancy IDE, just any regular text editor like Vim works. Easy to read, easy to reason about. Everything is easy.
Now I finally can focus on other non-programming passion instead of chasing endless language features in other programming languages.
I like Go. The language for mediocre SWE like me. I code, I go home, I enjoy life.
I think this is really the best feature of Go. It's a pragmatic language. The only complain I have is the incomplete/half backed reflect package.
As a user of Go and Ruby I can tell you that: Ruby is an awesome language, and there's nothing else that beats it at getting things done quickly and painlessly (unless you have a mathematical mindset, in which case nothing gets shorter and more elegant (at least the day you write it) than functional programming). It has many other defects, like safety in larger codebases, speed, some weird quirks, etc., but it's an awesome language, don't expect many other languages to be less painful than Ruby (on small programs at least).
Don't listen to those that say professional programmers won't use Go. The only reason a professional programmer will say hard no to Go is for extremely optimized or mission critical code that can't tolerate garbage collection (even though the GC in Go is very low latency) or need a more sophisticated type system, or numerical applications for which Go isn't really well suited.
If you compare Ruby and Python to Go, the transition might be hard if you haven't studied computer science formally. That said, it's a matter of making the switch, understanding that all those objects (boxes to put data in) that you had in Ruby, now are statically typed; this means that a box is of exactly one type and can't change midway (and even subtle changes need explicit casting/conversion). You also need to learn the difference between composition and inheritance, which might be tricky depending on how you learned to program... but beyond that, Go is a very small and easy-to-write-code-in language. You might like the syntax more or less, but it's easy to write good code in Go. It has many other gotchas and hoops you will need to jump through (var initialization, error handling, new vs make, weird append/len, learn to printf, learn pointers, etc), but it's a good step after Ruby / Python, if you need things bigger than what Ruby / Python allow you to comfortably manage.
A hello world might require defining a package, importing another package, and a more verbose print statement than python / ruby, but those are all great (and minimal) appliances when developing at a larger scale.
Things like lack of generics and type system makes you repeat things quite a lot (e. g. It was pretty impossible to write a true ORM or a DI system in go). I found the interface system quite awkward in that types implicitly implement an interface if they have the same method signature was weird to me. I think method overloading wasn't supported either, if I remember correctly.
I think people who loved Go initially probably came from C. It feels a lot like C with better standard library, memory safety and fast compile times. I like Go for the same reasons actually. But until there's at least some way to write generic code, it's an awkward language to anyone coming from more expressive languages. I program in C# mostly which is an extremely expressive language with classes, interfaces, generics, expression trees, linq, delegates (akin to function pointers). Go is far behind other languages for application development, my two cents. But for systems programmers used to C which is a very small language, Go must be like a transition to paradise :)
The build time of C++ can be really long. I noticed it too.
I'd say, invest your time wisely. Things are changing rapidly and old languages are usually solving problems that are not quite as important anymore as they used to be.
Im thinking of may be Rust or Go right now for more pragmatic reasons. Thank you for sharing your opinion.
When I started with Go I thought it was awesome. As my projects grew I found them to become less maintainable than my previous Java projects, and still less maintainable than my newer Rust projects.
I see the same .net C# stack, python, Ruby...... Java and PHP.
Facebook, Dropbox, Uber, Apple and literally thousands of other companies, big and small, use Go.
As for Dropbox I was convinced primary language was Python.
Their switch from Python to Go was pretty well documented on their engineering blog, but I can't find it, other than comments on HN like the one above.
Just like they use JavaScript, Swift, Java and Kotlin for mobile development.
Regarding Uber, one tree doesn't make a forest.
https://github.com/cloudflare?q=&type=source&language=go
Compared to the 11 projects they have released in Rust:
(My team actually uses Go, and almost ended up using some Rust, but decided in the end not to, for good reasons.)
Care to elucidate?
We needed to start a new service, and there was an initial prototype of the beginning of that written in Rust. Then, the pandemic happened. This means priorities shift. It makes more sense to not try to learn new things while trying to ship new things when there's so many other things going on. Additionally, it was going to be a bit harder to do it in Rust because there were already some useful code lying around in Go for some things that we'd have to port.
I am a PM these days so I have no real bearing on these decisions, but I support my team in their choices, and told them that I would no matter what. I obviously love Rust, but it's not the end-all be-all of languages, and I think the right call was probably made.
The sorts are in stdlib.
I know Docker is written in Go, and I believe the Kubernetes ecosystem uses it heavily, so those are compelling evidence for Pike's argument by themselves.
Deno comes to mind.
"The clusterfuck hidden in the Kubernetes code base"
https://archive.fosdem.org/2019/schedule/event/kubernetesclu...
Also they are quite heavy users of //go:generate.
At least, I think, k8s proves the ability of Go runtime.
We actually did write and deploy a small Go rescue agent for bare metal machines in Ironic. I'm not sure if that every got upstreamed though.
Rust has some obstacles to overcome to supplant Go on a wider scale though. Namely compile time and learning curve - especially in relation to concurrency and async which are key features to replacing Go as it's main draw is it's green runtime.
Yes, this is true. I programm in Go professionally, about 50% of my work time (other half is python). But in many cases, I have found that copying the same code to other places doesn't have that much negative consequences as one would believe.
I suppose I should say that there's a lot of different types of duplicate code, some of which really don't matter per se. Sometimes two bits of code are basically the same but are completely unrelated and as things change they will diverge, i.e. it was just incidental. Code that is duplicated with a strong functional relationship make changes a nightmare though.
On the other hand Rust hasn't had an oportunity for a killer app, in fact it's main sponsor project Servo/Firefox is pretty much the opposite. It's a browser, the definition of a complex and slow moving behemoth of a project.
I attribute Rust's slow takeoff compared to Go mostly to this phenomenon.
After Docker was written in Go and started popularising containers outside of the lxc/openvz/other hardcore guys doing their own stuff directly on top of the kernel APIs then it was only natural that stuff would want to integrate with Docker and using the same language was a way to capture some of that enthusiasm. This resulted in software like Flynn, Deis and Kubernetes all picking Golang. The resulting network effect is very strong.
To some degree this is also down to the strength of the Go standard library. It has an excellent HTTP and TLS stack, encoding/decoding etc. All of this made it a good fit to write software that is fundamentally fairly "dumb" quite quickly. i.e software the mostly shuffles bytes from one socket to another.
Rust will most likely not find such a killer app so I think people need to be realistic about how long it will take Rust to find a marketshare. The big difference though is one can actually make good arguments to use Rust over C/C++ where you can't in most cases for Go. C is the largest software ecosystem right now and Rust is best positioned to take some of this market share away.
However while I like Rust, it seems that it will become more of a C++ companion than a replacement, as it will take quite some time to get a foothold on systems where C++ is finally replacing C as of the last decade.
Rust does not have a comprehensive story for CSP concurrency like Go has. It does have a fair share of libraries/ecosystems (like Tokio), but not having something built in means that you have to hunt for the right set of libraries to work together. Meanwhile, in Go, every single library will just work with its concurrency model out of the box.
If anything, I find Rust returns flexibility to me in programming choice that Go made me forget I had. There are places where library support in Go is still richer, but language support has been stellar across the board in Rust.
It's easier to write code with standard threading tooling but I think CSP is much, much easier to debug and avoid deadlocks.
This has (still to my surprise) not borne out in real-world cases, at least to the extent Go channels "do CSP".
https://blog.acolyer.org/2019/05/17/understanding-real-world...
For writing a normal networked http/rpc/event stream service Go is probably more than good enough. It’s not a winner-take-all market.
A great example imo is Buoyant’s Linkerd. Rust dataplane proxy as well as a control plane written in Go. Two different requirements for each of those services drove the language requirements
Each language is a much more than its syntax. It is a complete ecosystem with network effects.
Rust can overcome go, if it would provide a lot of open source libraries, as well as growing number of programmers.
To do that, it would need the backing of big tech (e.g. MS), that have a big budget to evangelize it.
In any rate, the choice of language today is completely meaningless, since you can wrap anything in a container, and put a GRPC interface on top of it.
So the choice of lang should be based on the libraries for the task at hand and not on the lang itself.
For some reason YAML seems to be an attractive option to try to implement things that really should be imperative in a declarative sense. Other configuration languages are either semi-imperative (like the hashicorp stuff) or don't do that (I haven't seen TOML configs that do some of the thing that YAML does). My best guess is it's purely cultural, because that trend started with ansible, which, not to denigrate - in many ways did very good things.
The trouble is that 1% of the time always hits at a very inopportune time.
YAML is no different. You encode some configuration, something else makes it so.
Ansible kind of muddied things up because it looks like you're doing imperative things in YAML, but actually, you're doing declarative things and Ansible is interpreting your YAML in some particular way.
Is this really the case? I thought the prevailing wisdom is that Go was designed to be as simple as possible so newbs can pick it up faster. I find Go to be quite easy to read and follow along with even though I've never written a line of it. Rust on the other hand...
Its the devops language in last 2 places i've worked.
For others the original title was the title of the page:
Rob Pike interview: “Go has indeed become the language of cloud infrastructure“
Go was originally envisioned as a systems programming language. It was often called "a better C". This exposed Rob Pike's lack of experience in the area (IMHO) because anyone who had done any systems programming at all knew that garbage collection made any systems language a nonstarter.
Where Go succeeded was completely unintentional (as as often the case): it was (and is, IMHO) a better Python. Go hasn't attracted C, C++ or even Java programmers. It attracted Python refugees. So it's no accident the touted use cases ("cloud infrastructure") are orchestration problems.
But at the same time, "cloud infrastructure" here refers to what, exactly? Kubernetes? Etcd? I think those could've been written in anything. More to the point, users of those don't really use Go as a result, right?
I think even Google found that internally Go was cannibalizing Python usage and little else.
Lots of Go newbies bemoan the lack of generics. I was never one of them. It's not the problem people think it is (IMHO). I'll be interested to see what this solution is even if simply means we can stop having having this conversation about Go not having generics. That alone is a win.
So I think there's two areas where Go screwed up. two big and one small.
The small one is the lack of IDE support. You don't need to write an IDE. You just need to have sufficient integration into Jetbrains IDEs (at a minimum).
The first big problem--and it still is a problem--is the dependency management, in that there wasn't a solution to this out of the gate (unlike, say, Rust and Crate). Like I was shocked the first time I saw import 'github.com/some_random_user/...' I dare you to go look at any even moderately sized Go project and unravel the levels of dependencies (including repeated dependencies, which may or may not impact binary size, I'm not sure). It's kind of a mess.
The second is, oddly, on concurrency or rather multi-core and multi-CPU utilization. Aren't we still in the GOMAXPROCS environment variable era? Really?
EDIT: adding this quote from Rob Pike in 2012 [1] as it's a first-hand account of Go's original design goals:
> I was asked a few weeks ago, "What was the biggest surprise you encountered rolling out Go?" I knew the answer instantly: Although we expected C++ programmers to see Go as an alternative, instead most Go programmers come from languages like Python and Ruby. Very few come from C++.
[1]: https://commandcenter.blogspot.com/2012/06/less-is-exponenti...
Kubernetes was originally written in java and was transpiled to go.
Yeah, Rob Pike and Brian Kernighan definitely lacked experience in C and systems programming.
Rob Pike is probably most famous for his involvement in Plan9 (prior to this). I honestly don't know what this experience entailed.
What he seems to lack IMHO is sufficient curiosity about computing innovations not produced by him or the prestigious institutions he has worked for (Bell Labs, Google).
For example, someone might dismiss IDEs as being unnecessary because technically the same job can be done without them by a superior human being, so the solution is to just be a superior human being.
Or as another example, someone could completely dismiss syntax highlighting as being useless for the superior, implying that the solution to the problem syntax highlighting solves is to just also become a superior human being.
There are multiple layers to any solution, and if one of them is missing, the others are kind of shaken out of balance.
Rob Pike worked on Unix, helped design and implement Plan 9 and Inferno, and co-designed UTF-8.
Personally, I prefer acme; it's my daily driver editor.
Please, kid.
[1] http://doc.cat-v.org/plan_9/4th_edition/papers/
EDIT: In the past half hour, the original website went down, I don't think it is related to my link. Here is a different link with the same contents:
You don't write an UNIX derivative with a lack of system programming experience.
I think he meant Ken Thompson.
This is quite funny.
I remember when Go was released its creators were all like "GC is good enough now" and then Rust came around the corner blowing that whole premise out of the window.
The way I like to put it is Go devotees operate under 2 rules:
1. Whatever your complaint about Go is, it's totally not a problem (eg stop-the-world GC pauses, a longstanding issue that any experienced Java engineer could commiserate at length about);
2. Even if it is a problem--which remember it is not--it's totally fixed in Go 1.(N+1) where 1.N is the current version.
> eg stop-the-world GC pauses, a longstanding issue that any experienced Java engineer could commiserate at length about
This is a really interesting example, because it shows the lack of nuance in the anti-GC position. By your logic, because Java's 100ms GC pause times have been problematic (and yes, I'm aware that Java's upcoming GC will boast significantly improved pause times and that it's theoretically possible to tune a Java GC for low pause times even if no one has managed it in production) Go's 5us pause times will be equally problematic. Because a pause is a pause, right--why should we muddy the waters with considerations like "duration"?
Single- or double-digit microsecond pause times completely change the conventional wisdom about the kinds of applications GC is suitable for. There are still lots of hard real time applications that Go's GC won't be suited for, but there are many soft real time applications for which Go would be fine (or at least the issues will be libraries or support for target architecture/platform and not GC).
I wouldn't call my position strictly anti-GC. I would simply say that GC isn't the panacea it's made out to be and it is at least in part a false economy for at least three reasons:
1. You may need to tune your VM/GC parameters (as you mention). There's a cost to this;
2. GC can make it harder to identify memory leaks (which are still a problem through, for example, dangling references). I've worked on C++ systems where we instrumented many things including process size, which was basically just a flat line. As soon as it wasn't, you knew there was a problem. And this was for a server that read about ~6GB/second off a NIC.
3. Crashing a program due to accessing unallocated memory tends to give a more immediate signal of a problem, which is more often than not better for development.
> More to the point, users of [Kubernetes] don't really use Go as a result, right?
Yeah, they do actually. The vast majority of tooling in the Kubernetes space is also written in Go. I don't know about etcd.
> The small one is the lack of IDE support. You don't need to write an IDE. You just need to have sufficient integration into Jetbrains IDEs (at a minimum).
I might be missing what you're saying here, but Goland has been around for a decent while now, and the full IntelliJ plugin has as well. Before that, Atom or vi with some plugs I've since forgotten the name of worked well enough.
I agree with your overall points though, especially on dependency management.
Right. That's true now (and, like you say, has been for some times) but was not the case for years. I was responding to this point in the interview talking about initial development, not the current state of the world, just to clarify.
Also a better NodeJS and PHP.
Very interesting, can you elaborate with an example?
I also found there are subsets of Go aimed at filling this use case because of the GC issue [4] (eg Emgo [5], TinyGo [6]).
The Fuchsia Language Policy [7] also points to this. Fun fact: the networking stack in Fuchsia was written in Go because of Dave Crawshaw, who was (and I assume still is) a big Go advocate. I got into an argument with him once about stop-the-world GC pauses in Go. Anyway, the Fuchsia project has largely been unhappy with this ever since because there is an overhead to using Go that essentially you can't get rid of. Using memory and object pools can solve many but not all problems. And at some point I suspect they'll replace the networking stack with something not written in Go.
[1]: http://www-cs-students.stanford.edu/~blynn/c2go/
[2]: https://news.ycombinator.com/item?id=2535206
[3]: https://news.ycombinator.com/item?id=2535386
[4]: https://github.com/golang/go/issues/29147
[5]: https://github.com/ziutek/emgo
[6]: https://github.com/tinygo-org/tinygo
[7]: https://www.reddit.com/r/golang/comments/f92nkc/golang_is_no...
> Go was originally envisioned as a systems programming language.
Already with the release of Go 1 it was quite clear that Go has its best fit as a Server language. I don't think the usage of a language can really be planned. But looking at the previous works of the language authors, it's clear that they worked on solutions for distributed systems.
Also I think like 10 years ago people would have written stuff in C++ that today nobody would even remotely consider writing in it.
CLIs as well. Before Go, how did you build a native CLI in a garbage collected language?
Something like Java, .NET, OCaml, Haskell, D, Scheme, Common Lisp?
Yes, it isn't a typo regarding Java and .NET, there are AOT compilers for them since around 2000.
Visual Studio Code with the Go extension is very good.
Unless you want to bucket all systsme programming in to one bucket, I will say this
#1 If your goal is to make the most efficient and fast code (garbage collected or not), it is not clear gc-less languages are always better. Sometimes, not free-ing the memory at the first possible time will actually be good for you overall.
#2 If you are writing anything with serious mem usage in Go, you will optimize for allocations using sync.Pool or some other form of reuse. I am sure well written C++ code would have to do the same tricks. reduce >> reuse >> recycle. Both GC and free() are recyclers. Sure, one could argue GC could provide more control.
I 100% agree there are cases C/C++ or Rust would be better for 20% of the trickier situation, but let us not base it all on the presence of garbage collector.
Heck, some of my Java code works faster than even my Go code. I don't blame Java for having a garbage collector.
Which is why languages like C++ let you write and use your own allocators. There are lots of reasons to do this (eg word alignment).
And while I agree that it's certainly possible that delayed freeing may be beneficial I'm not convinced it's a problem in practice, at least when you compare it to how often common GC problems are (eg memory leaking (admittedly also a problem with manual allocation), out-of-control process memory growth and STW pauses).
#2 If you are writing anything with serious mem usage in Go, you will optimize for allocations using sync.Pool or some other form of reuse.
True, and I mentioned this in another comment: the idea that you can get around many GC issues by using object and memory pools.
> Heck, some of my Java code works faster than even my Go code.
Why should that be surprising? The JVM at this point has >25 years of engineering effort into it and (IMHO) it doesn't get half the credit it deserves from the devotees of the latest flavour of the month.
You're aware that the Go allocator is written in Go, right?
I've said this before, but even back in early 2014 Rob Pike had said that he regretted the term "systems programming" because people misunderstood him to mean it as a language for writing operating systems, when what he meant was a language for writing servers, although that later evolved to cloud infrastructure.
He answers this at 6:50 here, https://channel9.msdn.com/Events/Lang-NEXT/Lang-NEXT-2014/Pa...
What exactly do you mean by systems programming? Many successful databases (Cassandra, Kafka, ElasticSearch) are built on the JVM.
What I mean is things that are basically kernel-level or embedded.
https://www.ptc.com/en/products/developer-tools/perc
As does Go actually,
If it is not user visible but exists to manage app access to hardware resources, it's a system. So a memory management system would not be an app. Neither would power management or IO or a scheduler. But these systems might have administrative apps that admins login and use (like Microsoft's "Control Panel" app -- to control settings for the system code. Similarly some apps might have their own memory management system. But the principle still applies.
I understand there is definition creep as more and more layers sit between the user and the hardware, so todays application code might, in some cases, be viewed as tomorrow's system code, but this is the original meaning.
Honest question: other than parallelism and performance, what are the main reasons someone would switch from Python to Go?
I suppose the static analysis is quite cool too...
Go makes it easy to produce binaries for any target platform that execute without requiring a working Go installation.
This ease of packaging makes Go my default language when I'm writing developer tooling - for example, if I know that team-mates are going to have to run tools that I write but I don't want to make assumptions about their development environments.
There is one caveat: This only works well if you avoid platform-specific functionality (e.g. syscall package) and CGO.
IMHO Go is still inferior to Python for solo-developer projects.
My personal view is that dynamically typed languages are largely falling out of favour with Javascript being the one glaring exception (and even there if I were starting a project today I'd use Typescript).
On the single deployment binary issue, let me just underline that point with one word: virtualenv.
can you expand ?
Given that the language is relatively easy to grasp and that the standard library covers all the basic needs of a server application, that alone is a huge win.
Python still has more and better libraries for certain tasks though.
And in my opinion, it is a better Java. It succeeds in the niche where Java is used today: big softwares, shared by numbers of teams, working on different timezones and in different languages, and with varying degrees of skill.
Go is so stubborn it takes the fun out of programming and ironically that's a good thing: when code can't be tied to anything personal, it is much more easily shared with anyone who works on it. I have found that writing Go itself is definitely tedious, sometimes boring, and the fun is not there; _however_, fun is had when you have a working binary that is statically compiled, reasonably devoid of bugs and surprisingly performant. You take joy in actually using pieces of software, not in fine-tuning your formatting rules.
The language itself is boring, but it's the same boring for everyone and it happens to be working well so you tend to focus more on the solution than on the intricacies. This, IMO, is the success of Go
Better Java 14, definitely not.
I stop copy pasting code and using code generators around 15 years ago.
It's succeeded at basically the same niche that it was intended for. The difference, as another commenter points out, is Rob Pike's ignorance of how the world outside of Bell Labs & Google had changed. In the 80s you would've built networked microservices in C. In the 2010s you built them in Python or Node.JS. Hence if you have a new language that's a success in the microservice application-server market, it will end up cannibalizing Python and Node.JS, not C. Python cannibalized Java in the 2010s and late 00s, and Java cannibalized C in the 2000s and late 90s.
Go is still inferior to Python in prototyping, in web development, and in data-science. (I used it for the former 2 while still at Google, and decided it wasn't an upgrade.) But it's superior for production microservices, and that's an increasingly large market these days.
I was (2010-2017 to be exact) so I remember how this was communicated but internally and externally. This HN submission [1] to HN from 2011 reflects my recollection of that messaging that Go was a better C.
Even better, Rob in his own words [2]:
> I was asked a few weeks ago, "What was the biggest surprise you encountered rolling out Go?" I knew the answer instantly: Although we expected C++ programmers to see Go as an alternative, instead most Go programmers come from languages like Python and Ruby. Very few come from C++.
> Go is still inferior to Python in prototyping, in web development, and in data-science
On the data science point, I certainly agree that nothing looks likely to displace numpy/scipy anytime soon. Python's much-maligned GIL does simplify C module extensions and, I believe, was a key contributor in Facebook choosing to move to Mercurial many years ago (long before my time).
I wrote a fairly significant internal app at Google in Python and essentially swore I'd never do it again. The way I like to put it is that I don't want to write unit tests for spelling mistakes. I haven't used Go for Web development but I'd already choose it over Python just for static typing.
As an aside, I found the protobuf integration in Python to be basically terrible. Perhaps it's improved in the last 7+ years though.
[1]: https://news.ycombinator.com/item?id=2535206
[2]: https://commandcenter.blogspot.com/2012/06/less-is-exponenti...
† I say this because we already have an alternative to GNU Coreutils for memory-constrained embedded systems, in the form of Busybox. If Coreutils had to "serve two masters"—be both powerful at scale on big multi-core systems, and functional on tiny embedded boxes—then Go would make less sense. (Rust would be pretty good for that, though. See e.g. ripgrep.)
Jetbrains wrote their own Go IDE, goland. The rest of us use an LSP client that talks to gopls. I have used that for about a year and it does everything that I would want an IDE to do; accurate context-sensitive completion, an easy way to peek at signatures and documentation, jump to definition, accurate renames, import organization/cleaning, etc. I am continually surprised at how good a job it does when I'm 8 nested protos deep and it can still complete struct keys and values (i.e. you can type "&Foo{ B" and it will complete it to "&Foo{ Bar: &Bar{}").
> The first big problem--and it still is a problem--is the dependency management, in that there wasn't a solution to this out of the gate (unlike, say, Rust and Crate). Like I was shocked the first time I saw import 'github.com/some_random_user/...' I dare you to go look at any even moderately sized Go project and unravel the levels of dependencies (including repeated dependencies, which may or may not impact binary size, I'm not sure).
I am pretty happy with go's approach to dependency management in the Go module era. You can tell the Internet "a little duplication is better than a little dependency," which Go does, but they will do it anyway. I like that I have tools that can just vendor all my dependencies (mostly so that CI builds can be completely stateless and free from network communication), and that every module I use is declared in go.mod file that the toolchain itself keeps tidy and accurate. The mechanics of an upgrade are a breeze -- find the module and update the version number next to it. If I want to edit some module I depend on, I can just use gohack and it checks out the source code in a convenient location and edits go.mod to make go pick up that version of the code instead of the Internet's version. And, you can at least figure out why you have a certain dependency with "go mod why". Finally, I love that "go doc xxx.Foo" is aware of what modules my module uses, so that I always get the right symbol. I would kill for "typescript doc xxx" or "clang doc yyy". (Seems like other people prefer 80 browser tabs with random websites loaded to do the same thing. I like my command-line, or at least some IDE integration.)
I have used other approaches to dependency management and they haven't made me as happy as Go's. npm is slow and fills my entire screen with garbage, only to eventually inform me that "module A depends on module B which depends on module C which has a SEVERE SECURITY VULNERABILITY!!!!!!! and there's nothing you can do except whine on Github!" I like Bazel's WORKSPACE approach, allowing me to specify the exact sha256 of the compiler toolchain I want to use, and every version of every transitive dependency. Somehow I feel like nobody is that serious about dependency provenance, though, and they invent a lot of clever workarounds to avoid maintaining that file. But at least you can be correct and in control if you want to. (With go projects, the biggest problem I've had is keeping protoc, protoc-gen-go, and the Go proto library in sync. Go doesn't handle non-Go things, and that really sucks. You have manually manage that stuff, and people mess it up. Or use Bazel, which I've tried and came to the conclusion that nobody else has ever done.)
So all in all, I think Go's IDE support and dependency management meet my needs. I dare say I think they're great! All of this stuff is relatively recent, though, so there are probably a lot of people stuck with their code in ~/go/src/github.com/me/my-app that manually run "goimports" from time to time. That sucks. But there are better tools available!
Yeah, I should clarify: I was responding to Rob's point in the interview where he was talking about tooling and IDE integration from the outset rather than the current state of the world.
As for Jetbrains writing their own IDE... well, yes and no. Sure there's a separate product from a user perspective but IntelliJ has had a Go plugin for several years. You'll probably find Goland (just like all the other little-IntelliJs like AppCode, PyCharm and RubyMine) are little more than stripped down versions of IntelliJ's code base with a different default set of plugins. My point is that Goland wasn't a "new IDE from scratch" level of investment by Jetbrains and that a third-party plugin could've been largely equivalent.
You're right that go didn't have amazing tooling from day 1, but the tools it did have were enough for me to change my workflow. I never cared about autoformatting (too many knobs to turn) before gofmt, and now I can't live without autoformatters.
Coming from a dynamic language background, I also never really got tools that claimed to provide completion to work, so I ignored them for most of my career. I worked on a Java project at Google and Emacs worked fine. Now there is LSP, and I first tried it with gopls, and it worked perfectly. Now I can't live without it, and try to avoid having to work in languages that don't have a good LSP implementation. (I am also very annoyed by the multitude of C/C++ projects whose crazy bespoke build systems manage to make clangd not work. That's the reality of tools coming along late; sometimes it's too late to have an impact because people have gotten by without the tool and are happy to continue with that.)
I have also used a little Goland. At my last job, we replaced a lot of PHP and Expect scripts with Go programs, and it was quite the journey to keep people happy with their existing IDEs. (Everyone had the Jetbrains you-get-everything pack, so at least Goland was free.) Goland is really a from-scratch implementation of editor support for Go. They had no other choice... but it makes different decisions from the open source tools which is quite the pain point. (I remember it importing opentracing-go as a plain "github.com/whatever/opentracing-go" and let things in the file refer to that as opentracing.Whatever. goimports wrote it more explicitly as 'opentracing "github.com/whatever/opentracing-go"'. I still don't know how that works, but apparently it did, and so diffs would always oscillate between the two until we convinced Goland to just run goimports.
What I like about go's tool support is that it's done a good job of adding features for people that want them, and letting those people use the tools without upstream dependencies knowing about those tools. I started using go modules before it was particularly popular in the community, and it didn't cause much trouble. (People sure liked to force push to their version branches, though, causing go.sum checks to fail. And people would always blame the Go team for that, unfortunately.)
I have a hunch that not many people are using gopls, but it works perfectly on codebases where I know the authors never used it. That is a pretty big deal and is better than what a lot of other languages do. "We have this new thing... restructure your codebase to try it out." It's nice for new codebases, but even nicer when you don't have to do that.
So all in all, I am very satisfied with the tooling situation.
You don't know what you are talking about. Someone who has written operating systems, windowing systems, editors, many many tools & library functions, and well regarded books on programming doesn't exactly lack systems programming experience.
There are a significant number of systems that demonstrate otherwise: LISP machines, SmallTalk, Oberon.
Are you aware that Jetbrains makes an IDE for Go, called GoLand?