How Go was Made
sourcegraph.com
sourcegraph.com
Why is there a massive skew favoring Googlers having their proposals implemented compared to plebes? Did the stats change after rob's talk in 2012?
Why is there a massive skew favoring Googlers having their proposals implemented compared to plebes?
The AOSP acceptance decisions are made by Google employees. You are free to fork android, but Google favors features that are aligned with their objectives for the platform. See this thread for a tangible example: https://news.ycombinator.com/item?id=8803118
For more about Go and open source, see my opening gophercon keynote, at blog.golang.org/open-source.
We haven't completed the details yet. The most recent thinking is at https://groups.google.com/d/msg/golang-dev/4GOiSBCX568/HgyDT....
The closing keynote to the second GoLang conference, where all the keynotes were Google employees?
Asking about Go's reliance on Google deserves a better answer than your Groups thread, or this HN post. I love your writing in the post you linked, but a technical response on process still doesn't say anything about Google The Corporation's influence. Would you try again?
From an external viewpoint, Google the corporation has two influences. Firstly, it employs the language leaders (i.e. Russ). Secondly, Google frequently hires/gives to the Go team experts in some particular field in order to improve the implementation. For example transforming GC for the 1.5 release or moving to SSA for 1.6. As a result, Google probably has a big influence on what the important tasks are to work on. That said, it is still open source, and so outside contributors can still work on what they consider important. For example, the shared library development was all external.
I directly addressed Google's influence in my opening keynote; the text is at blog.golang.org/open-source. But I'll try to answer that interpretation of the question below too:
> Why is there a massive skew favoring Googlers having their proposals implemented compared to plebes? > Did the stats change after rob's talk in 2012?
One reason is that Google pays for the bulk of the Go development, especially development that needs design (for example, the new garbage collector, or core language changes). This is changing slowly: we are seeing other companies begin to pay for Go development that makes Go better in some way for them. Canonical is paying people to add support for shared libraries, which matters for Linux distributions like Ubuntu. Intel is paying people to make Go take more advantage of Intel features; Oracle is paying someone to port Go to SPARC 64. I mentioned this in my keynote.
Another reason is that we weren't clear about the vision for Go. Rob's talk in 2012 was the first crisp statement of that vision. I hope to have elaborated a bit in my keynote. We're still trying to articulate that clearly. I do believe that non-Google proposals have been more in keeping with Go since then, but I don't have data. That talk was an important step by itself, but certainly not a sufficient one.
We believe a significant reason is that we have never properly explained what makes a complete proposal. If you look at golang.org/doc/contribute.html, it is very clear about how to send code but contains almost no text about how to send a proposal or design. We believe that being clearer about what makes a complete proposal and what makes a complete design doc will help non-Googler contributors be more successful at those. Of course, we will have to wait and see what the actual effect is.
This was the point of Andrew's talk: to recap the history of how changes to Go have been proposed and made, and to admit that we realize we haven't been doing a good job at supporting non-Google proposals and are trying to address that, both by being clearer about what a successful proposal look like and by establishing a timely process for answering them and an historical record.
The focus here is the _success_ rate of non-Google proposals. The balance of Google vs non-Google proposals, which you seem to be asking about, depends mainly on how much development is being funded by non-Google companies or individuals. Having a clear process may help increase that, but it would be a secondary effect.
"Diverse teams are more innovative and adaptable than monocultures."
It really follows the pattern of biological diversity in ecosystems: a diverse ecosystem is able to thrive and overcome troubles more than a non-diverse one, which is actually quite fragile.
So when you say that 'groups of different people with different thinking styles' you are implying that, for example, black people think differently from white people for the sole reason that they are black and other people are white. Or in other words, that if you get a random black person off the street you can assume his views will be different from a white person's?
Isn't that kind of grouping and assumption exactly what we don't want to promote? I can't reconcile this in my mind.
Anonymous because I don't want to be shamed as a racist when I'm not, but a genuine question.
You could hire both a rich white guy and a poor black guy, in order to get diversity, and then find out they both love classical music, basketball, knitting, went to the same college and vote for the same political party.
If you're telling me you can look at the color of someone's skin and tell me what their cultural beliefs and background is then you're the crazy racist judgemental one!
For building diverse teams I think selecting people based on their skin color would be pretty dumb. Rather pay attention to their background. But you should be putting their skills first in either case.
If you are determining diversity by sex and skin color, I'm saying that you are making judgements about people's background, tastes, culture, etc that are racist and sexist.
You can't really create diversity directly (people feel comfortable around those like themselves), instead it happens organically because people turn out to be naturally different, BUT you can definitely act against it through bias based on gender, ethnicity, age, looks...
The goal is to let diversity thrive through environments and hiring practices that are free of unnecessary bias, but old habits take a long time to break.
Not at all. Instead, different people coming from different backgrounds tend to have different life experiences and perspectives, which influence how they approach the world and how they solve problems. It's not inherent to their race or gender, but merely a fact of having lived different lives.
If you get a bunch of people from different backgrounds and cultures then you'll have a better pool to draw new ideas and creative thinking from. Much better than a room full of ~22yo middle-class white guys.
You are thinking about Weiqi.
http://cowlark.com/2009-11-15-go/
In all seriousness, though, they could've just independently rediscovered a lot of wise design decisions. Strange if they'd never seen this one studying programming language history, though. I'm with the author wishing they applied the innovations that have happened in recent decades or even new ones. Julia is an example of an imperative one that tries to do that.
Read this summary of Robert Griesemer's talk (The Evolution of Go) then you don't have to speculate: https://sourcegraph.com/blog/live/gophercon2015/123645585015
"Hoare said of it, 'Here is a language so far ahead of its time, that it was not only an improvement on its predecessors, but also nearly all its successors.'"
Quite an achievement. It was had by using only experienced language designers on the committee. Modern committee-driven actions could learn from this technique of using people with brains and experience.
One of their smartest moves was how they marketed it. Many nice language designs happen but don't get any traction. Their design choices, tooling efforts, and big company support have been much better than the Java rollout and many minor attempts. My hope for the language is that it replaces the use of C# and Java in as many places as possible so people don't have to learn all that mess in maintenance phase. The simpler language should make for nice legacy code work, I think.
Bookmarking the article.
"The decision process was consensus driven."
"The first Go program was the prime sieve."
"In July 2008, the project became more serious."
Design by consensus generally produces inferior results. The first version of the most noteworthy innovations are often the result of a single person's effort. But it's also evident big projects are the results of teams. You can't produce something big without a team.
So what makes me uncomfortable is that I don't see one of the inventors of Go having produced a rough version one before pulling other people in (but to be fair, I simply don't know).
I'm more suspicious about the first Go program. Was computing a prime sieve the number one problem the Go creators had? Doubtful. When C was being written, the authors were also writing an application they deeply needed: UNIX.
Finally, the thought of making something "serious" spoils the general hack flavor of innovative solutions. It's like a beacon that ideas are about to start being policed, which sounds like the opposite effect consensus intends to produce. It may be a good idea to keep a project non-serious as long as possible instead of optimizing it prematurely.
Well, Ken Thompson did create B...
Why is it necessary to speculate about the design of Go? You can read this and decide directly what you think of its design.
http://talks.golang.org/2012/splash.article
Better yet, look at some existing projects and see how they are structured. Then make a judgment.
Or just walk away... There are plenty of other good programming languages out there.
Design by consensus is different than design by committee. In the latter, there is horse-trading and the like, so that basically everyone's ideas go in. In contrast, design by consensus means that nothing went into Go until all three - Rob, Robert, and Ken - agreed it was right. That's actually a higher bar than design by just one person.
The Go spec came before the Go implementation. That's actually important too - it's not an implementation-defined language - and the prime sieve was written as a complete example program for the spec. It's still a decent example of what you can do with just plain Go, without any libraries. Go was meant for networked servers, but a complete HTTP server would have been way too large (remember, there were no libraries yet!).
Go certainly had a target: the kinds of networked servers that Google builds. The first real Go programs were exactly those servers. But it takes a lot of code to build up to that. For much the same reason, you can bet that UNIX was not the very first C program; something like hello world (or maybe just 'exit 0') probably was. Judging Go by the prime sieve is like judging C by "hello world". Obviously greeting the world was not the number one problem the C creators had.
In the context of the talk, "serious" does not mean what you are saying. Around July 2008 the first draft of the current compiler and runtime started working on both Linux and OS X, but nearly all of the standard library, many key language features, and much reworking of the existing things were yet to come. Serious does not mean frozen here. It means that Go started being a focus of active development for more than just the original three authors. (In particular, Ian Taylor and I joined the team, and development accelerated quite a bit.)
Stepping back, the focus of Andrew's talk was how changes to Go were proposed and made, and how that process has changed over time. For more about the actual design, see Robert's talk, the Evolution of Go (https://sourcegraph.com/blog/live/gophercon2015/123645585015). And again the video will contain more nuance than the notes.
Cute. Nice to keep a dig[1] going. Since that horse has been kicked quite a few times over the years, how about those in the Go community address these language criticisms:
http://yager.io/programming/go.html
For specific examples from the aforementioned link:
The "correct" way to build generic data structures
in Go is to cast things to the top type and then
put them in the data structure. This is how Java
used to work, circa 2004.
Go has the null pointer (nil). I consider it a
shame whenever a new language, tabula rasa,
chooses to re-implement this unnecessary
bug-inducing feature.
1 - "[C++] certainly has its good points. But by and large I think it's a bad language." (source: https://en.wikiquote.org/wiki/Ken_Thompson)
2 - http://www.wisdomandwonder.com/link/2015/tony-hoare-sorry-fo...
"It is an extremely common occurrence for a programmer to accidentally forget to account for the possibility that a pointer may be null, potentially leading to (at best) crashes and (at worst) exploitable vulnerabilities."
Null pointers won't result in exploitable vulnerabilities in Go (not counting straightforward DoS-by-causing-the-program-to-panic). They might, in very rare cases, result in something exploitable in C or C++, because dereference of null is undefined behavior and the compiler is free to optimize accordingly. But in Golang the language semantics require the runtime to panic on nil dereference, so that won't result in something like RCE as it technically could in C or C++ (but isn't likely to).
While you thoughtfully address the concerns Mr. Yager presents regarding the potential result of using a null pointer in Go, IMHO the more relevant fact is that Mr. Hoare identified in his own words a mistake he made _fifty years ago_ (1965) by introducing "null."
The fact that Mr. Gerrand stated in the submitted article:
“What should a modern programming language look like”
Begs the questions:
Why is this mistake oft addressed in _other_ languages repeated in Go?
What are the benefits of _intentionally_ introducing this boundary condition into the language?
Put bluntly, there is no benefit to dragging 0x0 forward in "a modern programming language." The lack of an answer to a function/method call is trivially encoded with some form of an option[1] type.
The question ultimately is what is the shortest path to developing a functioning and correct program ? Compilers can be very smart and help you catch a lot of errors but they can also introduce friction if they become too slow. Everyone has different opinions about the "best" because we have different ways of getting to the end-result. Programming is a discovery process and some people prefer to think a lot upfront, some like to experiment and probably a different mix of both for every one of us.
All this to say that nil in Go is not an absolute mistake. It's just a trade-off like everything else. Since I took Haskell as an example, it's also much much slower to compile.
In any case, Haskell supports exploratory programming just fine. It's true that Haskell's definition of "exploring" (refine your design until you can convice GHCi and QuickCheck that it makes sense) is somewhat idiosyncratic from a Lisper or Smalltalker's point of view (refine your design until you can convince yourself that it makes sense), but that doesn't make it any less legitimate. It's still experimenting, except now you have a tool to validate the degree to which the experiment has been a success. It's experimental science, not experimental art.
---
On the specific topic of nils, I think it's fair to say they're an absolute mistake in a statically typed language first released as late as in 2007. Standard ML and OCaml, both much older languages, have parametric polymorphism, algebraic data types, freedom from nils, and fast compilers: Poly/ML and OCaml's bytecode compiler, respectively. In addition, both Standad ML and OCaml have unrestricted imperative facilities, so freedom from nils is perfectly possible in an imperative language.
---
Regarding the classes of errors that are still possible in type systems like Haskell's, I have found only one that really matters, and where the usual workarounds (e.g., phantom types) don't help. Haskell, being a mathematically-inspired language, favors programming in terms of "timeless" entities whose existence is unaffected by the number of times you use them. While appropriate for manipulating pure data (numbers, strings, lists, trees, etc.), it's awkward for manipulating ephemeral computational resources, such as file descriptors or open sessions.
A language that fares better in this regard is Rust. Rust has parametric polymorphism, algebraic data types, freedom from nils (are we seeing a common theme?) and type classes, but gives up on interactive programming in exchange for precise control over data structure representation and resource management. Rust's type system design understands very well the ephemeral nature of objects: When a non-`Copy` object is passed around elsewhere, it's truly gone from the current lexical context.
---
Now, I don't want to give the impression that I think verification is the be all and end all of computer programming. There's plenty of room for languages that let you develop "fast and loose" solutions, as opposed to follow some principled philosophy. (Ruby comes to mind.) However, Go's very own marketing suggests that this isn't what they're trying to achieve. They do want a language that enforces a programming discipline. They just happen to have designed a language whose discipline doesn't provide enough benefits to justify the freedom restriction.
One thing I would like to observe is that all the languages that you are citing that don't implement nil also implement patter-matching. All have something equivalent to the Haskell Maybe type which needs to unwrap the value upon utilization.
I would be open to admit that nil is an absolute mistake if you can demonstrate that unwrapping is as convenient as just checking for nil and then using the value (or present another mechanism that is as convenient).
In regards to possible class of errors in Haskell I can think of a few: any non-terminating program, any program that blows up the memory. It's possible to implement a Monad that doesn't obey the monadic laws. It's possible to inverse the order of arguments inadvertently if they have the same type. Any insufficiently specified type, like using Int when only a range of Int is acceptable or any String when only a URL is acceptable. I'm sure there are more ;)
If you want to build a largefail at any point computation that may abort at any point in the middle, you can use the MaybeT monad transformer. Or EitherT, if you want to supply an error message when you fail. Inside a `do` block, it looks just the same as imperative code. So no syntactic convenience is lost.
But the real benefit, at least in my opinion, is the ability to design APIs that allow less room for error. In the vast majority of cases, pointers and references simply aren't intended to be nullable in the first place.
> It's possible to implement a Monad that doesn't obey the monadic laws.
And it's possible to overload operators in C++ in ways that make no sense whatsoever as well. (Using the same operator for bit shifting and stream I/O, seriously?) At least Haskell has a culture of associating laws to type classes.
> It's possible to inverse[sic] the order of arguments inadvertently if they have the same type.
No worse than any other language. If anything, Haskell's type system makes more distinctions, so this specific error is less likely.
> Any insufficiently specified type, like using Int when only a range of Int is acceptable or any String when only a URL is acceptable.
Idiomatic Haskell would make custom types, with smart constructors if necessary. Using String when a URL is expected is simply bad API design in Haskell.
---
But, anyway, I'd rather not continue this subthread, since the topic is Go.
"I like Go. I use it for a number of things (including this blog, at the time of writing). Go is useful. With that said, Go is not a good language. It's not bad; it's just not good."
Since Mr. Gerrand decided to take cheap shots at other languages, I felt it best to present the community with genuine criticisms of the Go language which were _not_ expressed in that style.
I don't think I was taking any kind of "cheap shot." It is literally the origin story of Go. Google has a few unreasonably large C++ projects and the incredible build infrastructure to make that possible. It was in part Rob's dissatisfaction with this state of affairs that triggered the first discussions of Go.
And this is just an example.
This part of Java actually wasn't so bad. I would argue that Java didn't get better with the addition of generics in 1.5.
What makes opaque containers a particular form of debug hell is when working in a non-trivial code base. Lobotomizing the one assistant which never tires (the compiler) puts the onus on people to "just know" what is in there (for whatever definition of "there" is at the moment).
Call me cynical, but people are horrible at keeping these kinds of concerns straight.
Well there are some rare cases where you really need generic data structures, is not hugely annoying to use go generate and a template implementation. That being said, I'd still like to see generics in Go, providing the drawbacks are limited.
The other criticism about null pointers is just ridiculous. If null pointers make in into a list of your top ten pain points while developing software in a team, I'm going to call you out for not being honest to yourself. Null pointers have their benefits and drawbacks, but it's so far down the list of practical concerns that it shouldn't even feature in your decision on what language to use.
Does identfying fundamental deficiencies in a language now qualify as "bike shedding?"
> Well there are some rare cases where you really need generic data structures, is not hugely annoying to use go generate and a template implementation.
I don't understand how you draw this conclusion, as homogeneous collections are one of the most used forms of collections I've seen for many years. For an example of the precise issue losing the type within a container causes, see this example[1] of where a Go developer was struggling with an issue that is non-existent when types are retained.
As for using an external tool to do "template generation", XDoclet[2] did that well over a decade ago. It was a PiTA then and now.
> If null pointers make in into a list of your top ten pain points while developing software in a team, I'm going to call you out for not being honest to yourself.
"Those who do not learn from history are doomed to repeat it" (source: http://thinkexist.com/quotation/those_who_do_not_learn_from_...)
They certainly do, as the number of people involved in working in a system increases, the assumptions regarding subsystem interaction vary. By having the ability to create a "software singularity" at any point by producing a null, everything will have to be _manually_ checked at the invocation site. Just like what had to be done in C for decades.
Can you guarantee that everyone involved in the lifetime of a project will always use the same approach, have the same assumptions about how the behavioural contracts are fulfilled, and will _always_ check everything that needs to be ensured "non-null" at the call sites?
With option types[3], or even better with disjoint union types[4] where the type contains either an error _or_ the desired result, this simply isn't a concern. The developer _must_ deal with the possibility of maybe not getting something back from a call (or of getting an error _instead of_ the desired result).
1- http://stackoverflow.com/questions/20087233/golang-get-the-v...
2 - http://xdoclet.sourceforge.net/xdoclet/index.html
Robert addressed generics in his talk (summary at https://sourcegraph.com/blog/live/gophercon2015/123645585015; I believe the video will be available in two weeks). Quoting the sourcegraph summary:
> Generics is the single biggest language feature absent in Go. It’s often missed by newcomers to Go. But it’s more of a type-system mechanism. It’s unclear if it’s an essential language feature.
> Generics are incredibly complex in both semantics and implementation. There are considerable trade-offs to consider, such as do you want a larger binary vs. slower binary vs. larger source code.
> It’s also very non-orthogonal. It interacts with and complicates a lot of other language features.
> They have tried and implemented prototypes of generics in Go, but they haven’t found a good solution, so they’ve decided to hold off and keep thinking about it for the time being.
Robert also made the point that generics are very complex, much more so than most people realize. See also https://news.ycombinator.com/item?id=9622417.