Go says Wat
about.sourcegraph.com
about.sourcegraph.com
A lot of WATs are more bad practice than problems in the language, IMO (like WAT 8, modifying the return value twice in deferred functions, with the order being significant: who the hell writes such a code?)
A few ones, though, really are problematic. For instance I've been bitten more than once by variable shadowing, especially with error values. IMO, this is the weakest point of Go.
slices are not complicated, the issue is append, mutable slices and backing arrays being shared, that's what's really fucky about it.
And that interaction is absolutely a WAT. Especially the ability to separately append to two slices with the same backing array.
> A lot of WATs are more bad practice than problems in the language, IMO
Much the same could be said about the original presentation.
There absolutely are non-WAT sections in that article though. (4) is one.
I'm not sure views into a fixed-sized array is really confusing/strange at all. Whether you create multiple views in the array, mutate the views, etc. All interactions that occur on slices and arrays make plenty of sense. How you'd write something like append yourself if "Slice" was your type w/ a backing array, a start index, and a length is quite straightforward.
Whether there are multiple views or not is actually an implementation detail, and whether "append" chooses to allocate memory or not will change whether there are multiple copies or multiple views of the array.
All of that behavior is something you absolutely cannot rely on.
In a language that encourages concurrency, this indeterminism should be jaw-dropping. How many Go programs are safe only accidentally via an array's allocation policy (especially given how much code it requires to duplicate a slice)? Can you think of any other language that has such a design?
> Can you think of any other language that has such a design?
Sure, happens all the time because it fits the machine naturally though sometimes it may be hidden with unnecessary runtime costs by many languages that do force copies or reallocations due to immutability. C++ is a language that doesn't hide it. Your argument is like saying you're surprised that std::vector::emplace_back may or may not alter the array I had kept from std::vector::data previously.
Again, as I already stated in the previous comment the issue is not the slices themselves.
It's that they're doubling up as vectors and they're shitty at it: you can share a backing buffer between mutable slices and append to both and then all hell breaks loose.
That is the issue. And that is why Rust doesn't have that issue despite having the same concept of slices: you can't grow a slice, and you can't have multiple mutable slices to the same backing buffer.
No. C only has raw pointers so it's not trying to pretend slices are vectors, and C++ has actual functioning vectors with "slicing" being either a copy of the subvector or an iterator. Either way there is no confusion as to the capabilities of what you're given and its relation to the original data.
Isn't that the tired old argument from the C camp? "If you are careful you can write perfectly fine C without bugs".
DUH!
The whole idea is for the compiler and types to catch as much of this crap as possible, not to add accidental mental burden to the programmer.
Is it though? Slices seem to be passed by value, where the value is a pointer and some bookkeeping information (like a C struct of (data, len, capacity)).
Some operations manipulate just data (which is visible in the caller, since that 1/3 of the "struct" was a pointer passed by value) and some operations manipulate len and/or capacity (which is not visible in the caller, since that 2/3 of the "struct" was also passed by value).
Seems more complex than just hand-waving the details away with "pass-by-reference 101".
So, probably more like pass-by-reference-and-by-value 201.
Even if you pass a pointer, the pointer is explicitly copied to a new location in memory, which is why assignment to the pointer can overwrite the pointer, but not affect the original pointer (the memory location value inside the pointer-typed variable) outside the function scope.
Pass by reference is when assignment to a parameter name is transitive to the caller's scope, and can be seen in C#. [1]
[1] https://docs.microsoft.com/en-us/dotnet/csharp/language-refe...
Now in WAT 1,
func grow(s []int) { // s is deep copied.
s = append(s, 4, 5, 6) // changing 's' does not effect original slice.
}
Explicitly pass mutable slice, func grow(s *[]int) { // s is referenced.
*s = append(*s, 4, 5, 6) // changing 's' will always effect origin slice.
}
Explicit is better than implicit. type thing struct {
foo int
}
func f(x thing) {
x.foo = 42
}
func main() {
var x thing
x.foo = 1234
fmt.Printf("x before: %v\n", x)
f(x)
fmt.Printf("x after: %v\n", x)
}
This prints 1234 and 1234 as you'd expect. That's pass by value in action. But what if we mutate a slice? func f(x []int) {
x[1] = 42
}
func main() {
x := []int{1, 2, 3}
fmt.Printf("x before: %v\n", x)
f(x)
fmt.Printf("x after: %v\n", x)
}
Of course, this prints [1 2 3] and [1 42 3]. While you're passing the slice by value, the backing array is not copied.As you write more go, you will begin to fully see the existence of a backing array. Try this program:
func main() {
x := []int{1,2,3,4}
y := x[1:2]
y[0] = 42
fmt.Printf("x: %v, y: %v\n", x, y)
}
You know that [:] does no copying (there is a copy() function that copies stuff) and just shares the same backing array between each slice, so you aren't surprised when this prints "x: [1 42 3 4], y: [42]".Now, add some knowledge from the documentation about how append() works. You use the idiom slice = append(slice, element) because the documentation says, sometimes the slice is updated in place when there is enough capacity for that to occur, and sometimes the slice is copied to a new slice and returned.
This then leads to your "WAT" in WAT 2. Because you _know_ that the data you appended is actually in the backing array, but for some reason Go isn't showing it to you.
You can see this in action with a very poor example I just wrote:
func main() {
x := []int{1, 2, 3, 4, 5, 6}
y := x[0:0]
fmt.Printf("x: %v, y: %v\n", x, y)
fmt.Printf("intermediate result that's not saved: %v\n", append(y, 9, 8, 7, 6, 5))
fmt.Printf("x: %v, y: %v\n", x, y)
}
After all of this, x is [9 8 7 6 5 6]. So you _know_ that append is more than happy to mess with your backing array. You just need a slice with the right length to be able to see all the data in there.For that reason, I think WAT 2 is worthwhile. Most people know _just enough_ about slices to be dangerous, and so are surprised when there is an additional complication that they haven't thought about.
Until it doesn't.
type thing struct {
foo *int
}
func f(x thing) {
*x.foo = 42
}
func main() {
v := 1234
var x thing
x.foo = &v
fmt.Printf("x.foo before: %v\n", *x.foo)
f(x)
fmt.Printf("x.foo after: %v\n", *x.foo)
}
Or maybe that is what you'd expect, but then slices shouldn't seem so mysterious.I that surprising behavior? It is not only well-documented, but is in line with most other languages that offer a hash table / map / dictionary / whatever in the base language or standard library.
[0]https://blog.golang.org/go-maps-in-action - Header: Iteration Order
When I first learned about hash tables (in Perl), the book I used said that the iteration order over a hash should never ever be relied upon, and that it could change arbitrarily from one release to the next. If the order mattered, one should use a different approach (get the keys and sort them by whatever criteria), plain and simple. I guess that has sunk in pretty deeply with me. ;-)
The .Net framework provides an OrderedDictionary and a SortedDictionary, both of which make some kind of promise regarding iteration order, but I have never used them myself.
But it has ordered output, and doesn't invalidate iterators on insertion (hashmaps might, because they sometimes need to rehash).
Generally, the suggestion that I've heard is to avoid using std::map unless you really really what it specifically provides, because it's expensive and it's hard to know if you can safely relax those constraints.
Ordered maps don't require trees, and don't have that high a tradeoff: https://morepypy.blogspot.com/2015/01/faster-more-memory-eff... https://github.com/rust-lang/rust/pull/45282#issuecomment-33...
1. Integer indexes (string keys) in ascending numeric order
2. Other string keys (non-integer indexes) in insertion order
3. Symbols in insertion order
https://www.ecma-international.org/ecma-262/#sec-ordinary-ob...
Take this example from Scheme: R5RS specified two procedures, integer->char and char->integer, to bijectively map characters to integer character codes, but it said nothing about a specific encoding. Most Scheme implementations happily mapped characters to their ASCII codes or Unicode codepoints.
Not so Scheme48, which mapped each character to its ASCII code number plus a magic constant (believe it was 1000). Scheme48 provided library functions ascii->char and char->ascii, that did what most people actually wanted and its maintainers insisted that people use those.
Had significant code appeared in the wild that, say, simply subtracted the magic constant from the value returned by char->integer and re-added it again before calling integer->char, the maintainers probably would have either changed the magic constant, or changed the mapping entirely, in a future version.
The stable order that exists turns into another footgun when people start to hardcode that into tests, and it breaks out from under them when the compiler changes slightly.
Reading your [0], I couldn’t say whether go changes iteration order between iterations in the same run. I would guess it doesn’t.
I dislike Go for a billion reasons but I appreciate when they "break" APIs that explicitly never made those guarantees.
I'm actually impressed by the go devs here, it's a great way to avoid people making assumptions if your order is legitimately randomized.
If the name was hashmap then it wouldn't be surprising, but for "map" and especially "dictionary" I can see how people might expect them to be ordered without further information.
Here's a basic demonstration:
https://play.golang.org/p/8zMmPwtjcxG
In particular, note how this behavior can easily be forgotten when the semantics are hidden through variadic arguments that are passed an existing slice (instead of the automatically created one when you pass in actual variadic arguments).
At a fundamental level, the issue is that Go's design decisions make any modification of a shared slice dangerous, and the language provides no way to mitigate it.
A really fun one is divergent appends to slices with a shared backing array and enough capacity for an in-place append: https://play.golang.org/p/ZHWo3bFOR0X
func main() {
x := []int{0, 1, 2, 3, 4, 5, 6, 7, 8, 9}
fmt.Println("orig: ", x)
mutate(x)
fmt.Println("append: ", x)
mutate(x[:4])
fmt.Println("sliced: ", x)
}
Appending to the end of the slice creates a new slice and therefore the caller doesn't see the mutation.However appending to a slice-of-the-slice leaves capacity in mutate's copy-of-the-slice to append without allocating a new backing array. Therefore mutate happy bumps the len of its slice copy and mutates the callers slice!
This bit me once in real production code when reusing a []byte array to avoid allocations. The bug was obviously my fault, but this behavior can be easy to inadvertently trigger if you're trying to avoid allocations!
Edit: fixed code formatting. It's 2018 YC, please please please implement at least a subset of markdown.
Along similar lines: WAT 15. Under what circumstances do you expect a nil var return to become non-nil? Did you even know that was possible?
Yeah. Typed nils are a horrible, horrible feature: when you cast a value to an interface, it creates a fat pointer of (Type, Value). From a nil, that's (nil, nil) but from a nil Foo that's (Foo, nil). And since `==` just does a straight value comparison, (nil, nil) != (Foo, nil).
This actually has an official FAQ entry telling you to go fuck yourself: https://golang.org/doc/faq#nil_error
That said, the fact that the crowd only offered an incorrect guess four times out of 16 is telling. This really didn't have the same feel to me as the original WAT talk, which is really full of truly strange things.
Dump those rules for a much simpler insight:
When comparing interfaces, type and value must match to compare equal.
So, (nil, nil) compares NE to (*myFancyErrorType, nil) for the same reason that (float64, 0) and (uint16, 0) do.
It isn't all that complicated. I don't see a real and non-insane alternative to it. Do you?
What? Go is all about magic. All the std lib stuff that is able to take any type works by magic.
Those "obscure rules" (which I prefer to call being able to reason about code) look like they are going to get added in Go 2.
It's odd how programmers think polymorphism is this obscure thing when it's available in almost every typed language.
One general comment; not every question was intended as a WAT. Some were setups to introduce a WAT.
a := math.MaxInt64
fmt.Println(a)Looking up a key in a map you didn't "make" gives you the zero value. I think that's consistent with much of Go. But if appending to a nil slice works, assigning to a key should never panic, even if you didn't "make" it first.
old := x
new := foo(&x)
if old != new {
return &Bar{} // oops, return new(Bar) won't compile
}
That should at least be caught by linters.OTOH, it's very unlikely you type accidentally `true := x` in your code, and even if you did, that would very probably be caught by the compiler anyway.
Anybody who is still human and can do a typo.
You don't need to have a variable like "tue" to make such a typo. Since your writing an if statement, you already have "true" and "false" in mind, so you just need to be a little absentminded and voila, instead of foo := false you've written true := false. Plus, lots of naive editors will offer to autocomplete something starting with t to true (if they find the token true used elsewhere within the file).
>the compiler will tell you "true declared and not used".
Only if you don't use it. But a few lines below you could very well be using true legitimately too, in which case it wont tell you:
true := false
...
if condition == true {
// ...
}Good point!
Edit: but then, your expected original variable name, `tue` in my example, would be used while not yet defined:
true := false
if foo() || bar {
tue = true // Compilation error
}
...
if condition() == true {
...
}
Only possible case I can think of: you already defined tue, but tried to redefine it via variable shadowing: tue := true
...
if cond() {
true := false
if foo() || bar() {
tue = true // Will compile, but not the tue you expected
}
...
if condition == true {
...
}
}As you point out, accidentally redefining the meaning of len or new or close is far more likely.
I think it's crazy that this isn't a compile-time error, but it's not even close to "you can change the global value of builtin values".
On the other hand, why would I ever want to do that? It's not something one might do by accident. It's along the line of C letting you say
idx[arr]
instead of arr[idx]
It's unfortunate that it is possible, but there is no good reason to ever do this unless one is trying to confuse one's enemies.> The rationale is pretty simple: Only identifiers that must be keywords for syntactic reasons are keywords.
> In fact, in the very beginning there was some discussion as to whether things like nil, iota, etc. should be keywords. Eventually we agreed on the rule above which settled it.
https://github.com/golang/go/issues/18193#issuecomment-26492...
In Smalltalk, nil just contains the sole instance of the UndefinedObject class. It's not a hole in the type system. Instead, it becomes a paradox in the inheritance system, because it's used as a superclass.
Here's a WAT. Smalltalk is actually strongly typed. It's just that everything has the same type of Object. (The type system has no holes. However, it's the size of a thimble!)
The first two, for example, aren't WATs at all. The Go book is very clear that you need to use the result of `append` to get the modified slice. WAT 4, map traversal is unordered, is not at all a WAT. WAT 7, maps are reference types, is also not surprising at all. WAT 8 is also not a WAT, defers are defined to be processed in LIFO order, and the rest falls out of how named returns work. This was a waste of time :(
What exactly surprises you?
I just read the Go book and none of what I saw was surprising in the least. I consider a WAT something like Python mutable default arguments that are certain to surprise every Python programmer at some point.
https://golang.org/pkg/io/#Writer
>Implementations must not retain p.
Once you realize slices are just (pointer, length, capacity) structures, and the structure itself is copied by value, the first 2 WAT is pretty trivial.
WAT 3 becomes less surprising when you consider that a method on a pointer can be called and can return a valid result even if the pointer is nil.
WAT 10 just seems inconsistent. I said this before, and I'll say it again: shadowing does more harm than good.
The rest are pretty trivial for anyone who worked with the language for over a year or read the documentation carefully.
Still, nice list of small gotchas.