Fixing for loops in Go 1.22
go.dev
go.dev
Mar 21, 1992, 1:00:47 AM
Last-Modified: Tue Feb 25 17:34:30 1992 by Mark Kantrowitz
;;; ****************************************************************
;;; Answers to Frequently Asked Questions about Lisp ***************
;;; ****************************************************************
;;; Written by Mark Kantrowitz and Barry Margolin
;;; lisp-faq-3.text -- 16886 bytes
[...]
----------------------------------------------------------------
[3-9] Closures don't seem to work properly when referring to the
iteration variable in DOLIST, DOTIMES and DO.
DOTIMES, DOLIST, and DO all use assignment instead of binding to
update the value of the iteration variables. So something like
(dotimes (n 10)
(push #'(lambda () (incf n))
*counters*))
will produce 10 closures over the same value of the variable N.
----------------------------------------------------------------That's actually expected when capturing by reference.
Eric Lippert has a wonderful blog on the "why" from their perspective: https://ericlippert.com/2009/11/12/closing-over-the-loop-var...
I had a bit of trouble finding the original C# 5 announcement; that's hopefully not been lost in the (several?) blog migrations on the Microsoft domain since 2012.
I think it played a large part in helping get past the default-deny that any language change proposal should have. The other big one for me was the scan done over the open source code base and the balance of bugs fixed versus created.
Given how much of an uproar there was over changing the string type in the Python 2 -> 3 transition, I can't imagine this change would ever end up in Python before a 4.0.
Cue someone arguing about how bad Python is because it won't fix these things, and then arguing about how bad Python is because their scripts from 2003 stopped working...
I think this is a good case of Python not fixing things, given that a fix exists that solves both problems.
Eg # py 3.4
Which makes me think that a Go 2.0 or Python 4 will mainly be about removing branches and edge cases from the compiler more than making backwards-incompatible language changes.
Arguably C & C++ compilers do the same thing via -std=c99 and similar flags at compile time.
Anyway, nothing about this is special or different with python. I bet the transition to python 3 would have been much smoother if scripts could have opted in (or opted out) of the new syntax without breaking compatibility.
A runtime interpreter does not prevent Perl to do similar things via `use 5.13`
Python has `from future` with similar abilities, it would absolutely be possible to do the same as Perl and Go and fix what needs to be fixed without breaking old code. One could design a `import 3.22` and `from 3.22 import unbroken_for` and achieve the same thing.
The Python devs sometimes seem stubbornly attached to bugs. Another one: to reliably get Python 3 on Linux and Mac you have to run `python3`. But on Windows there's no `python3.exe`.
Will they add one? Hell no. It might confuse people or something.
Except... if you install Python from the Microsoft Store it does have `python3.exe`.
I’m not claiming any mystery about Python, just disputing how the modern version is invoked.
That was the old way. Python now recommends against installing Python3 in a way that does that, and most modern *nix don't.
On an Ubuntu 20.04 desktop VM:
python => Python 2.7.18 (default, Jul 1 2022, 12:27:04)
python3 => Python 3.8.10 (default, May 26 2023, 14:05:08)
python => -bash: python: command not found
python3 => Python 3.7.5 (default, Apr 19 2020, 20:18:17)
python => -bash: python: command not found
python3 => Python 3.8.10 (default, Jun 2 2021, 10:49:15)
Red Hat Enterprise Linux 7:
python => Some version of Python 2
python3 => Some version of Python 3
python => -bash: python: command not found # (use "python2" for Python 2)
python3 => Some version of Python 3
MacOS 10.15 (Catalina) does something interesting:
python => WARNING: Python 2.7 is not recommended.
This version is included in macOS for compatibility with legacy software.
Future versions of macOS will not include Python 2.7.
Instead, it is recommended that you transition to using 'python3' from within Terminal.
Python 2.7.16 (default, Jun 5 2020, 22:59:21)
python3 => Python 3.8.2 (default, Jul 14 2020, 05:39:05)This is not true on my Fedora 38 system, same with current Kali linux. Although, it is the case with Ubuntu 22.04.3.
(This is isomorphic to the usual victim-blaming discussion. Fault and blame vs some ability to make a difference; it's a shame that correctly pointing out a better strategy is both used to attack victims and attacked for attacking victims in the cases when that wasn't intended.)
It's worse. If you don't install Python from the Microsoft Store there will still be a `python3.exe`. But running it just opens Microsoft Store.
Imagine how confused one could be when someone typed `python3 a.py` over a SSH session and nothing happened.
Python doesn’t make breaking changes in non-major versions, so as mentioned by the upthread comment the appropriate place for this change would be in Python 4.
Given the above, I’m really not sure what point you think you’re making in that final paragraph.
What sort of problems are have you faced upgrading minor versions?
See https://docs.python.org/3.0/library/fractions.html#fractions... , https://docs.python.org/3.5/library/fractions.html#fractions... , https://docs.python.org/3.9/library/fractions.html (gone), https://docs.python.org/3.5/library/math.html#math.gcd
It means code doesn't care about the issue being addressed.
The feature is only justified if it changes existing code, such that bugs you didn't even know about are fixed.
I.e. people read about the issue, investigate their code bases and go, oh hey, we actually have a latent bug here which the change will fix.
Old code that is maintained will eventually be upgraded, which yes does come with work sometimes where you realize your code works on version X but not version X+10 and you do a combination of tests and reading patch notes to see what changed.
Code doesn't care about when it's written, only what you run it on, and with what compatibility options.
E.g. one possibility is that ten-year-old code that wrongly assumed the opposite behavior, and has a bug, will start to work correctly on the altered implementation.
Good language designers want to avoid both wasting developer's time and requiring ugly workarounds. Making a change that does both, especially if it doesn't break old code, is great imo.
If a fresh variable i is bound for each iteration, then an i++ statement in the body will not have the intended effect of generating an extra increment that skips an iteration.
If you want the other semantics, whichever one that is, the workaround is ugly.
add_n = []
for n in range(10):
add_n.append(lambda x: x + n)
add_n[9](10) # 19
add_n[0](10) # 19
for i := 0; i < 10; i++ {
go func() { fmt.Printf("num: %d\n", i) }()
}Comprehension in both Python and Haskell (for both lists and other structures) use the same order in both language, as far as I remember.
Right, and do-notation is the one everyone uses, because it's better. Python picked the wrong one.
> Comprehension in both Python and Haskell (for both lists and other structures) use the same order in both language, as far as I remember.
It may be the same order as Haskell but it's a terrible confusing order. In particular if you want to go from a nested list comprehension to a flat one (or vice versa) then you have to completely rearrange the order it's written in, whereas if you go from nested do-blocks to flat do-blocks then it all makes sense.
However, I can imagine a feature that we could add to Python to fix this: make it possible for statements to have a value. Perhaps something like this:
my_generator = \
for i in "abc":
for b in range(3):
print("foo")
yield (i, b)
my_list = list(
for i in "abc":
for b in range(3):
(i, b))
my_list = list(
for i in "abc":
yield from for b in range(3):
(i, b))
for i in range(10): print(i)
for i in range(10): print(i)
The limited whitespace-based syntax limits the potential for fun inline statement things, but it also completely dodges the question of what any particular statement should evaluate to when used as an expression.
Yes, I guess something like that. That was just meant as an example of how existing Python allows you to write loops on one line. It's not a good example for a meaningful comprehension in our alternative made-up Python dialect.
> The limited whitespace-based syntax limits the potential for fun inline statement things, [...]
Python already mostly allows you to use parens to override the indentation. They would just need to generalise that a bit. Btw, Haskell already does that:
Officially, Haskell has a syntax with curly braces and semicolons; and they define the indentation based syntax as syntactic sugar that desugars to ; and {}. But almost everyone uses indentation based syntax. The exception are perhaps code generators and when posting on a website that messes with indentation.
(And, because it's Haskell, the {}; syntax is just another layer of syntactic sugar for 'weird-operator'-based based syntax like >>=.)
In Python there is a nice tower of abstractions for iteration, but nothing more general than that, so it makes perfect sense IMO to use the syntax that directly evokes iteration.
The existing syntax is meant to mirror the syntax of a nested for loop. I agree that maybe it's confusing, but if you want to go from a multi-for comprehension to an actual nested for loop, then you don't have to invert the order.
It could work on the same things that Python's current list comprehensions work on. I'm just suggesting a different syntax. Comprehensions in Haskell originally worked for all monads too.
> And who is the "everyone" using do-notation? I don't see any analogous syntax in Lua, Javascript, Ruby, or Perl.
I meant that within Haskell, everyone uses do notation rather than comprehensions.
> The existing syntax is meant to mirror the syntax of a nested for loop. I agree that maybe it's confusing, but if you want to go from a multi-for comprehension to an actual nested for loop, then you don't have to invert the order.
You have to invert half of it, which I find more confusing than having to completely invert it. do-notation style syntax (e.g. Scala-style for/yield) would keep the order completely aligned.
And that behaviour is still accessible via a compiler extension.
https://idris2.readthedocs.io/en/latest/tutorial/interfaces....
https://idris2.readthedocs.io/en/latest/tutorial/interfaces....
add_n = [lambda x: x + n for n in range(10)]
add_n[9](10) # 19
add_n[0](10) # 19 add_n = [lambda x, n=n: x + n for n in range(10)]
add_n[9](10) # 19
add_n[0](10) # 10Also pylsp warns you about this
So it will warn in certain situations, but not all of them
It might complain about the race condition, to be fair, but the same issue can be reproduced without goroutines and it would be completely correct code per the semantics.
It's also one of the great foot-guns of C programming as there are so many other almost but not that idioms and it's never clear on casual inspection whether the result of an assignment was meant to be examined or the result of a comparison.
With the evolution of C and C sanity tools that rightfully flag such statements for double checking and the desire to not have spurious flagging, etc. it's more common in later C code to see (say)
if ((err = someFunction()) != NOERROR) { errorHandle(err) }
As with all idoms the general practice in the larger codebase and house code standard rules apply - there are other ways of doing similar things.
from functools import partial
from operators import add
add_n = [partial(add, n)) for n in range(10)]
assert add_n[5](4) == 9
https://github.com/python/cpython/blob/main/Lib/functools.py...
Here's a simplified version of that code that demonstrates the pattern.
class partial:
def __init__(self, func, *args, **kwargs):
self.func = func
self.args = args
self.kwargs = kwargs
def __call__(self, *args, **kwargs):
return self.func(*self.args, *args, **(self.kwargs | kwargs))
p = partial(add, 5) # -> an instance of partial with self.args = (5,)
res = p(4) # -> calls __call__ which merges the args and calls add(5, 4)But you are also right that the mechanisms in Python are different (on some suitable mid-level of abstraction) for those two.
This makes it clear: the underlying problem is NOT about for loops - it's closures that are broken.
(((i, j) for i in "abc") for j in range(3))
(Do take what I wrote with a grain of salt. Either the above is already a problem, or perhaps you also need to mix in list-comprehension expressions, too, to surface the bug.)
gs1 = (((i, j) for i in "abc") for j in range(3))
gs2 = [((i, j) for i in "abc") for j in range(3)]
print(list(map(list, gs1)))
print(list(map(list, gs2)))
[[('a', 0), ('b', 0), ('c', 0)], [('a', 1), ('b', 1), ('c', 1)], [('a', 2), ('b', 2), ('c', 2)]]
[[('a', 2), ('b', 2), ('c', 2)], [('a', 2), ('b', 2), ('c', 2)], [('a', 2), ('b', 2), ('c', 2)]]
Equivalents:
def make_gs1():
for j in range(2):
yield ((i, j) for i in "abc")
def make_gs2():
gs = []
for j in range(2):
gs.append(((i, j) for i in "abc"))
return gs
I think early binding would produce the same result in both cases.
>>> [[(i, j) for i in "abc"] for j in range(3)]
[[('a', 0), ('b', 0), ('c', 0)], [('a', 1), ('b', 1), ('c', 1)], [('a', 2), ('b', 2), ('c', 2)]]The bigger problem in Go is the for with range loop:
pointersToV := make([]*val, len(values))
for i, v := range values {
go func() { fmt.Printf("num: %v\n", v) } () //race condition
pointersToV[i] = &v //will contain len(values) copies of a pointer to the last item in values
}
Edit: it looks like they're actually changing both of these, which is more unexpected to me. I think the C# behavior makes more sense, where only the foreach loop has a new binding of the variable in each iteration, but the normal for loop has a single lifetime.
I wouldn't say either behavior is non-lexical. The only thing changing is which lexical scope these variables go into.
I used plural for a reason.
> there is no block in the program text that corresponds to that scope.
The scope starts at the for. There is a bunch of state that is tied to the loop, and if you rewrote it as a less magic kind of loop you'd need to explicitly mark a scope here.
What's non-lexical about it? You could replace "for" with "{ for" to see that a scope of "all passes through the loop body" does not require anything dynamic.
And surely whether a scope is implicit or explicit doesn't change whether a scope is lexical. In C I can write "if (1) int x=2;" and that x is scoped to an implicit block that ends at the semicolon.
Would you say an if with a declaration in it is non-lexical, because both the true block and the else block can access the variable? I would just say the if has a scope, and there are two scopes inside it, all lexical. And the same of a for loop having an outer and inner scope.
https://docs.python.org/3/reference/executionmodel.html#stru...
The problem is in the implementation of for-range loops, where the clear expectation is that the loop variable is scoped to each loop iteration, not to the whole loop scope (otherwise said, that the loop variable is re-bound to a new value in each loop iteration). The mental mode approximately everyone has for a loop like this:
for _, v := range values {
//do stuff with v
}
for i := range values {
v := values[i]
//do stuff with v
}
In Go 1.21 or earlier, the semantics are closer to this (ignoring the empty list case for brevity):
for i := 0, v := values[0]; i < len(values); i++, v=values[i] {
//do stuff with v
}
(In functional languages, this problem did not arise, since most variables are immutable and those that are not are syntactically marked and decidedly second-class. In particular, you would probably not implement a loop using a mutable counter or iterator.)
For example, in the following C++:
auto v = std::vector<int>{1, 2, 3};
auto prints = std::vector<std::function<void()>>();
auto incrs = std::vector<std::function<void()>>();
for (auto x : v) {
prints.push_back([&x]()->void {std::cout<<x<<", "; })
incrs.push_back([&x]()->void {++x;});
}
for (auto f : incrs) {
f();
}
for (auto f : prints) {
f();
} //expected to print 2, 3, 4; actually prints 6, 6, 6
Tried it out, OCaml does the expected thing:
open List
let funs = ref [ ] ;;
for i = 1 to 3 do
funs := (fun () -> print_int i) :: !funs
done ;;
List.iter (fun f -> f()) !funs ;; //prints 321I understand - but in those languages capture-by-reference has to be an explicit choice (by writing the &) rather than the default, which highlights the actual behaviour. The problem with the old Go solution was that it would apparently behave as capture by reference without any explicit syntactic marker that it is so, and without a more natural alternative that captures by value, in a context where from other languages you would expect that the capture would happen by value.
> Common Lisp and Scheme
I have to admit I haven't worked in either outside of a tutorial setting, but my understanding is that they are quite well-known for having design choices in variable scoping that are unusual and frowned upon in modern language design
> Ocaml
Your example shows that it captures by value as I said, right? For it to work as the old Go examples, i would have to be a ref cell whose contents are updated between iterations, which is not how the semantics of for work. If it did, you'd have to use the loop counter as !i.
And what I was trying to show with my example was that this kind of behavior would be observable in OCaml as well, if it were to be implemented like that.
So does not really have a good way to fix the issue, even by using a different keyword as JS did.
OTOH default parameters being evaluated at function definition make mitigating it relatively simple.
Parts of my previous job were terrible because it had JS functions thousands of lines of code long where variables were constantly reused (and often had to be unset before another block of code). That said, that wasn't the fault of function scope per se, but of a bad but very productive developer.
0: https://docs.python.org/3/reference/compound_stmts.html#the-...
It is so unintuitive...
It sounds like a lack of dogfooding, lack of review?
I disagree with this approach, despite the fact that you can logically explain this,
then it still is terrible design and as you see golang (and c# and other langs) designers realized it too and changed it
So, how can you even try to defend this design when even the designers decided to change it (despite the fact that changing lang is really hard)?
That is not actually relevant to the point.
> So, how can you even try to defend this design
I'm doing no such thing, I'm pointing out that your reasoning is faulty: dogfooding does not help when the behaviour is logical and obvious to the designer-cum-user, and same with reviewing.
While I can agree with dogfodding, then review doesn't have to be done just by the people that are responsible for the design/impl.
You can have external reviewers for (let's call it) sanity check
I think the main things that make it such a trap is that the variable type definition is implicit so the fact that it's a pointer becomes a bit hidden, and that easy concurrency means the value is evaluated outside of the loop execution more often.
No. Full disagree.
Array represents a concept of holding multiple values (let's simplify) of the same type.
Loop (not index based) over array represents concept of going *over* array's elements and executing some code body for each of it.
Now, if the behaviour isn't that loop's body is executed for each array element (let's forget about returns, breaks, etc)
then the design is terrible (or implementation, but that'd mean that it was a bug)
I have totally no idea how can you design this thing in such a unintuitive way unless by mistake/accidentally.
Basically, here are two pseudocode implementations. This is what currently happens:
i = malloc(sizeof(int))
*i = 0
loop:
<code>
*i = *i + 1
goto loop if *i < 10
secret = malloc(sizeof(int))
*secret = 0
loop:
i = malloc(sizeof(int))
*i = *secret
<code>
*secret = *secret + 1
goto loop if *secret < 10
Remember, simple cases work:
for i := 0; i < 10; i++ {
fmt.Println(i) // 0 1 2 3 4 ...
}
var is []*int
for i := 0; i < 10; i++ {
is = append(is, &i)
}
for _, i := range is {
fmt.Println(*i) // 9 9 9 9 9 ...
}
Loop in general or "for each" style loop, that's huge difference.
The 2nd one has a lot to do with collections.
>You can see that they are not crazy for picking the first implementation; it's less instructions and less code
Yes, it is not crazy when you're looking at it from the reverse engineering / implementation side
but if you start thinking about it from user's perspective then it is very bad behaviour
because they used "foreach" like loop which is a concept of walking thru every element of collection.
Normal "for" is like: repeat this code body as long as condition is satisfied
Foreach is more like: walk thru this collection
Look (c#):
foreach (var item in items) ...
for (int i=0; i<10; i++) { }
In the 2nd version it is possible to jumps ahead, back, do not move, etc. Generally play around "i's" values
Meanwhile I haven't seen yet any1 trying to do anything like this in foreach, because it is meant for just walking thru collection
The solution as I said elsewhere is to pop out the inner block to a separate function, where the value of the counter is captured when the outer function is called, not when the inner one runs.
for (int i = 0; i < n; i++) {
callbacks.add(new Callback(){
public void Call() {
System.out.println(i); //compiler error: local variables referenced from an inner class must be final or effectively final
}
});
}
for (var f : callbacks) {
f.Call();
}
for (int i : new int[]{0, 1, 2}) {
callbacks.add(new Callback(){
public void Call() {
System.out.println(i);
}
});
}
for (var f : callbacks) {
f.Call();
} //prints 0 1 2The fix is to make a new variable for each iteration, which is less obvious implementation wise but as per the post works better if you're enclosing over the loop variable.
for i, v := range []int{1, 2, 3} {
funcs = append(funcs, func() {fmt.Printf("%v:%v, ", i, v)})
}
for _, fun := range funcs {
fun()
} //prints 2:3, 2:3, 2:3
False. There are cases where it is useful to have the loop variable available directly. For example, you can add one to the loop variable to skip an iteration, which would not work with an iteration-local loop variable.
I do agree that there are reasons to modify the iteration variable in a C-style for loop, so I am surprised that those loops are being modified as well. C#, which went through a similar change, did NOT apply such a change for those for loops.
That might be the case, but my comp sci 101 was 15 odd years ago now and since then I have _never_ had to think about pointers vs values, until I started a Go project a few years ago. But even that was more comprehensible than the pointer wizardry we had to do in C/C++ back when.
I don't want to have to think about managing my application's memory, I much prefer being in the code, thinking of variable scope and maintainability which in a lot of languages automatically translates to healthy memory usage.
If you try to do something weird with variable capture, then any collections you accumulate data into (eg, for turning an array into a map), will behave differently than any declared variables.
Go is trying to thread the needle by only having loop counters work this way. But that still means that some variables act weird (it's just a variable that tends to act weird anyway). And I wonder what happens when you define multiple loop variables, which people often do when there will be custom scanning of the inputs.
You pass your counter into the function, it returns a function that remembers the original value, not the value as it keeps iterating later on in the caller.
That's an interesting definition. I thought you would either go with https://en.wikipedia.org/wiki/Functor_(functional_programmin... or with https://en.wikipedia.org/wiki/Function_object
https://en.wikipedia.org/wiki/Functor_(disambiguation) has a few more choices, but doesn't seem to have yours.
There is one somewhat similar problem in Java that you're maybe thinking of: anonymous classes that reference fields of the current object. I don't think that behavior is surprising, and there are very important use cases for it.
What Go is doing is perfectly sensible. The ability to capture variables is extremely powerful, and often desired. It's just the unexpected scoping of loop variables that introduces a problem. The following code is doing exactly what most people would expect, for example:
a := 0
incr := func() {a += 1}
print := func() {fmt.Printf("%d", a)}
print() // prints 0
incr()
print() //prints 1So no, Java didn’t have this problem.
Interestingly, the reason the old i := i trick works is not at all what I thought!
The trick, for reference:
for i := 0; i < 5; i++ {
i := i // the trick
go func() {
print(i)
}()
}
- The escape analyzer sees that the new `i` is passed to a goroutine, so it is marked as escaping its lexical scope
- Because it's escaping its lexical scope, the allocator allocates it on the heap
- One heap allocation is done per loop iteration, and even though the new `i` is captured by reference, each goro holds a unique reference to a unique memory location
What actually happens:
- Go's compiler has heuristics to decide whether to capture by reference or by value. One component of the heuristic is that values that aren't updated after initialization are captured by value
- The new i is scoped to the for loop body, and is not updated by the for loop itself. Therefore it's identified as a value that isn't updated after initialization
- As a result, the Go compiler generates code that captures `i` by value instead of by reference. No heap allocations or anything like that are done.
I recognize that the latter behavior is better, but if anyone with intimate knowledge of Go knows why the former doesn't (also) happen (is that even how Go works?) I would love to find out!
Closure captures are always by reference.
Heap vs stack allocation don’t affect the language semantics.
(that same adage applies to e.g. browser manufacturers having to implement bugs to not break certain websites)
It is intended that programs written to the Go 1 specification will continue to compile and run correctly, unchanged, over the lifetime of that specification. At some indefinite point, a Go 2 specification may arise, but until that time, Go programs that work today should continue to work even as future "point" releases of Go 1 arise (Go 1.1, Go 1.2, etc.).Yeah, I thought this kind of change wouldn't happen because of this promise.
In particular they said:
> The end of the document warns, “[It] is impossible to guarantee that no future change will break any program.” Then it lays out a number of reasons why programs might still break.
> For example, it makes sense that if your program depends on a buggy behavior and we fix the bug, your program will break. But we try very hard to break as little as possible and keep Go boring.
No, they didn't say that, they said it wouldn't be backwards-incompatible with Go 1. Relevant quote:
> [...] when should we expect the Go 2 specification that breaks old Go 1 programs?
> The answer is never. Go 2, in the sense of breaking with the past and no longer compiling old programs, is never going to happen. Go 2 in the sense of being the major revision of Go 1 we started toward in 2017 has already happened.
You can install Go 1.22 and your program will compile and run as-is. That's the promise. If however you opt-in to the changed for loop behaviour by adjusting your go.mod, the onus is on you to update your program accordingly.
It's only a backwards incompatible change if the developer makes a backwards incompatible change by updating the configured target version.
(I'm aware I'm probably being pedantic here, I understand the language used seems to imply you can just set it to v1.22 and it works but it's a bit more specific)
I remember thinking that the number of people who have created inadvertent bugs due to this design (myself included) would be significantly greater than the number of people affected by the fix.
This makes me wonder, though, what guarantees that a similar breaking change won't ever happen again in the future? If any change with #(programs fixed) >> #(programs broken) is accepted, we might as well remove the compatibility promise page [2].
---
https://twitter.com/go100and1/status/1690412229135601664
https://twitter.com/go100and1/status/1690587305806057472
https://twitter.com/go100and1/status/1690589791686119424
https://twitter.com/go100and1/status/1690591234715492352
https://twitter.com/go100and1/status/1690593184857145344
https://twitter.com/go100and1/status/1691456732151889920
Most of them are not mentioned in the proposal doc at all.
This should be enough to show that it still can wind up as a problem in Python:
funcs = [(lambda: x) for x in range(3)]
funcs[0]() # outputs: 2Python used to be worse; it used to share scope outside the list comprehension.
To get the desired behavior, you can pass x as a default argument to the lambda:
funcs = [(lambda x=x: x) for x in range(3)]
funcs[0]() # outputs 0When you do "for k, v := range someMap", is "v" of the map's value type (and one binding for the whole loop, copied before each iteration)? This would explain the problem, but I would have expected "v" to be a reference into the map, and I couldn't find the answer in a quick skim of the "For statements with range clause" in the spec. I'm probably looking in the wrong place because I touch Go rarely...
[1] https://go.dev/ref/spec#For_statements
edit: oh, the answer is in the "code block"-formatted table. Guess I had banner blindness. "v" is the copied value, not a reference. I'm surprised!
I'm asking because I think if it were expected that folks used large/expensive-to-copy map values, this construct would return a reference instead of copying. In Rust for example, the std library's "normal" [1] iterators return references.
[1] not those returned by into_* or drain.
The peer comments along the lines of "the expecation is it does what it does" are not so helpful from a perspective of learning to write code that is in harmony with the language philosophy.
Value is always a copy. Either a copy of a struct (in case of map[T]struct{}) or a copy of a pointer (in case of map[T]*struct{})
Anyway I believe the answer is that expensivetocopyvalue is not a type that exists in golang, because golang's "copy" operation is always a simple bitwise copy ala C struct copy / Rust's Copy trait, not like C++ copy ctor / Rust's Clone trait that can be arbitrarily expensive.
With array slots, the same issue is present but is a bit more explicit because those resizes happen with `mySlice = append(mySlice, ...)`.
Maps in golang are of reference type, just to be clear.
implementation: https://go.dev/src/runtime/map.go
https://github.com/adobe/kratos/blob/93246f92d53feba73743dbf...
https://github.com/StalkR/goircbot/blob/6081ed5d1d74f01767d7...
Basically the compiler is translating
go a.Monitor(b)
(&a).Monitor(b)
How does this work? If I pull in a package that decided to pin 1.22 (as they should) and I compile with 1.18, would it compile or error that I need to use the 1.22 compiler?
edit: Actually, though, I just realized what I am talking about is different than what you and your quote is talking about, which is what happens when you have a dependency that declares a different version, not the current module. Oops.
But your code, when compiled with go 1.22, will still have go 1.18 semantics.
Thus it will be actively dangerous using go 1.18. I understand it like that.
With go 1.19, you will get a compiler error.
But as go is not fixing security bugs in old releases and std library, I think it is dangerous to use them anyway.
A bit of a harsh way to phrase that. In my experience, the backwards compatibility promises have been very good, and the way you stay up-to-date with security fixes and bugs in the standard library is to upgrade Go.
I know that may strike terror in the hearts of developers used to the nightmare that major version upgrades can be in other languages, where a major version upgrade gets a multi-week task added into the task tracker, but it's completely routine for me to upgrade across Go major versions just to get some particular fix or to play with a new feature. I expect it to be a roughly five minute task, routinely.
The only thing that has bitten me about it is arguably not even Go's fault, which is its continuing advances in TLS security and the increasing fussiness with which it treats things connecting with old-style certificates. I can't even necessarily disagree... I would also like to upgrade them but while it's my server, the clients connecting to it are using certs that are not mine and it's out of my control.
I don’t think we disagree? There is no reason to use old version of go.
I speak about grandparent comment who wanted to still run go1.18. It is not a good idea to still run go1.18, as it doesn’t get security updates.
go: errors parsing go.mod:
go.mod:3: invalid go version '1.21.0': must match format 1.23
for _, informer := range c.informerMap {
informer := informer
go informer.Run(stopCh)
}
for _, a := range alarms {
a := a
go a.Monitor(b)
}
https://github.com/adobe/kratos/blob/93246f92d53feba73743dbf...
https://github.com/StalkR/goircbot/blob/6081ed5d1d74f01767d7...
The difference is that in one case, informer is an interface, so the method call resolves informer.Run immediately and there's no issue. In the other case, a is a struct Alarm, and gets copied by value, and the Monitor method takes a pointer receiver. So my original intuition was right, the compiler is essentially translating
go a.Monitor(b)
go (&a).Monitor(b)
func soldAnother(a *album) {
a.copies++ // note the pointer. Can use dot notation on pointer,
// equivalent to (*a).copies++) foo, err := getFoo()
if err != nil ...
bar, err := getBar()
fmt.Println(bar)
if foo, err := getFoo(); err != nil
It also introduces invalid states - what does getFoo return if it returns an error too. Do we change the API to return a pointer and return nil, or do we have a partially constructed object that is in an invalid state?
I'm not sure how thorough Go's escape analysis is, but nearly all programs that capture the loop variable in a closure and are not buggy right now could be shown to have that closure not escape by a sufficient thorough escape analysis. On the other hand for existing buggy programs, then perf hit is the same as assigning a variable and capturing that (the normal fix for the bug).
Google saw no statistically significant change in their benchmarks or internal applications.
https://github.com/golang/go/wiki/LoopvarExperiment#will-the...
Or is it just because it’s Golang and they’re “we’ll never release a go v2 even if we actually do release go v2 and call it v1.x”
Nim handles this with `closureScope` and `capture` ( https://nim-lang.org/docs/system.html#closureScope.t,untyped ).
It's debatable if anything non-automatic counts as a "fix".
Nim has for-loop macros that should make it possible to instead say `for i in captured(0..5): ...` or maybe `for closed(i) in 0..5: ...`. That might be a bit nicer, but would still be non-automatic. So, unnecessary qualifications can exist / persist.
Speaking of persistence, sometimes loop bodies evolve and the x:=x magic used to be needed but is no longer. Version control history might help decide if the extra step was unnecessary or vestigial, though it would surely slow things down (and perhaps run into intermediate uncompilable states of the code). No idea if David Chase and rsc looked at that aspect in their analysis.
To me, this clearly should be considered a breaking change, which I normally would expect to look out for when the major version number changes. I get that checking module definitions means unchanged code won’t break, but it might break code in ways I as a programmer would not expect when upgrading minor versions. It might be technically correct according to some definition, but lacks practicality, which has been a recurring feeling for me as I get into the language.
// FIXME: this apparently needs to be “fixed” ..
for _, v := range intvals {
go func(v0 int) {
fmt.Printf(“v:%v v0:%v\n”, v, v0)
}(v)
}But you're describing a linter, which just outputs a line on your terminal with a warning; I wouldn't want a tool like that to add churn and tasks to my codebase (even though I'm guilty of adding TODOs myself and leaving them for years because ultimately they're not important enough)
The next best alternative is introducing a new construct for this version. But then you either risk people still using the old one or you need to break lots of fine code by removing the old construct. So in this case the "in place" upgrade made the most sense.
Tying it to the declared compiler version is much like Rust's edition system or Perl's versioning, except tacked into an existing identifier rather than a separate variable. (The downside being that you are forced to make this upgrade at some point if you want to raise your minimum toolchains version. )
If it doesn't fail I have a couple more ideas, if the compiler can prove my double only ever interacts with ints then... just do what I mean it's provably correct.
package main // YUM! I WANT MOAR DWIMMY.... AND SIGILS AND BLESS AND UNLESS
func easy(one int, won float64) { print(one + won) }
func main() { easy(1, 1) }
in this case it captured the reference which at the time of invocation always pointed to the same value.
- for _, e := range es {
- pumpUp(&e)
}
+ for i := range es {
+ pumpUp(&es[i])
}
Go continues to be my favorite language and experience to build and maintain in.
They got so much right from the start, then have managed to make consistent well reasoned, meaningful and safe improvements to the language over the years.
It’s not perfect, nothing is, but the “cost” of maintaining old code is so much lower compared to pretty much every other language I have used.
We'll likely continue using Typescript as our main language for a while since we're a small team and it lets us share resources better, but we're definitely keeping an eye on Go.
Something we have experienced over and over is that devs moving from languages like C# or Java just love how easy and straight forwarding developing in Go is. They pick it up in a week or two, the tool chain is just so simple, there's no arguing around what languages features we can and can't use.
Almost everyone I've spoke to finds it incredibly productive. These people want to be delivering features and products and it makes it easy for them to do so.
Language abstractions exist to prevent having developers build their own ad-hoc abstractions, and you find this time and time again in languages like Go. You can read the Kubernetes code and see what I mean, they go out of their way to work around some of the missing language features.
Can you link to some of these workarounds? I'm curious to see whether they actually make a lot of difference. In theory (and I have no experience with any software project with more than ten developers working on it), they only made it more difficult by adding cleverness.
I'd love to hear of any resources that can help me understand the Zen of Go, because so far I just don't get it.
If you are the lucky ones not dealing with databases, I sort-of envy you...!
Proponents argue that this forced simplicity enhances productivity on a larger organisational scale when you take things such as onboarding into account.
I'm not sure if that is true. I also think a senior Python / Java / etc resource is going to be more productive than a senior Go resource.
Like a one-line list comprehension to transform a collection is suddenly four lines of go: allocation, loop iteration, and append (don't even start me on the append function). I don't care about those housekeeping details. Let me read the business logic.
Modulo extremes like Java, the bottleneck for programmers understanding code is about semantics, not syntax -- effectively never the literal SLoC in source files. It's not as if
for i := range x {
x[i] = fn(x[i])
}
x.transform(fn)
What's the go equivalent to
x.map(fn).filter(predicate)
Probably something like
var output []T
for _, val := range input {
if newval := transform(val); allow(newval) {
output = append(output, newval)
}
}
For some operations, the Go style of explicit, mostly in-place mutations produces more complicated code. Whether that's balanced out by the code being "simpler" is not clear to me, but I haven't worked with Go.
I would not say it's obvious what the machine is doing in the Go example though. For example it wasn't clear to me that append() mostly doesn't copy the full vector, but does a copy of the slice pointer. I had to look it up from a blog post, because the source for append() is gnarly
https://github.com/golang/go/blob/go1.16.7/src/cmd/compile/i...
Well I guess you do have to grok the language spec and semantics in order to understand how builtins like append behave, I'm not sure that's avoidable.
x.transform(func(value int) string { return fmt.Sprintf("%b", value) })
But this applies to other languages as well, in JS (which has a function shorthand) I prefer to extract the predicates and give them a meaningful name.
between
var y []int
for _, x := range x {
y = append(y, function(x))
}
let y = x.iter().map(function).collect();
Go is not a high performance language which made a lot of decisions that don't lend its usage to be nice in scenarios where people want C and Rust. However, with the hype around it, the management continues to make decision, to everyone's detriment, to utilize Go in performance sensitive infrastructure code which one could write in Rust or C# and achieve much higher performance.
Go is all about mechanical sympathy, and is absolutely a high performance language. I guess it all depends on your context, though. If you're used to writing assembly or C, things may look different.
(A "for" loop expresses much more mechanical sympathy than a list comprehension, as an example.)
But at least in the context of application services -- programs that run on servers and respond to requests, typically over HTTP -- Go is the language to beat. I've yet to see an example of a program where the Rust implementation is meaningfully more performant than the Go implementation, and I've got plenty of examples where the Go implementation is much better.
However, it's a dangerous tool that some people just can't be trusted with. Second, if you go full FP style, then you can't just hire a Go developer, they need additional training to become productive.
Here's an example of functional programming within Go taken far: https://github.com/IBM/fp-go/blob/main/samples/http/http_tes.... It basically adds a DSL on top of Go, which goes against its principles of simplicity.
There was another great resource that explains why functional programming in Go is a Bad Idea; one is function syntax (there's no shorthand (yet?)), the other is performance (no tail call optimization), and another is Go's formatter will make it very convoluted; I think it was this one: https://www.jerf.org/iri/post/2955/
People, esp from a Java-esque class based world want class inheritance and generics and all that jazz. I’ve found at work like 50% of methods and logic that has some sort of generic/superclass/OOP style abstraction feature only ever has 1 implemented type. Just use that type and when the second one shows up… then try to make some sort of abstraction.
For context, I can’t remember the last time that I actually used “interface{}”. Actual interfaces are cheap in go, so you can define the interface at use-time and pretty cheaply add the methods (or a wrapper) if needed.
If you’re actually doing abstract algorithms and stuff every day at work… you’re in the minority so I don’t know but all the CRUD type services are pretty ergonomic when you realize YAGNI when it comes to those extra abstractions.
Edit: also f** one liners. Make it 2 or three lines. It’s ok.
Similarly I’m not sure you would like working with Typescript in my team. Our linter is extremely pedantic, and will sometimes force you to write multiple lines of code for what could probably have been a one liner. Not always, mind you, but for the things we know will cause problems for some new hire down the line. (Or for yourself if you’re like me and can’t remember what you ate for breakfast). The smaller the responsibility, the less abstraction and the cleaner your code the easier it’ll be to do something with in 6+ months. Now, our linter is a total fascist, but it’s a group effort. We each contribute and we alter it to make it make sense for us as a team, and that’s frankly great. It’s nice that the ability to do this, and the ability to build in-house packages, is so easy in the Node ecosystem, but it’s still a lot of work that Go basically does for you.
So the zen is in relinquishing your freedom to architect the “linguistics” of your code and simply work on what really matters.
I’ve never used Interface{}.
In which universe? They have to constantly patch the language up and go back on previous assumptions.
But, going on a well-trodden path slower than the pioneers is not a big achievement.
Smart men learn from the mistakes of others.
Robert Griesemer, Rob Pike, and Ken Thompson are objectively smart men and pioneers and have learned from lots of mistakes both they and the industry made.
Go embodied a lot of those learnings out of the gate.
If the bar is to be perfect out of the gate that’s impossible and I can’t think of any language that could pretend to be so.
Go was very good out of the gate and has slowly but surely evolved to be great.
Nothing is perfect to begin with, and I think you could probably be a bit kinder to Golang here.
-- https://en.wikipedia.org/wiki/Green_thread
Also see https://stackoverflow.com/questions/5713142/green-threads-vs... , https://docs.oracle.com/cd/E19455-01/806-3461/6jck06gqe/inde... .
Sure, it may not be perfect, or as elegant as other languages, but it's consistent and predictable.
Reference for this?
>just an academically interesting property - had zero real life impact.
Only because Java runs on the JVM, which preserves enough type information that an exception will be thrown if you try to treat a list of String as a list of Integer. In a language that compiles to native code with full type erasure, that could give rise to serious security problems.
You can just imagine what Go critics would be saying on HN if Go's generics implementation allowed that kind of thing to happen!
No, you need to go way way out of your way to make a real world example for that - otherwise it would have been discovered by a bug, not by academics. It’s similar to Java’s generics being Turing complete - cool, but you can never make use of that/write it accidentally.
> Fortunately, parametric polymorphism was not integrated into the Java Virtual Machine (JVM), so these examples do not demonstrate any unsoundness of the JVM.
In Go there’s pretty much only the http package, and any 3rd party packages extend it and have the same ergonomics.
For a while my biggest gripe was package management but it’s a dream where we are now.
It's somewhat amusing to see Go rediscover old ideas in programming language theory, given the stance against PLT that the Go developers took in the early years of the language.
"It must be familiar, roughly C-like. Programmers working at Google are early in their careers and are most familiar with procedural languages, particularly from the C family. The need to get programmers productive quickly in a new language means that the language cannot be too radical."
And not including sum types despite having a sum-type-shaped hole in the language (`if err != nil`).
And some of the discussion about "why no generics" seemed kind of divorced from existing PL knowledge on the topic.
Go is just simply badly designed, relying on hard-coded functionality a lot.
Go also has some really weird stuff in it, such as named return values.
Frankly, the lack of sum types hurts the most. The language would just be a lot better with a unifying Result type in the library. And don't give me any of that "oh, they tried to keep the language simple!" stuff.
Intuitively, sum types are laughably simple. Everyone understands "It's one of these possible values, so you need to check which one it is and then handle that situation." They are more simple than enums on a conceptual level! Sum types are just not how C-programmers think about the world.
There is a lot of discussion of sum types in Go at https://go.dev/issue/19412.
I also vehemently disagree with them, and I think that the way code is factually written in practice is on my side: People who propose sum types commonly refer to the Option<T> or Result<S,E> types in Rust. These are types which are almost exclusively used as return types.
Interface types are the opposite. They're used as input types, and almost never to distinguish between a concrete, finite list of alternatives. They are used to describe a contract, to ensure that an input type fulfills a minimal set of requirements, while keeping the API open enough that other people can define their own instances.
The fact that Go in fact does not use interface types for its error handling is a pretty good argument in favor of that, I'd say.
The thing is, at this point it doesn't matter, sadly. Adding sum-types to the language now would be unsatisfying. You would really need proper integration as the primary tool for error handling in the standard library (and the ecosystem), and that's unlikely to happen, even less likely than a Go 2.0.
EDIT: Just to make it clear, I think not wanting to add sum types to the language is understandable at this point. The real shame is that they were not in the language from the beginning.
(Separately, I'm not entirely sure what you mean when you say that Go doesn't use interface types for its error handling, since the language-defined error type is in fact an interface type.)
Frankly, I'm just the type of person who doesn't understand why it is possible to silently drop error values in Go (even accidentally), see
f, err := os.Open("foo.txt")
g, err := os.Open("bar.txt")
if err != nil {
return
}
It seems like a pretty severe flaw for a language that takes pride in its explicit error handling, and a deep dive into why this flaw was acceptable to the creators would be really interesting.
For now though, instead of sum types we ended up with features such as named returns (???). I imagine some of the complexity here was about not wanting to introduce an explicit tuple-type, since a Result<S, E>-type doesn't compose with multiple returns. (I feel like there should be some workaround here. Maybe anonymous structs + some sort of struct/tuple unpacking, but I could see it getting gnarly.)
> I'm not entirely sure what you mean when you say that Go doesn't use interface types for its error handling, since the language-defined error type is in fact an interface type.
What I meant is that this specific usecase of sum-types (ie. error-unwrapping) is not something that interfaces in Go are used for. Error-handling in Go is done via multiple return values. This goes against the common claim that "sum types and interfaces are too similar/have the same uses", and should count for something, considering that explicit error handling is a big component of Go.
I'm not personally concerned about examples like os.Open, where the result must be used. Sure, you can ignore the error. But a failure will show up very quickly. I'm not saying that this is not an issue at all, but I believe it's a less important one.
Part of the reason for Go's behavior is the idea that "errors are values" (https://go.dev/blog/errors-are-values). And part of it is that the simple fmt.Println("Hello, world") doesn't require any error handling. And part of it is the desire to make error handling clear and explicit in the program, even if the result is verbose.
Rob Pike and Ken Thompson are theory-unaware. Sure. Yes. This position is good and defensible.
I misspent a few thousand hours of my life on the 9fans mailing list long ago when Pike was very active on it, and my non-joking assessment is that ever since he finished his PhD or shortly after, Pike has probably felt he knows all he needs to learn about programming-language design except for the things he and the people in his immediate social environment invent.
Bell Labs was never good at designing programming languages. Did you know that in the Bourne shell (and possibly in all the other shells) you can have a statement of the form $foo = bar which will assign bar to the variable whose name is the value of foo? (Emacs Lisp, an old language, has the same functionality in the form of a function named "set", but most Emacs Lisp programmers know to avoid it.) Well, I found that statement in a shell script written by one the Bell Labs guys, and the shell script was not doing anything fancy like interpreting a programming language or defining a new PL feature (not that it is sane to do either of things in a Unix shell).
None of what I say is more than a wisp of a reason not to choose Golang IMHO.
I think Go is a perfectly fine language, and I respect the goal to stay clear of complexity, but when looking at any particular Go feature, then it's easier to explain the decision with "They are used to thinking in terms of C-idioms." than with any particular brilliance or awareness of PL-theory.
Sum types have been discussed since before the initial open source release of the language, at least according to some of the issue threads such as https://github.com/golang/go/issues/19412
If they're laughably simple, then please contribute a proposal for how to add them to the language. You'll find no one is really fighting against the concept of sum types.
The thing which I am personally salty about is that they weren't there to begin with, and that the language wasn't built with them in mind.
There's nothing casual about Go's approach to language design, and I haven't seen any evidence that they're unaware of what other languages do.
I also haven't seen much criticism of languages other than C++ or Java.
Non PL theory but related: incorrect implementation of monotonic clocks, and then refusing to fix it because “just use google smear time bro”.
The error design can return the value, or an error, or both, for some inexplicable reason and you can check it - or not - and if your function returned some indication of severe error, and you happen to not check it, you can totes just continue your program, in who knows what invalid state. Oh and also, because the devs are apparently deathly allergic to abstractions of apparently kind, you’ve got to do this janky little if err != nil check at. every. single. point. Which occludes your fundamentally important logic in pointless line noise, with zero opportunity for streamlining or improving the logic or guarantees.
https://fasterthanli.me/articles/i-want-off-mr-golangs-wild-...
However, that still doesn’t mean that it’s wrong.
(though I disagree that ".ext" is not an extension of an empty base name)
> a = []; for (i = 0; i < 3; i++) a.push(() => i); a.map(f => f())
[ 3, 3, 3 ]
> a = []; for (let i = 0; i < 3; i++) a.push(() => i); a.map(f => f())
[ 0, 1, 2 ]
> a = []; for (i of "abc") a.push(() => i); a.map(f => f())
[ 'c', 'c', 'c' ]
> a = []; for (const i of "abc") a.push(() => i); a.map(f => f())
[ 'a', 'b', 'c' ] var _loop = function (i) {
a.push(() => i);
};
for (var i = 0; i < 3; i++) {
_loop(i);
}
(If you’re curious how Babel deals with the more complicated cases of break/continue, labelled break/continue, and return, try it out at https://babeljs.io/repl.)
Could an LLM coupled with a constraint solver create the perfect language (for particular domains, e.g., performance)? Or, just use Rust ;-)?
In this particular case I imagine the issue was uncovered some time in the 1970s, which is when lexical scoping came to the fore in Scheme (in the US) and ML (in Europe). It's a fairly natural problem to run into if you're either thinking deeply about the interaction between closures (which capture environments) and loop constructs, or if you're programming in a language that supports these features.
Nil being another big one. Even more impressively they doubled down on this mistake with typed nils. Even if you explicitly do a comparison with nil you can still shoot yourself in the foot because it was a different nil than the one you compared against.
func foo() *bar {
// ...
if something_wrong {
return nil;
}
}
var x interface{}
x = bar()
if x != nil {
// Dereference x
}
https://dave.cheney.net/2017/08/09/typed-nils-in-go-2
If Dave Cheney says it hits every go programmer at least once, it caused hours of consternation for his coworkers, and it even has its own entry in the language FAQ, I don’t know what else to tell you.
Furthermore, even for experienced developers, there's a limit to how much context / rules / whatever your brain can keep. This footgun takes up space and intellectual energy that could be used for something else.
All things being equal, a language that doesn't have this kind of footgun is better than one that does: less experienced reviewers will let fewer bugs slip through, and more experienced reviewers will either spend less effort reviewing (meaning the mental energy can be used somewhere else) or will have more review capacity (meaning they'll find more bugs / improve the code more).
https://github.com/gwd/session-scheduler/blob/master/handle_...
Basically, I have several pages I'm rendering, which have common prerequisites regarding checks, and common handling processes (passing some sanitized data to a template). The *GetDisplay() functions take a structure from the "database" layer and sanitize it / process it for handing to the templates. The two *GetDisplay() functions return pointers to two different types, appropriate for the template to which they will be passed; and return nil if there's an issue.
So I have a map, `data` of type `map[string]interface{}` that I pass into the templates; and two different paths set `data["Display"]`; then at the end I want to check if either of the `*GetDisplay()` functions returned `nil`. So naturally, the first version of the code checked `data["Display"] == nil`, which was always false, since it was implicitly checking `data["Display"] == interface{}(nil)`, but the value in case of an error would be either `*DiscussionDisplay(nil)` or `*UserDisplay(nil)`.
I mean, sure, there are other ways to structure this; I could return an error or a boolean in addition to returning nil. But 1) the only reason to do that is to work around this language limitation 2) it's a "foot gun" that it's easy to fall into.
And sure, a golang developer who'd shot themselves in the foot a few times with this would catch it during review; but I don't think a bunch of newer developers would catch it, even if they had extensive experience in other languages.
data := map[string]interface{}{}
if ... reflect.ValueOf(display).IsNil() {
So first of all, the reason things are defined that way is to interact with the golang templating libraries. Secondly, your suggestion wouldn't really solve the issue in this case: content of "Display" is different for each web page, and so the only way to assign both types to the same value is to make it an interface.
> Any use of `package reflect` in application code is a similarly huge red flag. 99 times out of 100 it's a design error that ought to be fixed.
I'm not using reflect for fun; there is literally no other way to check for nil with interfaces (other than manually checking nil for all possible types).
At any rate, I wrote code in a way that's intuitive, at least to a C programmer (using 'nil' value as an indicator that there was an error); the code had a bug. Sure I could have rearchitected the whole function, and if this were a commercial product I may have. But it's a simple webapp to help scheduling discussions at my project's conferences; a quick fix that robustly works around Golang's deficiencies is perfectly reasonable.
EDIT: And honestly, there are exactly three ways of addressing this:
1. Separating the two page paths, duplicating all the logic which is common to the two. This makes it less DRY, which risks checks becoming inconsistent, increasing the chance that there will be a security issue.
2. Make the *GetDisplay() functions return a second value to indicate failure. This is honestly kind of a dumb thing to do to work around a language deficiency.
3. Continue to use 'nil' to indicate failure, and fix the check to be able to properly check for nil. This can be done by listing out the various possible values of 'nil', which is ugly, annoying, and fragile (since it would silently break if we added a third type); or it can be done using reflection.
#3 is obviously the most reasonable thing to do here.
if display := data["Display"]; display == nil || reflect.ValueOf(display).IsNil() {
<code>
if _, ok := data["Display"]; !ok {
<code>
var typeA Interface = (*TypeA)(nil)
println(typeA == nil) // false
println(typeA == (*TypeA)(nil)) // true
And generics is an example of a language feature they took their time for, to avoid making the same mistakes as e.g. Java did, where generics ended up taking up half the language spec and compiler / runtime implementation.
In any case - I seem to remember that they discussed the rationale for the original decision in the release notes for go 1.21, along with additional context.
For whatever it's worth, I don't see any evidence that Go is specifically antagonistic to programming language theory at all - the existence of first-class constructs like channels and closures suggests otherwise. There are always costs and tradeoffs involved in adopting certain theoretical paradigms, and PLT is subject to fashion as much as any other endeavour.
Go focusses on simplicity, and when talking about simplicity I really like this quote from Dijkstra:
"Simplicity requires hard work to be obtained and education for its appreciation, and complexity sells much better.” [0]
[0] https://www.cs.utexas.edu/users/EWD/transcriptions/EWD10xx/E...
Then again I also can't deny that the lack of ""advanced"" features forces you to keep your code simple, which makes reading easier. So while I hate writing Go, I like reading unfamiliar Go code due to a distinct lack of magic. Go code always clearly spells out what it does.
I could complain all day about things the language does obviously wrong, often in the name of simplicity. But after all my complaints I still admit it’s a very good choice for certain kinds of software and software companies.
Actually it surprises me we're still inventing languages where local variables can be mutated, which seems to be at the root of the problem here
Huh? Where’s the list? From the top of my head I think this is the only thing that repeatedly bit me, although I’m very aware of the behavior of for loop scoping. Linters save me nowadays at least.
Are there other things like that in the language that deserve a fix? Maybe things to do with json un/marshaling?
type noCopy struct{}
func (*noCopy) Lock() {}
func (*noCopy) Unlock() {}One weird thing that always goofed me up was that slices are passed by value but maps by reference. Always made it confusing how to pass them for serialization/deserialization. The compiler didn't complain it just panicked. Seemed like something the type system should catch.
I'd much rather have a crash than silently corrupting output.
We've known about the problem for a long time. I have notes from the run up to Go 1 (circa 2011) where we considered making this change, but it didn't seem like a huge problem, and we were concerned about breaking old code, so on balance it didn't seem worth it.
Two things moved the needle on this particular change:
1. A few years ago David Chase took the time to make the change in the compiler and inventory what it broke in a large code base (Google's, but any code base would have worked for that purpose). Seeing that real-world data made it clear that the problem was more serious than we realized and needed to be addressed. That is, it made clear that the positive side of the balance was heavier than we thought it was back in 2011.
2. The design of Go modules added a go version line, which we can key the change off. That completely avoids breaking any old code. That zeroed out the negative side of the balance.
As for validating your software, the answer is the same as its always been… tests, tests and more tests.
Local mutability is probably one of the most common uses of mutability. A lot of it is using local state to build up a more complicated structure, and then getting rid of that state. Getting rid of that use-case is just giving up performance.
They aren't local, but belong to the outer scope. The misconception in a nutshell.
This could have been prevented by having one person on the team with actual language design experience, who could point this issue out in the design process.
In this case, after 10 or so years, and thousands of production bugs, they backpedaled. How many other badly designed features exist in the language, and are simply not being acknowledged?
If you point it out, and you're right, will you be heard if you don't have a flashy metric to point to, like a billion dollars lost?
What if the flaw is more subtle, and explaining why it's bad is harder than in this very simple case, that can be illustrated with 5 lines of code? What if the link between it and its consequences isn't that clear, but the consequences are just as grave? Will it ever get fixed?
Seriously though, "having experience" and "getting things right" are two different things, although Golang got a lot of things right, and the parent is being unnecessarily harsh.
That decision only becomes painful when capturing variables by reference becomes cheap and common; that is, when languages introduce lightweight closures (aka lambdas, anonymous functions, ...). Then the semantics of a for loop subtly change. Language designers have frequently implemented lightweight closures before realizing the risk, and then must make a difficult choice of whether to take a painful breaking change.
The Go team can be persuaded, it's just a tall order. And give them credit where credit is due: this is genuinely a significant, breaking change. It's the right change, but it's not an easy decision to make more than a decade into a language's usage.
That said, there may be a kernel of truth to what you're alluding to: that the Go team can be hard to persuade and has taken some principled (I would argue, wrong) positions. I'm tracking several Go bugs myself where I believe the Go standard library behaves incorrectly. But I don't think this situation is the right one to make this argument.
The outcome of this go code in java would be as you'd expect, each lambda generated uses a unique copy of the loop reference value.
I suppose Java had many years after C#'s introduction of closures to reflect on what went well and what did not. Go, created in 2007, predates both languages having lightweight closures. Not surprising that they made the decision they did.
Your comment inspired me to ask what Rust does in this situation, but of course, they've opted for both a different "for" loop construct, but even if they hadn't, the borrow checker enforces a similar requirement as Java's effectively final lambda limitation.
C# 3.0 in 2007 introduced arrow syntax. I believe this was primarily to support LINQ, and so users were typically creating closures as arguments to IEnumerable methods, not in a loop.
C# 4.0 in 2010 introduced Task<T> (by virtue of .NET 4), and with this it became much more likely users would create a closure in a loop. That's how users would add tasks to the task pool, after all, from a for loop.
C# 5.0 in 2012 fixes loop variable behavior.
I think the thesis I have is sound: language designers did not predict how loops and lightweight closures would interact to create error-prone code until (by and large) users encountered these issues.
The change in Javscript doesn’t have anything to do with for…of, it’s the difference between `var` and `let`. And JS made the decision to move to `let` because the semantics made more sense before Go was even created (although code and browsers didn’t update for another several years). That’s why Go is being held to a higher standard, because it’s 10+ years newer than the other languages you mentioned.
1. Did JS introduce this change after Go's creation? Yes. (And also after C#.)
2. Did arrow functions support precede let and const support? Yes.
Answering the second question and finding the versions with support and their release dates answers the first question.
Arrow functions let and const
Firefox 22 (2013) 44 (2016)
Chrome 45 (2015) 49 (2016)
Node.js 4 (2015) 6 (2016)
Safari 10 (2016) 10 (2016)
This is somewhat weak evidence for the thesis though; these features were part of the same specification (ES6/ES2015), but to understand the origin of "let" we also need to look at the proliferation of alternative languages such as Coffeescript. A fuller history of the JavaScript feature, and maybe some of the TC39 meeting minutes, might help us understand the order of operations here.
(I'd be remiss not to observe that this is almost an accident of "let" as well, there's no intrinsic reason it must behave like this in a loop, and some browsers chose to make "var" behave like "let". Let and const were originally introduced, I believe, to implement lexical scoping, not to change loop variable hoisting.)
This is one of the things the language was designed to fix, by people that looked at the past 50 years or so of programming languages, and decided to fix the sorest pain points.
I would argue that var is an entirely different issue. If variables last the entire function then it's far less confusing to see closures using the final value. After exiting the loop the final value is right there, still assigned to the variable. You can print it directly with no closures needed.
> Russ Cox and the Go team learned that the loop variable capture semantics are flawed not by reflecting about how their language works, but through user feedback.
I think "user feedback" isn't the whole story. It's not just the Go team passively listening as users point out obvious flaws. I've noticed in other changes (e.g. the monotonic time change [1]) the Go team has done a pretty disciplined study of user code in Google's monorepo and on github. That's mentioned in this case too. This is a good practice, not evidence of failure.
[1] https://go.googlesource.com/proposal/+/master/design/12914-m...
They are huge names in the field, but honestly, they just suck at language design itself.
I'm equally sure that if you asked kubb, kaba0, and three other strongly opinionated folks for a list of good language designers, each of the <5 lists you get back would be very short, and there'd be no overlap between them.
Since "Go 1" was deemed complete and the "Go 2" project began in 2018, the direction of the language was given to the community. It is no longer the Go team's place to shove whatever they think into the language. It has to be what the community wants, and that feedback showed it is what they want.
When "Go 1" reached its natural stopping point and closed down, the "Go 2" project emerged to continue development of Go under the wants and needs of the community. That capture is being added now because the community has shown a desire to have it. The Go Team may use their expertise to guide the community in the right direction, but we are here because the "Go 2" project is community driven.
The original commenter seems unaware that the project changed hands.
I thought you were serious, right up to that bit. Well played. I hope.
Instead of making a mistake, they could have simply not.
See also RFC 9225: Software Defects Considered Harmful https://www.rfc-editor.org/rfc/rfc9225.html
Very few.
> If you point it out, and you're right, will you be heard if you don't have a flashy metric to point to, like a billion dollars lost?
If you're right yet don't have a better idea then what do you expect to occur?
> What if the link between it and its consequences isn't that clear, but the consequences are just as grave?
The consequence is your developers must be careful with loop variables or they will introduce bugs. That's not particularly "grave" nor even especially novel.
I'll admit, it's not a good ivory tower language, but then again, that's probably why I use it so often. It gets the job done and it doesn't waste my time with useless hypothetical features.
I find Go quite frustrating in how it decries how over-complicated some features are, and slowly comes around to realize that oh, maybe people designed them for a reason (who woulda thunk it?).
Is this a subtle nod to the billion dollar mistake?
Because they deliberately included the billion dollar mistake as part of the language.
Even if they knew better than to include the billion dollar mistake, they were probably aware that they couldn't make a popular language without including it.
Because most users demand it.
What do you mean by “hard”?
I find Rust syntax challenging to grasp in a specific way. Rust employs numerous symbols and expressions to convey statements, which makes reading Rust code a process of constantly navigating between different keywords, left and right. I have to create a mental map of what certain statements are accomplishing before I can truly comprehend the code.
In contrast, I find Go code relatively straightforward, especially for those familiar with C-like programming languages. This clarity is due to the deliberate verbosity of the language, which I personally appreciate, as well as the use of early return statements.
But don’t get me wrong. I enjoy programming in both Rust and Go when they are suitable for the task at hand, but I usually spend more time grappling with Rust’s syntax than with Go’s, because I often invest more time in understanding the structure and logic of Rust programs compared to their Go counterparts.
I have similar issues with Rust actually. There’s a lot of sugar used that you have to grok and that takes some time.
On the other hand Python, C#, Java all stick with a set of fairly familiar conventions. In terms of syntax (and only syntax), the learning curve is more intense with Go; perhaps similar to the initial alienation provided by JavaScript.
My experience has been that once you are being paid to learn a language these problems mostly disappear. Alas, no one ever paid me to learn Go.
You add them and the next thing people want are objects.
Go has structs and functions that work on those structs, good enough for me.
It could have been based on tuples and records (similar to SQL), but we somehow got typeless OOP.
Wasted opportunity.
Programmers are often at fault when a language complex, but I'd give them a pass when it's simply counterintuitive.