Go 1.21 Released
go.dev
go.dev
This is a very nice QoL improvement, I wish more languages made the change.
>>> a()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 3, in a
File "<stdin>", line 3, in a
File "<stdin>", line 3, in a
[Previous line repeated 894 more times]
File "<stdin>", line 2, in a
File "<stdin>", line 2, in b
File "<stdin>", line 2, in c
File "<stdin>", line 2, in c
File "<stdin>", line 2, in c
[Previous line repeated 97 more times]
RecursionError: maximum recursion depth exceeded
Mistakes are normal in any language. Debugging an infinitely deep recursion can be challenging in most.
1.21 63MB
1.20 95MB
1.19 142MB
1.18 135MB
1.17 129MB
1.16 123MB
1.15 116MB
1.14 118MB
1.13 114MB
1.12 121MB
1.11 121MB
1.10 114MB
1.9 98MB
1.8 86MB
1.7 78MB
1.6 81MB
1.5 74MB
> Go command The directory $GOROOT/pkg no longer stores pre-compiled package archives for the standard library: go install no longer writes them, the go build no longer checks for them, and the Go distribution no longer ships them. Instead, packages in the standard library are built as needed and cached in the build cache, just like packages outside GOROOT. This change reduces the size of the Go distribution and also avoids C toolchain skew for packages that use cgo.
> PGO in Go applies to the entire program. All packages are rebuilt to consider potential profile-guided optimizations, including standard library packages.
> The -pgo build flag now defaults to -pgo=auto, and the restriction of specifying a single main package on the command line is now removed. If a file named default.pgo is present in the main package's directory, the go command will use it to enable profile-guided optimization for building the corresponding program.
“In Go 1.21 the linker (with help from the compiler) is now capable of deleting dead (unreferenced) global map variables, if the number of entries in the variable initializer is sufficiently large, and if the initializer expressions are side-effect free.”
There also may be differences in debug symbols in the various binaries.
[1]: https://blog.felixge.de/waiting-for-go1-21-execution-tracing...
[0]: https://github.com/rs/zerolog/issues/571#issuecomment-166202...
Also in the lisp world, Clojure has a fastidious dedication to backward compatibility.
Perl 5 has maintained backward compatibility (though I am not certain if it is 100% compatible) by putting changes behind flags.
As for the latter, those breakages are relatively rare.
Besides that was only a quick set of examples, if I would bother going through my ISO copies, there would be a couple more to list.
Why do you think you need to specify release and nightly to successfully build some projects?
That’s like complaining that the exp packages are not stable.
> To support programs written for older versions of Go, nil panics can be re-enabled by setting GODEBUG=panicnil=1. This setting is enabled automatically when compiling a program whose main package is in a module with that declares go 1.20 or earlier.
An explanation of why this change was implemented is contained in the description of the implementing change, https://go-review.googlesource.com/c/go/+/461956. In short, calling `panic(nil)` before would make `recover` return nil, which is what's returned when there is no panic. `recover` doesn't have a `panicArg, ok = recover()` return value variant, so they fixed it this way.
I've mainly used Java which doesn't suffer from this particular problem. But I've often heard Java criticized on the grounds it captures values not variables, and that most other languages apparently capture variables. I remember learning Lisp at uni and being taught that it captured variables, for example.
So do other languages suffer from this problem as well? How do they deal with it?
1. the interaction between mutable locations and closures
In most languages with mutable bindings, if you close over a variable you're closing over a "cell" and will see future modifications to it. This is the case in Go, Python, Ruby, Javascript, C#, ...
A few languages avoid it e.g.
- Java, not because it "captures values" but because it only allows capturing "final" variables so you can't update the bindings (you can still mutate in place when capturing reference types)
- C++, because you have to specify the capture mode and it'll only be affected if you capture by reference (which you wouldn't do for a loop variable), so the likelihood of hitting this issue is pretty low
- Rust, because even if you capture by reference you can't modify a binding with an outstanding reference
2. the scoping of loop variables, if you're capturing the loop variable but each iteration has a different version of the loop variable then you don't really mind too much, because each closure will capture its own loop variable, this doesn't fix the above issue but it does fix the most common occurrence of it
That's what Go is doing, other languages which have done that are C# (when using a `foreach` loop), Javascript (when using `let` or `const` for the loop variable), ...
Also Ruby, kind-of: using the `each` method for iteration is very common, and since then the loop variable is a parameter to the block it's basically a per-iteration local, "for...in" still has the issue. I guess C# also has that when using the linq ForEach method but I don't know how common that is.
I think Go is doing the right thing here. I'm somewhat surprised they didn't always do this. (Dart has created a new loop variable for each iteration all the way back since before 1.0.)
In Lisp, you can easily write macros that have whatever semantics you want. You can easily write a my-dolist which is like dolist but freshly binds the item variable for every iteration, and next to it write a my-dolist2 which assigns a single variable.
In languages like Go, the providers of your language provide all the syntax from behind a wall, and make these decisions for you. Knights in shining armor duke it out on the rooftop of an ivory tower, while lower vassals blog incessantly about the important work being done and decisions being made.
By the way, MacCarthy's ancient Lisp (Lisp 1, Lisp 1.5) didn't have lexical scope, so it would not have made any difference whether each iteration stepped the variable with setq or bound it with let. There would just be the one dynamic variable in all cases, not captured by any lexical closures (those being nonexistent). You'd want to bind it at least once with let so the prior value is restored when the loop is done.
In Common Lisp, if we use a special variable as the loop variable, it likewise won't make a difference whether it is stepped or bound each time, since the binding isn't lexical.
That, combined with the fact that things are often poorly documented is what IMHO ruins the language ecosystem.
I want to like Lisp and I love its interactivity but I find the language a bit more "construction material" language than a ready-to-use immediately language. What is bad about that is exactly what you mention: everyone will come up with a set of macros that others cannot fully grasp.
If you have to reverse engineer someone's macros, you have all of the following going for you
The macros run at compile time and then go away (unless the program is based on a dynamic compiling paradigm). They have no behaviors at run-time; their generated code does. No matter how complicated the macro, you can just run it and capture the output code, and through multiple examples get an understanding of what it's doing.
If someone writes a buggered function, you may have to end up debugging it on a target system, perhaps an embedded one.
The sky is the limit there. If it's a function in a kernel, you may have to debug some race condition between it, some interrupt handler and a piece of hardware.
No such thing ever happens when debugging a macro itself.
A macro doesn't interact with complex application state, because that doesn't exist. You never have to attach some gigabyte database and get some objects into the right state before reproducing some expansion problem in a macro. Code generated by a macro could be involved in a problem like that, possibly as a root cause, but tracing through the macro itself isn't.
The people who write awful macros are usually not smart enough to do anything that is actually hard to understand. The problems you will find are lack of hygiene, multiple evaluations and such.
Lisp makes this so easy to write macros and they have been promoted to be the "my-powerful-dsl-is-the-right-way-to-solve-a-problem" that, IMHO, it works actively against team work unless it is really well-dcumented. Now, mix that with the fact that code is not usually deeply documented and you get a cultural problem into the whole ecosystem.
> A macro doesn't interact with complex application state, because that doesn't exist. You never have to attach some gigabyte database and get some objects into the right state before reproducing some expansion problem in a macro. Code generated by a macro could be involved in a problem like that, possibly as a root cause, but tracing through the macro itself isn't
Tracing the macro can be as difficult as its expansion. If it gets obtuse it will add a lot of time to your workflow. I know they are powerful but it is probably the last feature I would use in a team, with a lot of care, and for trivial things or for really justified cases and with good documentation. If your code is full of macros, forget about readability, you threw it out through the window.
> The people who write awful macros are usually not smart enough to do anything that is actually hard to understand. The problems you will find are lack of hygiene, multiple evaluations and such
Because writing macros is not that simple in the first place. I find much easier to write regular code and use the well-known macros than grasp anonymous macro code. There are things you can only do with macros though, like lazy evaluation. But macros must be managed with extra care. For example, maybe instead of a macro it is better to use a higher order function with a closure to keep the code understandable than burying things in layers of macros.
You have to get your talking points all in a row.
For example, what does the following Python code print?
funcs = list()
for i in range(10):
funcs.append(lambda: print(i))
for f in funcs:
f()Specifically, I think, any language with closures where the loop variable exists in a scope enclosing all loop iterations with a value that is updated each iteration rather than a fresh variable in a scope local to each loop iteration has this problem.
In Ruby this wouldn't affect idiomatic looping methods like #each, but would seem likely to affect the less-idiomatic for loop, which is built on #each but has the control variable in the surrounding scope. (Not 100% certain how it desugars though.)
I don't know how it desugars, but it is indeed affected when using `for...in`
funcs = list()
for i in range(10):
funcs.append(lambda: print(i))
del i
for f in funcs:
f()
funcs.append(lambda i=i: print(i))Maybe this example where it returns the closures from a function is convincing?
def fs():
funcs = list()
for i in range(10):
funcs.append(lambda: print(i))
return funcs
for f in fs():
f()Java forces you to final copy the variable.
let funcs = [];
;(() => {
let i;
for (i = 0; i < 10; i++) {
funcs.push(function() { console.log('i=',i); })
}
})()
for (let f of funcs) {
f.call(null);
}
x1 := uint64(9007199254740993)
x2 := uint64(9007199254740992)
fmt.Println(uint64(math.Max(float64(x1), float64(x2))))
min/max and math.Min/Max are not equivalent.
The correct way to write min/max before was to use comparison operators. math.Min/Max was there because it requires tricky special cases for handling -Inf, Inf, NaN, and negative zero.
If anyone on the Go team is reading this; thanks! And, as hacker news must always be the armchair commentator, if I may ask for an addition in go 1.22: encoding/yaml. Build it in. Go and YAML are the languages of devops, and I've written so many scripts with JSON as a configuration file only because Go has a JSON parser built-in, but not YAML.
FYI, encoding/yaml was recently requested, and was declined in https://github.com/golang/go/issues/61023#issuecomment-16106..., for reasons I largely agree with.
(I work for Google but not on Go)
Sounds like a feature. Less Yaml in the world is a good thing.
In particular, crypto/elliptic was an unfortunate API that has been deprecated in favor of the new crypto/ecdh. Most applications can migrate (on their own time, as we don't break backwards compatibility even for deprecated packages) and get better security and performance. (A very small portion of applications might need lower level applications, in which case they can use third party modules based on the stdlib internals, like filippo.io/nistec.) You can read more on my Go 1.20 [1] and Go 1.21 [2] posts.
[0]: https://golang.org/design/cryptography-principles [1]: https://words.filippo.io/dispatches/go-1-20-cryptography/ [2]: https://words.filippo.io/dispatches/go-1-21-plan/
1) speed to update it in the case of a bug / security issue.
2) backwards compatibility.
Getting a new Go release out is likely a lot more time consuming than bumping up the crypto library outside of there. And those libraries do not require full backwards compatibility compared to the Go standard library. So they can do breaking changes if need back without breaking the Go contract.
Though, in hopes you see this, I have to ask: Is there any sort of plan for if you wanted to step down from a maintainers role of the go crypto packages?
You're sort of known as "the" go crypto person, so I guess it occurred to me that your bus factor might be quite high. Any thoughts you can give to assuage fears in that regard? Is there anything the community can do to share the load, short of becoming cryptography experts?
As C++ replacement, I would stay is too lacking in features. For that I would rather reach out to C#, D, Java (with OpenJ9/Graal), or even Haskell/OCaml.
Really like how go handles parallelism and creating threads.
I thought I was the only one!
Go already compile fast, and it still gets faster release after release, love it!
I'd love to see Map/Filter/Reduce, but starting with the simplest set of generic functions and getting people comfortable with them is probably a good idea.
Go idiomatically is very imperative and introducing a new programming style at the same time as introducing generics might be too much change at once.
I'd also like to see an option and a result type at some point so we can stop doing nil checks all over the place.
Glad to see this pushed. We went to look how Go was handling constant time comparisons and noticed it was not. This is not considered a serious security issue, as attacker controlled private key attacks are generally considered out of scope, but it's still nice for the library to be safe. (See https://github.com/golang/go/issues/53849)
Everyone that I've ever talked to who went through the transition has a similar story. In practice no one even incidentally relied on the old behavior. Go rolling it out as an experimental opt-in feature is a good idea, but it'll probably turn out to be unnecessarily cautious and they could have gotten away with just unconditionally enabling it.
Seems like they thought about that. I think it's a good change.
Way back when C# actually went through that change unconditionally (LangVersion was not a thing yet), and there was barely any breakage.
For `for ...; ...; ...` loops, personally, I think the conclusion is the inverse.
One breaking case:
func main() {
defer println()
for counter, i := 0, 0; i < 3; i++ {
defer func() {
counter++
print(counter)
}()
}
}
More surprising cases: https://github.com/golang/go/issues/60078#issuecomment-15443...
Have you proved it?
If you want to argue, you really need to show an example of where the change causes a bug in any scenario that matters. With a real, preexisting, program.
I totally disagree with your logic. I think you don't have the right logic here.
At least, the proposal should be accepted when the experiment period has lasted for a year. But now, the proposal has been accepted before the experiment period started. Is it weird?
If you want evidences, please read my recent tweets: https://twitter.com/go100and1
My logic: every domo code matters.
Proving the existence of a program that contains code which will become buggy is easy: share the code. Show actual program code, not a hypothetical; nobody is saying the experiment doesn't change behavior, people are saying it only changes behavior that doesn't matter to real world code.
On real world code discovered so far, the experiment would fix bugs.
It is not my (and all other Go programmers') responsibility to do this. Here, I just shows the possibility that the proposal will do harm, and help others to find the broken cases easily. However, surely, Go programmers, in their spare time, can help the proposal's supporters to do this. But, again, it is strange that the proposal has been accepted before such attempts are made. :D, so weird.
And, please read my tweets, there are several factors which are unrelated to finding broken code. In fact, the change will lead to more error-prone, instead of less error-prone.
- dependency hell..
Recently had to update unmaintained project by 5 dependency versions higher. It was so painful with GO.
As for upgrading old libraries which might have broken backwards compatibility, I can't offer much.
Interestingly, commiting vendored files is a common point of contention between devs. Some argue that it adds bloat to the project repo.
Unless I'm working with TB sized monorepos, I always vendor and commit, because hardware is cheap and my time valuable. I have 10 year old projects that build just fine without internet connection because all deps are commited with the project.
As a bonus I get a free diff of every single line that changed when a depency is updated.
Oh wait... You mean because of breaking language changes?
It is unlikely that software will fix a bumbling human. No matter how hard it tries, humans will always find some new way to do something stupid.
Logging always felt like something that (1) almost every project needs, and (2) there are too many 3rd party packages, with no "de facto" choice.
This is particularly painful when working with 3rd party library code. There's a high chance that the consumer of library code uses a different logging package to the library author.
Having structured logging in the stdlib is fantastic, because now there is a "go way" to this that libraries can use.
1 "We expect existing logging packages to coexist with this one for the foreseeable future."
2 "We expect that this proposal’s handlers will be implemented for all popular logging formats and network protocols, and that every common logging framework will provide a shim from their own backend to a handler"
Group is pretty much https://pkg.go.dev/go.uber.org/zap#Namespace
Record is the interface to log sinks, not something the typical programmer worries about.
What Go structured logging libraries predate Zap's 2016 creation? Only one I can remember is Logrus, which was using type Fields map[string]interface{}, the bad qualities of which are kinda the whole reason for Zap's creation, and slog follows the Zap-style API[1].
[1]: Though ignoring many of the optimizations..
When skimming it I mostly saw the common API structure that I usually see in libraries / write myself when I need a logger, so I generally welcome the standardization.
- Logger -- created by New, accepting a Handler, providing a fixed set of level-based logging methods, asserting concepts of Attrs and Groups, not parameterizable by consumers
- Handler -- with two default implementations, asserting concepts of Attrs Groups and Records, parameterizable by consumers as long as they follow the semantics defined by the interface
- Attr -- arbitrary concept that maps to a k/v pair in a log record
- Group -- arbitrary concept that namespaces a set of k/v pairs in a log record
- Record -- arbitrary concept that requires a timestamp (expensive to compute), a level (one of a specifically defined enum which cannot be changed), and a PC stack pointer (obvious issues there)
I've never seen a logging package which meets these requirements.
To me it absolutely makes sense as the default and standard for 99% of applications, and the API isn't much unlike something like Zap[0] (a popular Go structured logger).
The attributes aren't an "arbitrary" concept, they're a completely normal concept for structured loggers. Groups are maybe less standard, but reasonable nevertheless. The timestamp is not required - the documentation specifies it can be left as the zero value and shall be ignored in that case.
I'm not sure if you're aware that this is specifically a structured logging package. There already is a "simple" logging package[1] in the stdlib, has been there for ages, and isn't particularly fast either to my knowledge. If you want really fast you take a library (which would also make sure to optimize allocations heavily).
The domain concepts defined by slog are unquestionably abnormal.
From a structured logging package I would expect a far simpler Logger API with a Log method something like
Log(pairs ...KeyValuePair) error
Log(r Record) error
So s/KeyValuePair/Attr/g?
Re varargs vs record, varargs are usually used in Go because they're very ergonomic to write inline, so they work very nice with loggers.
Group sounds like a fairly arbitrary concept, I agree, but still very reasonable for something that's supposed to standardize things. It would totally make sense for e.g. different libraries to each inject their own group with key-values into the context.
I'm not sure what's the problem with Handlers? This is supposed to be a standard library package that is setting... well, standards that other libraries will adhere to. Handlers let you plug your own output formats.
The package is not supposed to be "as bare bones as possible". Just "good default others will be able to integrate and compose with".
I've used my fair share of structured loggers too, and this really is par for the course and a completely reasonable set of things to include in a structured logging package.
---
It's worth noting that the previously-mentioned Zap, which is probably the most popular structured logging library in go right now, contains all the same concepts, just differently named:
Attr -> Field
Group -> Namespace
Record -> Entry
Handler -> Encoder/Writer
There is no reason to distinguish an Attr or Group or Record from a (set of) KeyValuePairs. They're all the same thing.
And I don't agree that Zap is the most popular structured logging library. It's one of many, none are the clear winner.
But, whatever.
A Handler is an abstract type, a Logger is a concrete one. A Logger lets you have a large user surface (and without virtual calls); and Handler, a small implementer surface. (The handler is not much beyond what you proposed, plus an optimization to avoid materializing complete record+KV pairs if the level isn't met.)
The concepts also map fairly directly to logrus and zerolog (which has an absolutely enormous surface to avoid boxing).
A Handler is something that transforms log events to concrete outputs (files, disks, writes to stderr, etc.) via side effects.
A Logger is what programs use to produce, transform, etc. log events.
Both are abstract interfaces with arbitrarily many possible implementations. Both are defined in terms of their user-facing capabilities. I'm not sure how those definitions would differ. Both accept log events and do something with them. That interface is the same.
More specifically, users want the surface area Logger has. They do not want a single function with a complex object specification, just like they don't want a function per type.
What is a record context? What is a level? These are concepts built on top of a structured logger, which manifest as specific key=val pairs in a given structured log event. They don't need -- and shouldn't have! -- first-order representation in a low-level structured logger API.
What is an entry point? If I'm writing some structured logger implementation, I expect that I should need to provide precisely one method:
func (x *MyThing) Log(<set of key=val pairs>) error
From my perspective, the problem with the KeyValuePairs API is that it’s inescapably slow and allocation-heavy. I’m glad that a logging API built into the standard library is usable in more performance-sensitive contexts. It’s easy enough to wrap it up in a KeyValuePairs-style API if you’d prefer that, and you’ll now have the ability to unwrap your logger and interop with other libraries that expect the standard slog interface.