More powerful Go execution traces
go.dev
go.dev
The ecosystem as a whole decided that errors should be useful, values, and properly bubbled up - and that your application should not be so hard to reason about that you need a stack trace to know where in the deep recesses of your code something went wrong.
I love stack traces in Java, but I don't miss them at all in Go.
As it stands though, the consensus is that it doesn't add enough value, it's too expensive (cost/benefit), errors are normal program flow, and 3rd party libraries can add the functionality if one decides they need/want it in their application.
Curious how Go is better than other languages in this regard.
Errors are just values. They don't have any special meaning nor is there any special ceremony to create them. A panic must start from calling panic(); there's no function or special statement to create or return an error.
It might be possible to augment every `error`-typed return such that the concrete type is replaced by some synthetic type with a stack trace included. However, this would only work because `error` is an interface, so any function returning a concrete-typed error wouldn't be eligible for this; it would also add a lot of runtime overhead (including, at the very least, a check for nil on every return); and it would cause issues with code that still does == comparisons on errors.
On the whole, I think error-wrapping solves the problem of tracing an error well enough as the language exists today. If errors are going to start having some magic added to them, I think the entirety of error-handling deserves a rethink (which may be due, to be fair).
Don't feel bad, I've tried to do this in some places, but I'm not sure it's worth it. It adds a ton of boilerplate to Go's already verbose error handling, since you need to wrap every error that gets returned from libraries.
There are probably lots of situations where it's worth it, though.
A crash, sure, that might need a stacktrace to debug. But that's already in place.
https://docs.oracle.com/javacomponents/jmc-5-4/jfr-runtime-g...
More awesome work from the Go team. Thanks!
As one of the people who worked on the optimizations mentioned in the article, I'm probably biased, but I think you can expect those claims to hold outside of artificially pathological workloads :).
We're using execution tracing in our very large fleet of Go services at Datadog, and so far I've not seen any of our real workloads exceed 1% overhead from execution tracing.
In fact, we're confident enough that we built our goroutine timeline feature on top of execution traces, and it's enabled for all of our profiling customers by default nowdays as well [1].
[1] https://blog.felixge.de/debug-go-request-latency-with-datado...
In general, Go tooling makes up for a lot of the intrinsic issues with the language, and even shines as best in class in some cases (like coverage, and perhaps tracing too). All out of the box, and improving with every release.
If you don't like Go then fine, no problem. Everyone dislikes some things. No hard feelings. But why comment here? What's the value of just stinking up threads like this with all these bitterly salty comments that barely engage with what's being said? Your comments alone consist 13% of this entire thread, and more if we could all the pointless "discussions" you started. It's profoundly unpleasant behaviour.
The commenter said more than that.
"In general, Go tooling makes up for a lot of the intrinsic issues with the language, and even shines as best in class in some cases (like coverage, and perhaps tracing too). All out of the box, and improving with every release."
Best in class?!?
That's "let's run the race detector and get some coffee" overhead, not "let's leave it on in dev" overhead.
Still cool that they have it available!
I do think most gophers, instead of tests, use a combination of prayer and automatically restarting crashed processes when they inevitably panic from a race, which seems to work better than you'd expect!
It represents the flow of data between channels/threads as a sequence diagram using PlantUML. Today I implemented core.async/onto-chan! (it spins a thread that will take items from a sequence and put them onto a channel). Here's what it looks like:
https://pasteboard.co/VhvDroREOvOQ.png
It's especially useful in Clojure, as the experience with channels is not as polished as in Go (or so I heard): when a put or a take hangs, your only recourse is to sit and squint at your code. This tool will color them red, so you can immediately spot what's wrong. It also allowed me to spot a channel spaghetti plate I had not anticipated and wouldn't have remarked otherwise.
For now I've taken a "maximalist" approach which includes inspecting buffers, so it incurs a heavy runtime penalty that's only ok at dev-time (+ it keep tracks of the whole trace, so no flight recording in sight for now).
In the future I'd like to give it a proper interface (it's just a svg file with hoverable elements for now), maybe using constraint-based graph layout to lay out the sequence diagram using cola.js/adaptagrams or the sequence diagram from Kiel university Kieler project when it's integrated into the Eclipse Layout Toolkit. Thesis from the developer of this module:
https://web.archive.org/web/20220519080528id_/https://macau....
Say what you will about the language itself, the Go stdlib is one of the most useful and pleasant standard libraries I've ever worked with. It gets better with every release, and great care is taken to make it featureful, performant and well documented. These days for most general things I rarely have the need to reach for an external package.
Agree 100 % ! And it gets better as time evolves.
For example the recent `log/slog` introduction to stlib kills the need for 99.9999% of people to use third-party logging, because structured logging is now in stlib.
Same with the new http mux. Many people will now be able to migrate over to stdlib because of the richer functionality, and only the small community of outliers who are doing stuff like regex mux parsing will need the third-party libs, but with time no doubt regex mux will make its way to stdlib too.
I do think the situation is better than in Python and overall not bad after 12 years, but it's not as clean and minimalist as it could be.
- context.Context is two different barnacles. The first is the thing itself; do I "need" context here? Probably should add it just to be safe. The second is the myriad of older libraries that don't use it or have tacked it on with a duplicate set of functions/methods. Library authors seem to not understand the point of major versions or else are afraid to pull that trigger.
- http.Client is a barnacle. First, there's the fact that you have to close the response.Body even if you don't use it. Then there's the difficulty of adding middleware or high-level configuration to it. Then there's the fact that they broke their own rules for context.Context and instead of taking it as a parameter in the methods, it's stored as part of a struct. You can work around many of these problems for your own code, but not when using third-party libraries.
- The entire log package is a barnacle. Thankfully we now have log/slog but log existed for so long that lots of third-party libraries use it. Even if the library authors knew to avoid the footguns that are Fatal/Fatalf/Fatalln and Panic/Panicf/Panicln, context and level support are spotty at best.
- The (crypto/rand).Read and (math/rand).Read debacle. Really, the entire math/rand package is a barnacle. Thankfully this has also been addressed with math/rand/v2 but the old one will live on for compatibility.
Heh. I wrote some code that called log.Fatalf with an error message, and then later added a verbose switch when running the program (kind of an `if !verbose { log.SetOutput(os.DevNull) }` type of thing).
Sure enough, when I ran it with some input that led to run into the log.Fatalf, the program exited, but no error message was written out. Cue quite some head-scratching!
Anyway, using the term "execution tracer" in Go goes back to the initial design doc from 2014: https://docs.google.com/document/u/1/d/1FP5apqzBgr7ahCCgFO-y...
https://www.infoq.com/articles/debugging-go-programs-pprof-t...
Many other language use the same technique, see Python with: https://github.com/bloomberg/memray
When you add a additional term like "execution" you are specializing the term to mean a event stream of "execution". In areas I am familiar with, that would normally mean a trace of the complete execution of the program down to the instruction-level so you can trace the precise "execution" of the program through the code.
What is described here would, in the terminology I am familiar with, be more like... a thread status and system event trace just applied to goroutines and the Go runtime, respectively, I guess? It does also include the stack trace at the time of the event, so it does have more data than that, but that is qualitatively different than a instruction execution trace that allows you to trace the exact sequence of execution of your program.
- testing
- benchmark
- profiling
- cross compilation ( that work from any platform, like compiling a windows .exe from your raspberry pi for example )
- some linting
- documentation
- package mgmt
- bug report
- code generation
- etc...
Java is probably more advanced in some fields ( like tracing / profiling ) but it lacks others.
> cross compilation ( that work from any platform, like compiling a windows .exe from your raspberry pi for example )
This, I think is one of Go's best selling point.
The only thing one could arguably argue that Go does better is value types, but even that requires careful coding so that escape analysis is triggered, and in that sense, it is only a matter to use a JVM implementation like GraalVM, OpenJ9, Android ART, PTC, Aicas, Azul.
Coming from ts, tooling like tsconfig has a lot of options, but sensible defaults can be set with a single flag like strict mode. If some org has some specialized needs, they can dive into the configuration and get what they need.
With golang, not only would it be a lot for any single team to offer all those features at a decent level of polish, the golang culture in particular is very, very resistant to small bits of comfort because of dogma like "worse is better". It's kind of similar to Haskell's "avoid success at all cost".
The commonJS/module transition is a nightmare. The fact that something like 'prisma' exists - a c-written 'query engine' that turns prisma js into sql.. wut?
This ecosystem is on a highway to hell literally.. I really hope Bun works out, because I do like Typescript, I do like programming in it - but I'm absolutely done with spending hours upon hours figuring out configs in tsconfig, jest, package.json eslintrc, prettier, vstest and whatever the next 'new' abstraction is. In Go I can just focus on the code and forget about the rest
Also there’s a lot of go tooling that doesn’t come from go team itself because go standard library exposes first class code introspection utilities. go vet is an example of this
You shouldn't need anything (except strict mode) in a new TS project.
That said, TS is quickly becoming the antithesis of what Go tries; every release for the past few years I've seen now are all features where I'm like, "I will never need or use this". Some conveniences have been improved on - like better type inference - but that will mainly allow for cleaning up workarounds.
"some orgs with specialised needs" are going to be legacy projects, either older TS versions that didn't have the features that it does now, or JS codebases. This isn't generally an issue with Go projects, most of which are greenfield. That said, there's some X to Go conversion projects that produce less than ideal code though, like usage of the `interface{}` type that at this point is a code smell.
For years until 1.19 the Go GC has had only one tuning parameter (GOGC).
You really, truly, just run.
As for Go, good luck doing just run when a code repo breaks all those URL hardcoded in source code.
* the go module proxy ensures repos are never deleted, so everything continues to work
* changing to a new dep is as easy as either a replace directive or just find and replace the old url
If a module author deletes or relocates their module, the old module is not deleted or renamed from the module proxy. It is kept around forever. Code that depends on it to not break does not break.
If they relocated and you want to update to the new location, perhaps for bug fixes, then you do have to do a bit extra work (a find and replace, or a module level replace directive) but it’s a rare event and generally a minor effort, in my opinion, so I don’t think this is a significant flaw.
For most users most of the time they don’t need to think about this at all.
You're on a tear in this thread being wrong about how Go works, but I'm really curious what extremely specific series of events you're imagining would have to happen to lead to this outcome. If I use a dependency, it gets saved in the module proxy, I can also vendor it. You would have to, as a maintainer, deliberately try to screw over anybody using your library to accomplish what you describe.
Being a mantainer has nothing to do with some kind of gentlemens code of condut.
This sounds like something you would hear 10 years ago in relation to the CMS garbage collector. Since Java 9, G1 has been the default gc for multi core workloads and is self-tuning. The CMS gc was removed 4 years ago. If you do need to tune G1, the primary knob is target pause time. While other knobs exist, you should think carefully before using them.
We run all of our workloads with vanilla settings.
Additionally, of course it would be nice to have more GC options, such as choosing between a throughput-oriented GC design and a latency-oriented one, having the option of a compacting GC of some kind to avoid fragmentation, or even having a real time GC option.
Go has chosen a very old-fashioned GC design with very few tuning parameters possible, but even so it only exposes a very basic form of control.
Also, the OpenJDK JVM supports live debugging and code hotswap, going so far as to de-optimize code you're debugging on the fly to make it readable. Go doesn't support live code reload even in local builds.
The current format has very limited tooling; "go tool" has some extremely rudimentary visualization. There is gotraceui [1], which is much better, though you need to use Go trace regions to get much useful context.
There's a proposal to support Perfetto [2], but I don't know if anything has come of it.
Here's to the continued success of Go and other sanely-designed languages.
We detached this subthread from https://news.ycombinator.com/item?id=39710822.
I love Go, used it since the 1.0 release and have used it at work for years. No complaints about the language. But for the longest time Go didn’t have quality dependency management and pulling in dependencies was annoying. Building your programs with the large, high quality stdlib was the path of least resistance.
Since Go 1.0 (2012) most new languages have coalesced on good dependency management. Rust, for example, copied Ruby’s bundler idioms since before 1.0 (2015). People who needed good quality libraries were able to pull them in with minimal hassle. That’s why they didn’t need a large standard library to succeed. I’ve written more about the tradeoff here - https://blog.nindalf.com/posts/rust-stdlib/
To you, I’d suggest using less charged language like sanity. In this case there’s a good technical case for both ways, and it’s not productive or nice to imply that people who choose a different path from you are insane.
To the grandparent, I'd suggest keeping the language as it is. Status quo in the current programming ecosystem is often insane, and when it is, we should call it out.
> it’s not productive or nice to imply that people who choose a different path from you are insane.
In a general sense, I agree. However, when it comes to programming, insanity is so widely accepted as normality that I think it would be counterproductive to blunt the language used to describe it.
As an example of insanity for those who may (justifiably) think I'm talking out of my bottom - in NodeJS ecosystem the standard library is so bad that it is considered acceptable to use dependencies for even the simplest tasks, which caused the `leftpad` incident which broke countless programs.
Leftpad could've just as well be a part of some standard library, like...
> in terms of convenient String routines, Go is still a bit inferior to Python
... Go strings library :) https://pkg.go.dev/strings
Standard libraries can pick their scope of functionalities and depth (or completeness) for each functionality. Nowadays every programming language is expected to come with a good string support, which is about the scope. But there are a lot of string operations you can support. PHP for example has `soundex` and `metaphone` functions for computing a phonetic comparison key. Should other languages support them as well? I don't think so, and that's about the depth or completeness because you can never support 100% of use cases with standard libraries alone. Ideally you want to cover (say) ~90% of use cases with a minimal number of routines.
Leftpad was clearly due to the lack of depth in JavaScript and Node.js standard libraries. JavaScript now has `String.prototype.padStart`, and an apparent name difference suggests a good reason that some standard library may want to avoid it: internationalization is complex. A common use case is to make it aligned by padding space characters, but that obviously breaks for many non-Latin scripts [1]. And yet many people tried to use it, so `left-pad` was born with a suboptimal interface, and we know the rest.
HTTP support is different. I totally agree that HTTP is something you want to support in a sorta native fashion, but a standard library is not the only way to do that. In fact it is not a good place to do that because it is generally slower to change (Go is a notable exception but only because its core team is very well funded and strongly minded). Python did support HTTP in its standard library for a long time, but it doesn't support HTTP/2 or HTTP/3 at all and Requests or urllib3 are now the de facto standard in Python HTTP handling. Modern languages try to balance this issue by having a non-standard but directly curated set of libraries. Rust `regex` is a good example, which may be surprising given than even C++ has a native regex support. But by not being a part of the standard library, `regex` was able to move much faster and leverage a vast array of other Rust libraries, and it is now one of the best `regex` libraries throughout all languages. That's what nindalf wanted to say by different "ways".
[1] For example, `"한글".padStart(5)` will give you `" 한글"` but its visual width is larger than 5 "normal" characters. This is not merely a matter of fonts and Unicode has a dedicated database for the character width ("East Asian Width"). Some characters are still ambiguous even in this situation (e.g. ↑), and the correct answer with respect to internationalization would be: don't, use a markup language or (for terminals) a cursor movement instead.
The only downside of Rust is that you have to pull more than 100 dependencies to build a simple hello world server (Tokio + Axum | Actix), then you need a database driver, most likely something that depends on SQLx (not sure here how many, dependencies), but one of my projects got near 500 transitive dependencies with only 20~ish direct dependencies.
By the way: You still need a DB driver with Go. They just provide an interface, which is cumbersome and error prone to use. That's why the community package sqlx exists.
To create a hello world in Rust, one just needs to add two crates, which in terms pull many other crates. So yes, too many total deps for my taste, but an Axum server is much more feature full than a server build with Go's std lib.
This doesn't sound extreme. The statement, "designed with sound judgement," doesn't carry the implications that you're defending from.
Too many people on HN when discussing about programming languages unfortunately…
There’s also the philosophy that the language core should be as minimal as possible. I’m not 100% sold on it, but there’s definitely a valid argument to be made in favor of it.
Yes, I very much agree. The comment I was replying to however was referring to “other languages”.
- The csv package. Compare Go's csv package with Rust csv crate. The package in Go's std lib is close to useless compared to the Rust community crate.
- Logging. No tracing, limited providers, just recently added structured logging.
- The xml package is very slow and cumbersome to use compared to modern element-tree-like implementation. For larger docs you probably need to resolve to a community package.
- If I'm not mistaken, Java has a build in http server, which usage is probably very low. Instead people are using Jetty / Netty / Tomcat (? my Java times are long ago)
To repeat myself: I personally like the std lib approach of Go. I disagree to any narrow view on this topic.
Go is following not leading.
To this day ISO C and ISO C++ still don't include TCP and UDP on their standard library, that comes from POSIX, the UNIX APIs that didn't make it into neither ISO C nor ISO C++.
- Direct memory access through `unsafe`. Python kinda does through `ctypes`?
- Tightly integrated assembly language; just pop Go-flavored assembly into your package and you can link directly to it.
- Statically-compiled code with AOT; no bytecode, no interpreters, no JITs.
Therefore what really sets Go apart is that it gives you all of these rich standard library capabilities in a relatively lower level language.
Of course I kind of understand why Rust and C++ don't put e.g. a PNG decoder in the standard library, I think this is somewhat an issue of mentality; those are things that are firmly the job of libraries. But honestly, I wish they did. It's not like the existence of things in the standard library prevents anyone from writing their own versions (It doesn't stop people from doing so in Go, after all), but when done well it provides a ton of value. I think nobody would claim that Go's flags library is the best CLI flag parsing library available. In fact, it's basically just... fine. But, on the other hand, it's certainly good enough for the vast majority of small utilities, meaning that you can just use it whenever you need something like that, and that's amazing. I would love to have that in Rust and C++.
And after experiencing OpenSSL yet again just recently, I can say with certainty that I'd love Go's crypto and TLS stack just about everywhere else, too. (Or at least something comparable, and in fairness, the rustls API looks totally fine.)
Already available in ESPOL and NEWP (1961), Modula-2 (1978), Ada (1983), Oberon (1987), Modula-3 (1988), Oberon-2 (1991), C# (2001), D (2001) and plenty others I won't bother to list.
> Tightly integrated assembly language; just pop Go-flavored assembly into your package and you can link directly to it.
Almost every compiler toolchain has similar capabilities
- Statically-compiled code with AOT; no bytecode, no interpreters, no JITs.
Like most compiled languages since FORTRAN.
Except in reality in does not work, like you can't easily create a single binary out of most C/C++ project.
You always going to fight with make / GCC / llvm and other awful tools with errors that no one understand. It does not matter if the underneath tool / language is supposed to support it, can a developer make it work effortless or not.
In Go you download any repo type go build . and it just works. I can download a multi millions line repo like Kubernetes and it's going to work.
If you believe that regarding Kubernetes you're in for a surprise regarding reproducible container builds.
> Like most compiled languages since FORTRAN.
Yes. But you didn't list "compiled languages since FORTRAN", you listed:
> Python (1991), Java (1995), .NET (2001), Smalltalk (1972), Common Lisp (1984), Ruby (1995), Perl (1993).
"Go's standard library is a shining example of what all standard libraries should strive for. Yet, we still have some languages who's developers refuse to include even a basic http API in their standard libraries in an age where even embedded systems have started to speak http. Imagine if if the same had happened with TCP and UDP...
Here's to the continued success of Go and other sanely-designed languages."
You then moved the goal posts by talking about stuff that wasn't in that comment.
As such I am also allowed to move my goal, mentioning that
"Already available in ESPOL and NEWP (1961), Modula-2 (1978), Ada (1983), Oberon (1987), Modula-3 (1988), Oberon-2 (1991), C# (2001), D (2001) and plenty others I won't bother to list."
Are all languages that compile to native code.
"Ah but what about C#?!?", it has had NGEN since day one, Mono/Xamarin toolchain has supported AOT since ages, Windows 8 Store Apps used MDIL toolchain from Singularity, replaced by .NET Native for Windows 10 store apps, Unity compiles to native via their IL2CPP toolchain, and nowadays we have Native AOT as well.
And I will had that Java has had native AOT compilers since around 2000, even if only available as commercial products, with Excelsior JET, Aicas, Aonix, Webspehre Real Time, PTC, and unsafe package as well, even if not enjoying an official supported state (nowadays replaced with Panama for exactly the same unsafe kind of stuff and low level system accesses)
Programming languages have been largely just shuffling around features other languages have for the last 50 years now, and I can only go back that far because when you get back to the very first languages, they're unique and first by default. Even when a language is first to do something, it's generally only the first for a feature or two, because how would anyone even make a programming language that was almost entirely made out of new things anymore? Even if someone produced it, who would or could use it?
You seem to spend a lot of time upset about claims nobody is actually making.
On a similar note, iPhone did not invent cameras, MP3 players, cellular broadband modems, touchscreens, slide to unlock, or applications.
Regarding C++, it’s based on a standard, the situation is a bit different. You have a variety of implementations. Imposing anything beyond the typical use case the compiler implementations have catered to would induce an insurmountable overhead on the implementation coherence. Therefore, I believe it’s more reasonable to have application-level logic in the realm of community maintained libraries. In addition, C++ is huge syntactically and the stdlib is immense, but it’s more focused on algorithmic tasks rather than on quickly building mini-servers.
Besides, the Go community has a myriad of reinvented wheels, ranging from logging over caching to maps, and until recently HTTP server libraries. The logging story for example has just recently led to a discovery of the patterns desired by the community, mature enough to accommodate structures logging in Go’s stdlib. Similarly for error handling. Robust and settled approaches turned a de facto standard make total sense to be included.
.NET is different again, having a center of gravity with Microsoft and the .NET Foundation, with typically one preferred library for a given task, contrary to Java and Go. Centralized and decentralized, the classical dichotomy.