I mean, it makes all sense while you're developing and testing, but it should be reasonably possible to strip it out from production build binaries, and instead put it in a separate file so that if you do get a crash with a stack trace, then some external script can transform the program counters to line numbers, not have it embedded in every deployed binary.
The reason is that Go's standard library provides functions that allow retrieving a backtrace and symbolicating that backtrace at runtime:
* https://golang.org/pkg/runtime/?m=all#Callers
* https://golang.org/pkg/runtime/?m=all#CallersFrames
Unlike in C or C++, where you can link in something like libbacktrace [0] to get a best-effort backtrace, those Go functions are guaranteed to be correct, even when functions have been inlined. This is no small feat, and indeed programs compiled with gccgo will often be incorrect because libbacktrace doesn't always get things right when functions have been inlined.
Unlike in C, there are no macros like __FILE__ and __LINE__, so there is no alternative to runtime.Callers (short of preprocessing your Go source code).
Why couldn't the Go team introduce a flag that strips symbols, while making clear to people that they should only use it if they are okay with backtraces looking like
#1 0x00007ffff7ddb899 in __GI_abort () at abort.c:79
#2 0x0000555555555156 in ?? ()
#3 0x0000555555555168 in ?? ()
#4 0x000055555555517d in ?? ()
#5 0x0000555555555192 in ?? ()
In every project I’ve been on, I was asked to ramp up in the first few weeks by fixing small bugs here and there in the core codebase, not by setting up CI/makefiles/build infra/etc...
I actually think I might be totally missing your point, since I don’t really understand what you mean or how you arrived at the conclusion that it’s a sign of dysfunction.
But also this entire section could be compressed and seekable and only expanded on demand. No need to leave it as a bloated uncompressed blob.
If stripped binaries don't work for you, nobody is saying you should be forced to use them. You would presumably just not avail yourself of the option.
(As an aside -- what's wrong with your symbolification infrastructure at work? This is something I spent significant time working on in a past job, so I'm always curious how other people are doing it).
Specifics? The automation to associate cores with specific builds is bad or non-existent; the automation to load up GDB with the right files and corresponding sources or source branch is bad/non-existent; the fileserver storing symbol data is slow and sometimes remote to the developer across a very thin pipe; etc, etc. All to some extent foot-shooting by my organization. But we have TBs and TBs of build artifacts and hundreds of unique daemons producing cores and also have to debug kernel cores and multiple product branches.
Part of the pain is perhaps that we're cross-building for an embedded FreeBSD-derived system, but the majority of our developers use Linux or Mac, so we can't necessarily use host-native (or host-native-only) tools.
I think basically the situation is begging for someone familiar with the problem-space to sit down and bang out a FUSE filesystem or two (e.g. fetching sources on-demand as GDB reads them, instead of checking out a full multi-GB repo copy) along with a shell or Python script to load up a core. But no one has done it yet and management doesn't really prioritize developer tools.
.NET has AOT support since ever via NGEN, followed by Mono AOT, Xamarin,IL2CPP, Bartok, .NET Native and a couple of less know projects.
On Java side AOT has been always supported by JVMs for embedded deployments like Aonix (now part of PTC, PTC, Aicas, ExcelsiorJET, IBM J9.
And nowadays there is SubstrateVM as well, renamed recently as Graal Native Image, part of OpenJDK since Java 11.
Yes, Java is interpreted and JITed, but on the bytecode level. When compiling sources to bytecode, line number information is written to class files.
I get the point about external symbols and location database, but oftentimes time is precious and having fully laid out stack trace in the log will allow you to get to the root much faster.
You can also set up a service that automatically symbolicates everything in a log file as soon as it is generated, before a human ever even looks at it.
Granted, yes, this is slightly more complicated, but the point is that the toolchain should let the developers choose which strategy they want to use.
Absolutely not. Developers making choices about stuff like that is asking for troubles. I'm an SRE - and this is how production issues start, because some dev decided that somehow, in production, something like this should not be there, until problems pop up, but then it's too late.
For us it's simple, the moment it hits the CI/CD pipeline, it's a production build, even if it happens just end up on some test or staging environment. If you possibly need it there, you need it on production. Our way of working means that the exact same build artifact should be promotable to production.
Of course this is partially out of necessity; Windows is proprietary software, so they don't want to give you full debug info. But then in practice their tooling is just so vastly superior. You can fire up WinDbg, a rock stable debugger, attach to a random process and get a proper backtrace including full symbols for all the proprietary Windows stuff, because they run a public symbol server and all the symbol data you need for your Windows build is downloaded in a few seconds, fully transparently. And they ship the same symbol server with their tooling so you can run it for your own binaries, too. You can't do that on any Linux distro without starting to manually install random -dbg packages.
(And don't get me started on trying to debug crashed processes on Linux. A big reason Android has their custom libc is so that they can install default signal handlers for things like SIGSEGV, where on a normal Linux system that process goes immediately to core dump and is essentially fucked for a debugger wanting to look at its state.)
Then your production binary will give you stack traces like [0x12345, 0xabcde, ...], but you can use the separately-stored files to symbolicate them and get the source file/line info.
Not sure if this is possible on all platforms but it at least is for all combinations of {C, C++, Objective-C, Rust} and { Linux, macOS, iOS } .
And if that added operational complexity is not worth the size savings, you can freely choose not to do it, and things will work like they do in Go.
Go's insistance not just reinventing the wheel but on actively ignoring core infrastructure improvements made in the last 20 years is bizarre.
Here is an example from my past. As embedded programmer I went and added manually a hundred lines of constants, which initially were just an array generated at start and increased that code by about 5%. Why? Because I gained 5 ms in execution speed. And in embedded world that's huge. Especially when your code is executed on the lowest 10 ms cycle. So the department head approved such a huge change because the code size doesn't matter, you can always buy a bigger chip, but if your car don't start because the watchdog keeps resetting your car's on-board computer then speed in code execution is everything.
No.