Comment by userbinator | Hacker News Reader

Comment by userbinator | Hacker News Reader

The purpose of this data structure is to enable the Go runtime system to produce descriptive stack traces upon a crash or upon internal requests via the runtime.GetStack API.

In other words, the Go team decided to make executable files larger to save up on initialization time.

Something about this whole thing just seems wrong. How often does (perhaps should) an application crash? How often does (again, perhaps should) it need to retrieve its own stack? ...and how much of the binary is being taken up just for that purpose?

Of size/performance trade-offs and use cases

Why is startup time even the question when the common-sense approach is to simply compress this rarely-used table and decompress it upon the first time it's used, not upon every startup? That's assuming it is always absolutely necessary to have in the first place, since loading a huge executable isn't going to be fast anyway.

I feel like this is a case of "the tail wagging the gopher".

Eh, it's a bit of a simplification I think. Certainly the pclntab is consulted in more situations than just application crash; for example, when logging, you can have a source line prepended to the beginning of the line, which certainly uses the pclntab. I would be pretty surprised if there weren't a lot of other cases where the pclntab is consulted.

for example, when logging, you can have a source line prepended to the beginning of the line, which certainly uses the pclntab

It needs to go through the table to find which source line corresponds to the current instruction pointer? That's the only reason I can see for needing it, and a very roundabout way of getting information which is known at compile-time and could be simply a constant whereever it's used, much like C has __FILE__ and __LINE__.

That's not enough for the stack traces. A crash in Go is way more informative than failing an assert in C. Whether the extra memory is worth it is a separate questions.

> for example, when logging, you can have a source line prepended to the beginning of the line, which certainly uses the pclntab.

Naive question: Wouldn't that be trivial to hard code at compilation time within the logging code? Why would it need to go look that up from the pclntab?

In C you have the preprocessor that can be used to hardcode that. In Go plenty of things already rely on the pclntab and doing lookups is cheap so you just use the pclntab. Go is definitely a less kludgy language for being able to drop the need for a preprocessor, despite the consequences.

I don't get this argument, or rather, I don't the apparent corollary that needing pclntab is somehow connected to not using a preprocessor.

I'm probably missing your point, since one certainly don't need a preprocessor for line numbers in logging statements. Unless the language - like C - actually implements the include/import mechanism through a preprocessor. In which case the need for the dependency becomes rather self evident.

>I'm probably missing your point, since one certainly don't need a preprocessor for line numbers in logging statements.

So, libc doesn't define anything like pclntab. About the closest you can really get is DWARF2 call frame information, which is generally treated as debug information (because, well, it _is_) and stripped in release builds. Further, not all platforms will use DWARF2, and the way it is embedded in binaries differs.

(Example: In Win32, when you are using MinGW, DWARF2 will be embedded as PE sections, which have 8 character long names; this is not long enough for the DWARF section names, so a special PE extension is used to specify the longer section names.)

What i'm trying to illustrate is, it's really, truly not possible to go from a PC value, to a line number in a source file, in pure C, at runtime. However, of course, as you have no doubt noticed, everyone still manages to print out line numbers in C source code. And they do this using the preprocessor.

If you dig deep enough into pretty much any logging library that offers line numbers in C and C++, you will find a macro that passes through __FILE__ and __LINE__. And if you keep tracing, you will probably also find the 'function' you call to log is a macro that eventually calls this macro. As a quick example, here is one in glog, a logging library used at Google:

https://github.com/google/glog/blob/41f4bf9cbc3e8995d628b459...

Pantheios is another popular logging library, and it defines macros that can be used to control the prefix.

https://github.com/synesissoftware/Pantheios/blob/177dc5fcff...

If you look into your logging library, it is quite likely that you will find that at the end of the day, it boils down to the __FILE__ and __LINE__ macros. When all of the macros are computed for your log line, the preprocessor subs in a string literal and line number for the filename and the line number that the macro was originally invoked from.

An uncompressed table in the binary can be demand paged when needed instead of dirtying pages.

Yes, the binary will be bigger meaning that more IO will incur when deploying the application e.g. in docker/k8s.

What that extra IO is surely an annoying thing, I'm not sure I'd trade that off for a huge increase in heap allocation right before getting a stacktrace. Since we're talking about containers, it's not uncommon for containers to be configured with tight memory limits and having an OOM during that process could mask the real reason for a panic.

Would it be possible to have a more succinct data structure or even directly index the compressed table without having to decompress the whole thing?

This feels an awful lot like stripping, can someone explain why need this? Logging is mentioned below but that can be solved in other means. I really have no clue how Go uses pclntab, but it does sound like you could serve that separately like is often done with debug symbols, especially if you offer "one-true-binary" to customers.

It is not a practice I'm found of while debugging, and it also sounds like that table could be be swapped out most of the time.

Go actually walks its stack very, very frequently during calls to runtime.morestack, which happens whenever a routine needs more than the default stack size.

This does not need pclntab.

Reply on news.ycombinator.com