Hacker News Reader

Comment by asveikau | Hacker News Reader

Using sbrk(2) as an application stack seems like a bad idea. A lot of times that is where malloc gets its heap addresses from.

The article even says this but not strongly enough:

> It is worth mentioning that the C library malloc might be using and caching the results of sbrk it may be unwise to mix this trick with your C library malloc, but it is very useful in new programs that you have no intention of using the C library malloc with.

No it is not a good idea to write "new programs" to apply hacky uses of sbrk(2) and conflict with malloc. Any use of sbrk(2) outside of a malloc implementation is pretty suspect as code smell. And you will be hard pressed to find any program or library that doesn't use malloc, short of embedded use cases.

View on HN

Not only malloc, but the entire libc, as no functions in it explicitly guarantee that they do not call malloc and/or mess “your” brk range. You can guess it by what it does and ENOMEM in possible return status, but that is a stupid idea. sbrk is basically a remnant api and is now either an implementation detail of libc memory management or a fool’s crutch [1].

[1] https://opensource.apple.com/source/Libc/Libc-1272.250.1/emu...

Yes. Notably stdio typically uses the heap. So I hope this author isn't planning to use printf for New Programs.

I don't, at least not when I care about performance.

stdio is a big unknown with highly variable performance across platforms and libc-versions.

Sure, yeah but it’s not like using printf is a problem for error messages or logging. Using write() for that kind of stuff is more hassle than it’s worth.

Oh it so is!

I can’t count the number of times I’ve gotten 20-50x speed ups just turning off logging. Assuming printf is free (or even cheap) is the sort of thing that’ll get you into trouble. It might not always matter, but when it does if your application is parsing strings at runtime at high volume, it’s going to hurt!

I rarely log messages anyway, preferring to set state that I can then printout on command (e.g. siginfo)

So did you just turn logging off, or did you in fact write a better custom implementation? And how much faster did it go? Did you actually measure that? What kind of system was that on?

My first thought is, probably there was an fsync() or similar after logging each message. Also, printf() and friends have to lock the streams, so there might be some contention somewhere in case you're multi-threaded. That's not a problem that you can solve without some custom routing of logging messages.

Using printf, you can easily write dozens of megabytes of the most complicated formatting code on contemporary systems (I'm just pulling a number out of thin air here). More than any system should want to log.

People who deal in megabytes have different problems than people who deal in petabytes.

Usually just turn it off. At least at first. Note I’m talking diagnostic here; If I need the log messages obviously I have to write a custom streaming system even if the system printf is fast enough because the API complicates recovery.

Printf can also vary in performance by 10msec or more based on the internal state. That’s not good enough since my entire application has a hard limit of around 30msec. I can’t even do one printf — even every N messages (for any N) because I’ll never catch up.

> People who deal in megabytes have different problems than people who deal in petabytes.

I just hope that this is not meant ad-hominem. At the very least it's a bad reply, unless you are dealing with petabytes per second. (Btw one of my current projects is a text editor that can handle gigabytes in memory; local operations take 5-50 microseconds. So I have reason to think I'm not entirely clueless).

> I rarely log messages anyway, preferring to set state that I can then printout on command (e.g. siginfo)

That sounds much more reasonable to me giving the volumes that you cite, and why shouldn't we use printf() for that?

Why would printf "vary in performance by 10msec" or more, in ways that another application wouldn't? For how much data is that? How many printf() calls?

Anyway I shouldn't have gotten in the woods here. The blanket statement that printf() is slow and therefore an obscure API like sbrk() is a better use, is nonsensical for a guide that seemingly gives general advice for memory allocation.

> you can easily write dozens of megabytes

I meant to add per second.

You might not.

Your dependencies do.

And maybe you do in small amounts.

You think you are gaining something by misusing sbrk(2), but you are leaving a timebomb in your code. By your writing it seems like you are smart enough to know this.

From a performance perspective, it won't matter in large amounts if you put a stack on a high level heap or mmap its pages or use this silly sbrk(2) thing. You know this.

It seems like sbrk is a bad idea because it's a legacy API that manipulates global program state in a very error prone manner. Is there ever any reason to use it instead of mmap?

More generally, I was very confused by the entire stack section. Why sbrk instead of mmap? Why should C provide special functions for creating or using a stack? Isn't a stack just a region of memory and a pointer into it? Isn't their high performance entirely due to data access patterns that facilitate cache locality while minimizing instruction count and pointer dereferences?

I may well have some misconceptions, but if so this article did nothing to address them.

Edit: Aha! From the mmap man page:

> MAP_STACK (since Linux 2.6.27)

> Allocate the mapping at an address suitable for a process or thread stack.

> This flag is currently a no-op on Linux. However, by employing this flag, applications can ensure that they transparently obtain support if the flag is implemented in the future. ... some architectures may (later) require special treatment for stack allocations. ...

Don't forget MAP_GROWSDOWN

> MAP_GROWSDOWN

> This flag is used for stacks. It indicates to the kernel virtual memory system that the mapping should extend downward in memory. The return address is one page lower than the memory area that is actually created in the process's virtual address space. Touching an address in the "guard" page below the mapping will cause the mapping to grow by a page. This growth can be repeated until the mapping grows to within a page of the high end of the next lower mapping, at which point touching the "guard" page will result in a SIGSEGV signal.

The strange thing [to me] is that, in a discussion of stack allocation patterns, sbrk is a deeply confusing red-herring.

'Little-s' stacks require bulk-allocated pages that are then carved up into object allocations as required, and de-allocated in last-in-first-out ordering. No alloca or sbrk required.

Reply on news.ycombinator.com