Dietlibc – a Libc optimized for small size | Hacker News Reader

Dietlibc – a Libc optimized for small size

fefe.de

88 pointskick40 comments

Open articleView on HN

A comparison of dietlibc, uclibc, musl, and glibc, by the author of musl: http://www.etalabs.net/compare_libcs.html

Came here for this- musl is a very nice libc and I don’t find the comparison too biased personally.

Another factor to consider is release cadence:

- musl: 4 releases so far in 2019, 2 in 2018, 3 in 2017[1]

- uClibc: No releases since 2012[2]

- uClibc-ng: 2 releases in 2019, 3 in 2018, 6 in 2017[3]

- dietlibc: only 1 release since 2013[4]

[1] https://www.musl-libc.org/releases/

[2] https://uclibc.org/

[3] https://downloads.uclibc-ng.org/releases/

[4] https://www.fefe.de/dietlibc/

is libc something that generally requires multiple releases every year?

It needs to keep up with underlying changes in the operating system from time to time.

Given the chip architectural changes in the last 6 years, I'd hope to see at least more than one release in that time frame.

Often (not allways, naturally) it's enough to just rebuild with a newer compiler to leverage the new arch features.

Out of interest, what are some new things (those bringing real performance gain) from the last 6 years which a standard library needs to adapt to actively?

Edit: also note that OP made this pointed on purpose, the last release is from 2018, one year ago, not 2013...

The standard library benefits a great deal from ASM optimized implementations of functions. New instructions get added that can speed things up significantly.

the compiler will almost never automatically generate this code.

This is all true, but I’m not sure the authors of these libraries are that interested in performance.

musl has 4 for 2019.

musl-1.1.24.tar.gz (sig) - October 13, 2019

musl-1.1.23.tar.gz (sig) - July 16, 2019

musl-1.1.22.tar.gz (sig) - April 9, 2019

musl-1.1.21.tar.gz (sig) - January 21, 2019

http://www.musl-libc.org/oldversions.html

Good note - I missed those because the format of the text was different on the page I linked to. Fixed!

> - dietlibc: only 1 release since 2013[4]

you missed the first entry (20180924).

> Another factor to consider is release cadence

It's questionable that "more releases" == "better". Sometimes software is (ideally) just done (except for security issues). I use a lot of software that has no significant changes since decades. Regarding a libc-replacement I couldn't name a single important thing for the last years.

But of course, no release could also be a bad sign, like beeing abandoned or that is has just no users (testers).

> Regarding a libc-replacement I couldn't name a single important thing for the last years.

* Supporting new syscalls, such as getrandom(), or extra options on existing syscalls (e.g., socket options).

* Support for libc-components of new hardware versions, such as the register state information that you get on signals.

* Supporting new versions of C, such as C11.

* Improving performance of threading and memory allocation, among other things.

* Updating Unicode tables for locale information.

I don't think openbsd libc has ever kept the same version for two consecutive releases (six months). Which doesn't even count bug fixes as a version change. There's always something to add or remove.

Is there a reason none of these replacement libcs are available for Windows?

Because part of what libc does is bridge the OS and user space. This is very much OS dependent. You can get a rough idea by looking at where it makes "syscalls."

That's not true for the vast majority of libc. It's usually file IO, memory mapping and command line IO that use system calls and everything else builds on those or just modifies memory.

Threading as well. It means porting requires significant work.

How so? I've dug into the source of musl and allocators like jemalloc. Windows has VirtualAlloc as its analog to mmap and a simple function to print to the command line. Starting threads is similar. Most of libc is pure C. Even most of musl is optimizations for different platforms and compilers. The simple straightforward versions of functions are not anywhere near the complexity of what gets used most of the time.

Most libcs aren't cross-platform; there is mingw, which provides an alternate libc on windows. I'm not actually sure why the libcs aren't cross-platform; I'm currently working on one which plans to support freebsd, linux, and windows.

Well.... there's WCRT: http://www.ibsensoftware.com/files/WCRT-1.16.104.zip

Should note that glibc now supports c11 threads. AFAIK there was some problem with their initial specification that caused them to want not to implement them, and c18 amended that so they're now comfortable supporting them.

Also note that openrisc has been mostly superceded by risc-v, which afaik glibc and musl both support; don't know about the other libcs.

These are rather old though:

  musl 1.1.5
  uClibc 0.9.33.2 (Buildroot 2015.02)
  dietlibc 0.32
  glibc 2.19

musl is at 1.1.24 at the time of writing.

I like the proverb on the top of the page:

Person who say it cannot be done should not interrupt person doing it. --Chinese Proverb

However, I think it maybe a fake proverb.

https://www.npr.org/2018/06/12/619140377/after-ivanka-trump-...

I recently encountered dietlibc while trying to debug a segfault in integrit [1]. As packaged in Debian, integrit is a static binary using dietlibc. For some unknown reason I found it completely impossible to get any usable stacktrace out of it with gdb and the usual tools and after a few days of frustration I gave up on finding the cause.

[1] https://github.com/integrit/integrit

Yes, part of the usual process of making a tiny binary is jettisoning much of the overhead that debuggers depend on in order to find their way through an external program. Not just the debugging symbols themselves, but things like section header tables and maintenance of linked stack frames in the running code. Dumping all of that saves space, at the expense of making the resulting binary more and more opaque.

It’s probably because the code optimized out the frame pointer; if you really wanted a backtrace you’d need to look back at the stack to see where the return address was.

I used dietlibc as a replacement for glibc in supermin, when making an initramfs. As background the size of the initramfs is key for boot performance since qemu takes quite some time (where "some time" means milliseconds) to load a large initramfs; and the binaries in the initramfs are usually statically linked. The final size of the init binary was about 1/40th as big as when using glibc:

https://github.com/libguestfs/supermin/blob/c97b3917068597a0...

I don't do anything special to my initramfs, and the largest of them that I've generated over the years is 16 MB in size. For my desktop with a 3 TB drive, having the initramfs take up half a meg instead won't make any appreciable difference.

I could see it being very useful on resource constrained microcontrollers or something, though.

It makes a huge difference when trying to boot an appliance virtual machine in milliseconds. Paper: http://git.annexia.org/?p=libguestfs-talks.git;a=tree;f=2016...

I believe the point is boot speed. Larger initramfs take longer to unpack.

Interesting project.

What is the footprint of this compiled for a MCU, something like a STM32?

This libc is for Linux, so I guess you can't compile it for a bare-metal system like STM32.

For microcontrollers the most popular open source option is newlib[1].

[1] http://sourceware.org/newlib/

Keith Packard maintains an excellent "picolibc" that I have used recently (and contributed to, a little). It was formerly called newlib-nano, and consists of newlib with the string functions replaced with cheaper (slower) versions from avrlibc and various other slimming changes (e.g. reduction of struct reent stuff, and some locale stuff, by default).

https://github.com/keith-packard/picolibc

If I remember correctly, dietlibc was part of a whole bunch of DJB-fanboy releases by fefe way back when dinosaurs and qmail ruled the earth. Not that much happening since then, I think he's mostly CCC-ish-blogging these days.

If I remember correctly it grew out of frustration with glibc size which turned into research into how to make a libc as small as possible. No DJB involved.

> Not that much happening since then, I think he's mostly CCC-ish-blogging these days.

Latest release was last year and the changelog[1] suggests it was a group effort.

Do I smell a personal dislike for the author?

[1] http://www.fefe.de/dietlibc/changes-0.34.txt

Neither for the author nor for DJB. I was on the mailing list when dietlibc and libowfat (the alternative, non-traditional, DJB-inspired core library) were created. Was an interesting time and I liked the philosophy way more than e.g. the whole 'suckless' stuff (which is more in the Plan 9 corner).

I don't regard "fanboy" as a derogatory term, by the way.

There was quite a lot of development and thoughts back then, but in recent years I hadn't heard a lot out of that corner, and in German circles I mostly saw von Leitner's name mentioned when it came to some digital rights thing (our various German and EU "privacy" laws).

> I don't regard "fanboy" as a derogatory term, by the way.

I do. Whenever I read "fanboy", I assume the author is at least no fan of the subject (in this case I assume you don't like DJB). Otherwise it would just be a "fan"... ?

Reply on news.ycombinator.com