ZPL: (Almost) C99 Powerkit | Hacker News Reader

ZPL: (Almost) C99 Powerkit

github.com

I feel like every sufficiently large C project ends up reimplementing the same things: binary-safe (or at least explicit-length) strings, dynamically-sized arrays, hashmaps, a set of typedefs for integers and booleans (if the project predates wide C99 support), a macro to count the number of elements in an array, bit manipulation functions, locale-independent string functions…

It's a sad story about the state of the C standard library.

When I see nice C libraries, it does make me want to write new C code. But then I am reminded of the many non-library problems with C, like the `uint64_t m = 1 << n;` footgun, the ubiquity of pointer parameters yet lack of syntax to show data flow direction, the inability to specify the width of an enum value…

Despite everything, I do like C, especially compared to C++, but there's so many little things that could be improved and yet have been stagnant for decades. And that's without taking about the massive safety issues.

To play devil's advocate: it's because one size does not fit all. C gives you the flexibility to tailor your implementation for your specific needs. Whereas in a higher level language, you'd have to fight the language / standard library if your requirements don't match the "few" true ways to do things.

That said, too many projects believe they have special needs, when they're better served just using e.g. apr in the first place.

This argument does not make sense to me at all. C has a standard library, so why can't it be a good one? There's also many things about the language that could be improved without sacrificing any of its low-level qualities.

There are many reasons IMHO:

- C is designed by committee, so proposals to expand the standard library move slowly.

- Specifications suitable for general computing may not be so for embedded micro-controllers. TBF this is slowly changing as well, given that a ARM CortexM can have double-digit kilobytes of memory now, and is almost as fast as a PII.

- Philosophical: some C users maintain that C's supposed to be a "portable assembler" and higher level abstractions are beyond the scope[0].

This may sound tongue-in-cheek but: you should use C++ instead. Yes it has a lot of flaws of its own, but IMHO it really lives up to the spirit of being a "better C". It has (almost?) everything you said wanted in it standard library. And just because you're using C++ doesn't mean you have to go all-in with template meta-programming, objects, etc. I've written very C-ish code in C++, and only used parts of the C++ standard library I absolutely needed. Last time I did that I was able to "un-C++" my code later to satisfy upstream requirement (no capes!^W C++) relatively easily, after iterating with C++ first.

[0] I know there are counterarguments that C's never been that and it's more a "convenient lie." Nonetheless the that's the spirit of the argument.

C++ lacks some of the good things added to C after C89, unfortunately.

Other than restrict, with dubious usage validation, I fail to see what.

Finally, lack of vendor motivation.

Nobody's making big money on C compilers, no huge company with deep pockets feels they would benefit significantly from a "stronger" C.

> Most parts of the library are written in pure C99. There are however, some additional components that require C11 to work (notably in the Threading module). As long as you are using minimal version (ZPL_NANO), and manually enable modules, the C99 support should be just fine.

I think it would be better to describe it as a C11 library. I didn't know what to make of 'almost C99', which might refer to limited use of compiler-specific features, for instance.

There are compilers that don't support C11. Knowing that downgrading to C99 is a simple procedure is worthwhile.

Sure, but that wasn't my point. almost C99 is unclear, unless you already know the intent. It's telling me which language the framework isn't written in.

90% C99, 10% C11 would be fine.

It looks like ZPL has its own memcpy() implementation which is a work in progress. https://github.com/zpl-c/zpl/blob/dfcb9538967f381363eaef10f9... That leads me to believe feedback is desired so assuming the author posted this as a Show HN kind of thread, here's some of what I've learned implementing that function.

It goes pretty quick on the machines I've used so far to have an indirect branch with overlapping moves. Here's what that looks like in "pure c" for gcc and clang. https://github.com/jart/cosmopolitan/blob/master/libc/str/me... memcpy is hard to implement in pure c99 because aliasing pointers to non-char types can trigger ubsan errors. With rep movsb it's a good idea to check cpuid for erms support. In that case it'll be fastest for bigger moves but is still slower for small moves. See the chart at https://justine.lol/cosmopolitan/index.html which compares various methods. Also take into consideration that if you call zpl_memcpy() the compiler doesn't know that's memcpy() so it can't perform some of its inlining magic at the call site. So if you want things like null propagation it might be better to do that as a macro.

Regarding array.c I implemented a bunch of similar routines myself and eventually chose to not worry about capacity and just call realloc every time. I'm not sure if the standard specifies this, but I'm pretty sure every realloc implementation in practice, under the hood, should track the capacity and grow it at two power sizes. So even if you're just constantly appending by one char the cost should amortize itself out to be cheap in the long run.

Testing performance of memcpy should be more exhaustive than just running on single CPU: https://github.com/ClickHouse/ClickHouse/issues/18583#issuec...

I don't feel like you evaluated my work fairly because you used a modified version of Cosmopolitan memcpy() that had the CPUID checks removed, and then you judged it negatively for not having the CPUID checks.

It is very interesting to see your carefully choosing a specific layer of abstraction to somehow find the optimize both portability and performance. I agree that C99 might not always the right choice.

BTW, I am not the ZPL author, but I collect different C libraries as a personal stash of toolkits, and thought this might be interesting to other C programmers.

> I collect different C libraries as a personal stash of toolkits

Might you have a write-up or link for these, and if so might you be willing to share it?

I have been using Sean Barrett's libraries [0], as well as his curated list of other people's single-header libraries [1], and, like you, I am always on the lookout for new things to add to the collection :)

[0] https://github.com/nothings/stb

[1] https://github.com/nothings/single_file_libs

I'm glad you brought up Sean Barrett's work since the ZPL codebase references it so STB was likely a source of influence for Dominik. It's one of those projects where I just see its impact everywhere I look and I believe the success is well-deserved.

Oh hey, you made APE and redbean! Very cool projects, and something I always wondered if it was possible (I once tried a different strategy, failed, and then never tried again). I was very pleased to see someone had succeeded.

Thanks! I talked with one other person so far who discovered the idea before me. I'm glad I was able to contribute running the marathon it required. All we need to do now is make it simple and universally accessible to everyone who needs it.

Yes, I probably will get some reviews on some my experiences with them later.

I also use stb.h, and very impressed that one can put so many things together in one file. Essentially I feel C is very good at doing what most other languages are struggling at - make code running 99% of the time fast, but when it comes to that rest 1%, it takes a lot of repetitive works. I think an "essential-collection" could greatly reduce the unnecessary effort.

This looks like a nice and concise library. Single header file, many useful functions, and a usable license. The only thing I really dislike are all of the typedefs.

  zpl_u64 zpl_crc64(void const *data, zpl_isize len);

Const correctness, good! But why is the size parameter signed? Why not use size_t and uint64_t like the C stdlib?

The zpl_u64 seems to be because they're trying to support older versions of the MSVC compiler[1]. As for why they're using a signed size parameter, I'm not really sure. One idiom they appear to use often is[2]:

    for (...; x--; y++)

where x is a size-type of some kind. They may be defaulting to signed size types to avoid errors in case somebody decides to change to above to:

    for (...; x-- >= 0; y++)

There's even a macro zpl_size_of:

    #define zpl_size_of(x) (zpl_isize)(sizeof(x))

But the real weirdness is in the other macros:

    #define cast(Type) (Type)

...just why? And

    #define ZPL_MULTILINE(...) #__VA_ARGS__

is supposed to be a way of having multiline literals in C. But, everything inside the "..." has to be valid C tokens. It might work as a string literal in many cases, but it's misleading at best and broken at worst.

I'm not sure what the advantage of the following is vs. just setting the bits using & and | which every C programmer should already be familiar with:

    #define ZPL_MASK_SET(var, set, mask) \
    do {                                 \
        if (set)                         \
        (var) |= (mask);                 \
        else                             \
        (var) &= ~(mask);                \
    } while (0)

There's surely no excuse for this:

    #define zpl_global static        // Global variables

While a lot of the library looks useful, stuff like the above makes it feel like it's written by somebody who knows C, but doesn't feel fully comfortable writing it.

[1]: https://github.com/zpl-c/zpl/blob/master/code/header/essenti...

[2]: https://github.com/zpl-c/zpl/blob/27c80bd5807da5238777bfcba8...

These extra wrappers are most likely when library writer trying to cover windows platform, which has almost never been a good place for ANSI C programming.

> There's surely no excuse for this

The "excuse" is greppability since static has three completely different meanings in C and the desire to clearly denote global variables is fairly common. The same reasoning goes for the cast define as well.

Whether you find value in these small abuses of the preprocessor depends on your coding habits, but I don't really see much reason to criticize them. If the author finds them helpful, what's the harm? There isn't much room for confusion and they're not forced on the users of the library.

From a quick look at the code it seems that zpl_isize is used to represent indices, counts, etc and in those cases -1 is often used as an "invalid" value with special meaning (e.g. in the hashtable -1 is used as a "not found" index).

A bit sad that they did not go for

    uint64_t zpl_crc64(size_t len, const unsigned char data[len]);

Is that even portable?

It's a C99 variable-length array declaration, so the syntax is valid; but unfortunately it doesn't actually do anything useful. The 'const unsigned char data[len]' declaration just decays to 'const unsigned char *data' -- the first level of array-ness is discarded.

[ there was a bit of discussion about this earlier in the month: https://news.ycombinator.com/item?id=26349903 ]

It is useful for static analysis, even the standard people say that they will move into this kind of declaration.

> 15. Application Programming Interfaces (APIs) should be self-documenting when possible. In particular, the order of parameters in function declarations should be arranged such that the size of an array appears before the array. The purpose is to allow Variable-Length Array (VLA) notation to be used. This not only makes the code's purpose clearer to human readers, but also makes static analysis easier. Any new APIs added to the Standard should take this into consideration.

from http://www.open-std.org/jtc1/sc22/wg14/www/docs/n2086.htm

Which was dropped in C11, with clang and gcc probably being the only C compilers on Earth that might still support it on C11 and C17 mode.

It was not dropped. VLAs were deemed as optional but (if I am not mistaken) this specific syntax is not optional, only local VLA declarations.

> with clang and gcc probably being the only C compilers on Earth that might still support it on C11 and C17 mode

Are there any other C11 compilers even?

Deemed as optional is a synomim for dropped to any compiler that is yet to fully implement them, as they aren't required to support it for ISO C certification.

Several commercial compilers for embedded development, mainframes and surviving UNIXes, VC2019 since 16.8.

Additionally ICC apparently never supported VLAs that well.

Is "Powerkit" some common term I'm missing here? At first I thought this was related to some kind of Apple "Kit", written in straight C instead of Objective-C/Swift.

I think that it is called "Powerkit" in a sense that it gives C programmers some tools that make daily programming task easier.

Actually the C++ Framework for Mac OS made by Metrowerks was called PowerPlant.

So unrelated to the Zebra Programming Language?

This is beautiful. I think I am going to toy around with it in a project I have been dreaming about.

is there any other similar libraries i should know about? frankly the developer is right, I hate re-implementation of lists and other crap in modern languages.

I have a complete crap kernel include file i use when I write kernel modules. ( Though You will find me no longer publishing source code. Client is moving to freebsd, after a linux kernel bug that got labeled "WONT FIX" by linus & crew )

So client goes "WONT SUPPORT" on 5.x linux. Linux userland is a damn mess.

Reply on news.ycombinator.com