Hacker News Reader

Comment by stephc_int13 | Hacker News Reader

If you are using C and do some non-trivial work with strings you should either use a good library to handle strings or build your own.

It is not that difficult in practice.

The old C std lib is, in my opinion, outdated, obsolete and a very bad fit for complex string handling, especially on the memory management side.

In my own framework, the string management module is using a dedicated memory allocator and a "high level" string API with full UTF8 support from the start.

As a general rule, I think that the C std lib is the weakest part of the C language and it should only be used as a fallback.

View on HN

> The old C std lib is, in my opinion, outdated, obsolete

...and has been since most of us ever used C.

I think one of the major failings of C was the lack of a good standard library that updated with the times.

Actually, I believe a rich standard toolbox was one of the best features of python, and helped with its success.

Which is why most applications ping back into POSIX when available, not that fixes the security issues with the standard library.

Which parts of posix? What can I #include in a posix environment to get better string handling in C?

(Or maybe I’m misinterpreting your comment?)

You are misinterpreting my comment.

First part of my comment relates to C library in general, second part of my comment refers to strings and arrays, even if not explicitly.

Fair, though there's always the crossover point where you need to interact with the OS, 3rd party libraries, protocols, etc. It's not difficult to miss a spot where your utf8 string gets mangled, truncated, etc.

Yes, do not trust the OS, use the minimal API surface, and build your own toolkit, you won't have to do it often.

> the C std lib is the weakest part of the C language and it should only be used as a fallback.

I've been musing for a while now: what would it look like if we were to discard the C library and design a new one, leaving the language itself intact?

I think it could be very nice.

C is not perfect, there are some parts of the syntax that I strongly dislike, like casting or function pointers declaration...

But it is overall a good enough syntax, much simpler than C++.

Amending the syntax is fun but rapidly becomes a slippery slope; soon enough you find yourself designing a new successor language, as has been done many times before. Simply scrapping the mostly-unhelpful C stdlib and inventing new, modern abstractions for allocation, IO, text, threading, etc seems like a more tractable problem.

I fully agree.

It has the same fundamental problem, though: you have to rewrite most existing code, which hinders adoption. In this case, it might actually hinder it more than also improving the language itself, since people would be more willing to take that leap if there are more benefits to be had from it.

Stdlib is probably the most successful library in the history, not sure how it is “unhelpful”.

There are several libraries or projects where people have done exactly that.

You often end up with some kind of structure, or variations of structures, for strings:

    struct string {
      size_t length;
      char data[];
    };

    struct string {
      size_t length;
      size_t alloc;
      char *data;
    };

Those are just examples. The tricky part is figuring out the different ownership use cases you want to solve. Because C gives you so much freedom and very little in the standard library, you end up with a lot of variations. You might use reference-counted strings, owned buffers, or string slices, etc. You might want certain types to be distinguished at compile-time and other types to be distinguished at run-time.

An example can be found in the Git source code.

https://github.com/git/git/blob/master/strbuf.h

The history of changes to this file is interesting as well. This is a relatively nice general-purpose string type—you can easily append to it or truncate it.

IMHO that does not solve the main problem, that is individual lifetime management.

I've seen many libs using this style of strings, not convinced by the practicality.

What is "individual lifetime management"?

It sounds like you’re rephrasing part of my comment back to me, or maybe I’m misinterpreting what you’re saying.

If you’re not convinced of the practicality, it sounds like you are simply not convinced of the practicality of doing string processing in C at all, which is a fair view point. String processing in C is somewhat a minefield. Libraries like Git’s strbuf are very effective relative to other solutions in C, but lack safety relative to other languages.

No, I simply am using a different approach, still in C, where strings are simple char*, null-terminated, nothing hidden with magic fields above the base address of the string.

The trick is to pass an allocator (or container) to string handling functions.

If/when I want to get rid of all the garbage I reset the container/allocator.

Yeah, you should have just said that in the first place.

I’ve seen similar approaches, e.g. with APR pools, and if your application can work within those restrictions, it’s very convenient.

You can backport Rust standard library to C using https://github.com/eqrion/cbindgen .

The old MacOS (pre-X) did just that. Strings were all "Pascal strings", ie. with the first byte containing the length of the actual string.

Building blocks for memory were also very different from stdlib, notably the use of Handles, which were pointers of pointers, so that the OS could move a block of data around to defragment the heap behind your back without breaking the memory addressing.

Pascal strings are also kind of bad though. All sub-string operations need allocation, or have to be defined with intermediate results which aren't "really" strings, so in that sense it's not an improvement on Zero-terminated strings. Equality tests are cheaper which is nice, since strings of different lengths compare unequal immediately, but most things aren't really improved.

C++ string_view is closer to the Right Thing™ - a slice, but C++ doesn't (yet) define anywhere what the encoding is, so... that's not what it could be. Rust's str is a slice and it's defined as UTF-8 encoded.

D's strings were defined to be UTF-8 back in 2000. wstring is UTF-16, and dstring is UTF-32.

Back then it wasn't clear which encoding method would turn out to be dominant, so we did all three. (Java was built on UTF-16.)

As it eventually became clear, UTF-8 is da winnah, and the other formats are sideshows. Windows, which uses UTF-16, is handled by converting UTF-8 to -16 just before calling a Windows function, and converting anything coming back to UTF-8.

D doesn't distinguish between a string and a string view.

What’s the ownership story for string views?

They don't own anything. It's just a pointer and length. They don't allocate/deallocate.

I mean clearly something needs to own the buffer for a new string.

Sure, but that's not the string_view's problem, you can't just make string_views, the string you want to borrow a view into needs to exist first.

Imagine you go to a library and insist on borrowing "My Cousin Rachel", but they don't have it. "Oh I don't care whether you have the book, I just want to borrow it" is clearly nonsense. If they don't have it, you can't borrow it.

Walter is talking about D, and he said this:

> D doesn't distinguish between a string and a string view.

In C++ std::string owns the buffer and std::string_view borrows it. If there is no difference between the two in D, then how is this difference bridged?

You can use automatic memory management and not worry about it. Or you can use D's prototype ownership/borrowing system. Or you can encapsulate them in something that manages the memory. Or you can do ownership/borrowing by convention (it's not hard to do).

Automatic memory management makes copies?

No. Another word for automatic memory management is garbage collection.

I guess I should rephrase. Let's say I have a string, which owns its buffer. What happens in D if I take a substring of it? Does a copy of that section occur to form a new string?

A lot of people don't know about this but Microsoft is taking steps to move everything over to utf-8.

They added a setting in Windows 10 to switch the code page over to utf-8 and then in Windows 11 they made it on by default. Individual applications can turn it on for themselves so they don't need to rely on the system setting being checked.

With that you can, in theory, just use the -A variants of the winapi with utf-8 strings. I haven't tried it out yet as we still support prior Windows releases but it's nice that Microsoft has found a way out from the utf-16 mess.

The A-variants had problems years ago, which is why D abandoned them in favor of the W versions.

I don't mind seeing UTF-16 fade away. We've been considering scaling back the D support for UTF-16/32 in the runtime library, in favor of just using converters as necessary. We recommend using UTF-8 as much as practical.

> with the first byte containing the length of the actual string

And the wheels fall off with the first string longer than 255 characters.

Which is why Free Pascal strings are so awesome. I've personally stuffed a billion bytes on one, without issues. They are automatic reference counted, and as close to magic as you can get. You can return one from a function without issue.

However, Free Pascal has the worst documentation of any major project I've ever encountered (The exact opposite of Turbo Pascal), so I can't link to a good reference. Their Wiki is a black hole of nuance and sucks all useful stuff off the internet.

You can fix this issue by using a variable width integer encoding for the size.

It might fix that particular issue, but you still have the same problem that NUL terminated strings have: it's not possible to cheaply create views/slices of a string using the same type.

I remember that era well! During the first few years I used C, I never touched its standard library at all, using the Mac Toolbox instead. This was a common practice, which later carried over into C++.

The creator of this library (antirez) is a regular here on hn.

I believe this is used by Redis.

https://github.com/antirez/sds

Glib answer (but also relevant because mentioned in the article, too): it would look a lot like how a lot of people write C++.

>Glib answer

A Freudian slip, methinks.

How so? First definition I find of glib is "(of words or the person speaking them) fluent and voluble but insincere and shallow", which is mostly what I meant about that answer. There was some sincerity in my answer, but certainly somewhere in the border space of irony and sarcasm, which many people do take as insincerity.

https://en.wikipedia.org/wiki/GLib

The problem is that it's just too tempting to write something like

    my_function(my_var, 3.6, "bzarflo", my_other_var, false);

The string handling functions are part of the story, but the null-terminated char * is produced when the compiler reaches a string literal, and writing code without being allowed to just use string literals when it's convenient tends to feel like coding with oven mitts on.

It's entirely possible to write a wrapper function with a short name to convert string literals to actual string objects.

    my_function(my_var, 3.6, $("bzarflo"), my_other_var, false);

Isn't that much more of a mouthful, and as long as 'my_function' knows to free it, then you're A-OK! The only trouble is '$()' isn't legal in standard C, so a real solution would have to be something like 'str()'.

This caught my eye the other day & looks quite promising, though I haven't spent much time looking at it so I can't comment on it's memory safety:

https://github.com/tylov/STC

I'm in the WG14 and my opinion is that there isn't one good way to do strings it all depends on what you value (performance/ memory use) and the usage pattern. C in general only deal with data types, north their semantic meaning. (IOW We say what a float is not what it is used for). The two main deviations from that are text and time and both of them are causing us a lot of issues. My opinion is that writing your own text code is the best solution and the most "C" solution. Te one proposal i have heard that i like is for C to get versions of functions that use strings that take an array and a length, so as to not force the convention of null termination in order to use things like fopen.

Has WG14 considered adding slices to C? [1] Introducing slices would naturally give way to a better string library.

[1] https://www.digitalmars.com/articles/C-biggest-mistake.html

Its been 50 years so pretty much everything has been considered. In my opinion the mistake was not having arrays decay in to pointers but rather arrays should be pointers in the first place. An array should be seen as a number of values where with a pointer pointing at the first one. I think adding a third version of the same functionality would just complicate things further. (&p[42] is a "slice" of an array) Another thing I do not like about slices that store lengths, is that they hide memory layout from the user and that is not a very C thing to do.

If you think about arrays as pointers, you will get a lot of things wrong, e.g.

float m[10][10];

it not a an array of pointers, but a 2D dimensional array with 2D memory layout.

You are right, sizeof is the other big difference. I think these differences are small enough that it was a mistake separate the two. The similarities / differences do make them confusing.

How would you express a 2D memory layout with only pointers?

An array of pointers to arrays? Basically, a `T**` C#'s "jagged" arrays are like this, and to get a "true" 2D array, you use different syntax (a comma in the indexer):

    int[][] jagged; // an array of `int[]` (i.e. each element is a pointer to a `int[]`)
    int[,] multidimensional; // a "true" 2D array laid out in memory sequentially

    // allocate the jagged array; each `int[]` will be null until allocated separately
    jagged = new int[][10];
    Debug.Assert(jagged.All(elem => elem == null));
    for (int i = 0; i < 10; i++)
        jagged[i] = new double[10]; // allocate the internal arrays
    Debug.Assert(jagged[i][j] == 0);

    // allocate the multidimensional array; each `int` will be `default` which is 0
    // element [i,j] will be at offset `10*i + j`
    multiDimensional = new double[10, 10];
    Debug.Assert(multiDimensional[i, j] == 0);

Yes, this is people with pre-C99 compilers that do not support variably modified types sometimes do. It is horrible (although there are some use cases).

I plan to bring such a proposal forward for the next version. Note that C already has everything to do this without much overhead, e.g. in C23 you can write:

  int N = 10;
  char buf[N] = { };
  auto x = &buf;

and 'x' has a slice type that automatically remebers the size. This works today with GCC / clang (with extensions or C2X language mode: https://godbolt.org/z/cMbM57r46 ).

We simply can not name it without referring to N and we can also not use it in structs (ouch).

You know what i think about auto :-)

How is this not a quality of implementation issue? Any implementation is free to track all sizes as much as they want with the current standard.

Either a implementation is forced to issue an error at run time if there is an out of bounds read/write and in that case its a very different language than C, or its feature as-if lets any implementation ignore.

Tracking sizes for purposes of bounds checking is QoI and I think this is perfectly fine. But here we can also recover the size with sizeof, so it is also required for compliance:

https://godbolt.org/z/qh7P93Tcd

And I agree that this is a misuse of auto. I only used it here to show that the type we miss already exists inside the C compiler, we simply can name it only by constructing it again:

char (buf)[N] = ...

but we could simply allow

char (buf)[:] =

and be done (as suggested by Dennis Richtie: https://www.bell-labs.com/usr/dmr/www/vararray.pdf)

GLib is an alternative to the standard library.

> especially on the memory management side.

Libc string functions don't manage memory. They can be used no matter where your strings are stored. It is more of a choice between generality vs convenience in common cases.

A lot of them require the string to be null terminated rather than taking a length.

I think this is the main culprit of the libc string functions, you have to provide buffers to store results, and the responsibility of managing those individually can be annoying, and bug prone, resulting in vulnerabilities.

Passing an allocator (like Zig) or a container (like in my framework) to anything that needs to allocate some memory to store a result is both explicit, low overhead and quite convenient in practice.

Note that they're annoying, bug prone, and vulnerable in 3rd-party memory managed libraries, too, but you can just say it's someone else's problem then.

You cannot e.g. store a string as a slice of another string (unless the slice reaches the end).

Do you have any recommendations for good open source C standard library replacements instead of rolling your own string manipulation functions?

> use a good library

Such as?

Is your framework open sourced?

Not 100% decided yet, but it is very probable that I will open source it.

"If you are using C and do some non-trivial work with strings you should either use a good library to handle strings or build your own."

   unsigned int str_len(const char *s)
   {
     register const char *t;
   
     t = s;
     for (;;) {
       if (!*t) return t - s; ++t;
       if (!*t) return t - s; ++t;
       if (!*t) return t - s; ++t;
       if (!*t) return t - s; ++t;
     }
   }

I still use this instead of stdlib strlen. Of course I also use software everyday that I know uses stdlib strlen. For most C programs dealing with strings I just use flex and yyleng, which in turn uses the stdlib strlen. Using flex for small jobs is overkill but it's quick and convenient. I am a hobbyist programmer; I write so-called "trivial" programs.

That said, this exact function is used in some "non-trivial" software written by someone else and that person is IMHO a better C programmer than any HN commenter I have seen, most of whom do not let the public see the code they write anyway. Go figure.

NB. I am not the author; this is in the public domain. The author is djb.

I think this naming style should be considered obsolete.

- This function will return the number of bytes, not of characters or codepoints. - str and len are both abbreviations, we should use full words when possible - We can also be more explicit about what the function does, it does not simply returns the string length, it counts characters (or bytes in this case)

Here is how I would name it:

u32 CountBytesInString(char* string); u32 CountCharactersInString(char* string);

And on the implementation side, this work can be done with SIMD instructions, and be really freaking fast, but still, it should be explicit for the user that the work is O(n) complexity, not exactly free.

"- str and len are both abbreviations, we should use full words when possible -"

u32 and char are abbreviations.

Yes, this is true. And this is a tradeoff, I think that basic types are so widely used that we can use this style of abbreviation without much ambiguity.

strlen is also pretty unambiguous, but I still have to check what strstr means.

C isn't Java. Even Niklaus Wirth in Pascal, Oberon, and the like avoided naming their identifiers too long. 'GetStrSz()' is enough to achieve (most of) what you want, assuming certain naming conventions:

- Makes it clear that this returns the number of bytes, assuming a naming convention where `sz` refers to size (in bytes) and `ln` refers to length (in some other unit which would be specified in the type). Note that in C, 'characters' refers to bytes. It's a flaw in how C names its types, yes, but I wouldn't say it should be any different just because other languages do things differently.

- It doesn't use full words because I don't think it needs to. Abbreviations are OK as long as every (invested) party agrees that they're sane, and I think they're pretty sane.

- It makes it explicit that it is performing a calculation (hence, is O(n)) via 'get'.

I don't think all this is necessary, though - I actually think 'strln()' is enough. First, because characters means bytes, I can assume that this function is getting the number of characters (bytes) in a string. I wouldn't expect it to give me anything else! Second, in C, if strings were a struct of some sort, I'd expect to be able to get their length via 'str->ln', which would be O(1). The fact that the length is found through a function in the first place signals to me that it's doing something behind the scenes to figure that out. Remember - that's just my opinion, which I admit is extreme - but I think yours is just as extreme.

Naming is extremely important, and while strlen is a very basic and hardly ambiguous example, consistency is key and I believe that good naming rules should be applied globally or at least at the framework level.

I think that full words and verbs are easier to read and avoid ambiguity.

I guess this is a matter of style and preference.

This anecdote reminds me of the Mutazt type, something I found in a new codebase I was asked to debug. I had to dig for almost an hour to find exactly what this type was.

Turns out it was a char*, a C string. Buried under 4-5 levels of abstractions.

Mutazt = Mutable ASCII Zero Terminal.

Out of curiosity, why do you use this? I expect the builtin strlen() to be even more optimized than this. There's a lot more you can do than a simple loop unrolling.

Especially when there's a decent chance the compiler would replace the loop with a call to strlen.

Don't do this. libc string functions are usually done with hand tweaked assembly or as a builtin by the compiler. They will be faster than this.

This sort of practice is very outdated. The last time it made sense for performance reasons was probably the early 90s or earlier.

Additionally, strlen is not one of the C string functions you want to replace due to defects, the same way you would want to do with crusty old strcpy. If you're working with C strings there is nothing wrong with strlen. (Just don't call it redundantly in a loop body ...)

I've written my own strlen equivalent and benchmarked them against default on different compilers, processors and environments, and they almost always are faster or the same speed.

Default libs are sometimes very optimized but very often they are not, unfortunately.

If you care about performance, you should not rely blindly on the the defaults.

A long time ago I wondered about the performance of memcpy on the Nintendo DS, for sure they would have provided a hand optimized version? And yes, it was handcrafted ARM assembly code, but my own version turned out to be twice as fast.

They simply forgot to use a simple prefetching trick in their implem.

Nintendo DS has probably had a lot less scrutiny than a major libc or recent GCC or clang [though you can probably target its ARM processor with that]. Also, for an older embedded platform they may choose to do optimization for code size rather than cycles or clock time.

I'm going to have to doubt the start of your comment. Having seen a lot of libc implementations I think you are better off not wasting time optimizing strlen. Also memcpy, probably memcpy moreso. Most memcpy()s I've seen in the current century are using SIMD instructions and the like. And compilers don't even bother emitting a call to libc for it anymore, they do it as a builtin.

On the contrary, I expected the Nintendo DS SDK to be well optimized, performance of memcpy can be critical on such a constrained hardware. And it was optimized, just not with the best tricks.

I got the prefetching trick from Intel source code, except that I replaced the PLD instruction by a simple dummy load.

And about strlen, you'd be surprised, some implems are very good, and some are not, depends on the compiler and the library. I've ran benchmarks, I was surprised too.

To be honest, I don't really need super fast strlen, but I was curious and also learning to write fast SIMD code, basic string handling is a nice exercise.

I think this expectation doesn't vibe with my understanding of how people used to think about embedded or consoles. You shipped them and they were done. The games industry was also often trying to ship quickly. Small teams too. Latest tweaks to memcpy or fine tuning or revisiting the finer points of an already adequate SDK is low priority.

By contrast, many more people are updating optimizations to GCC or clang for arm, more frequently and over a longer timeframe.

You're probably right about consoles, and I was surprised to be wrong, but I checked, just to be sure.

GCC and Clang are very nice compilers, but they are a different thing than the std lib. glibc, musl, the Windows C Runtime, iOS, Android, all have different implementations, sometimes outdated.

Gcc and clang are relevant here because memcpy is a builtin. It will not call libc for this.

Same is true of MSVC.

It's been that way on most modern compilers for about 20 years.

That's why I'm saying rolling your own may be futile. Compilers, not just libc, have paid a lot of attention to getting those things fast.

What is true for memcpy (especially on small buffers) is not systematically true for string functions.

But testing is always there to the rescue, and these days we have Godbolt.

Any data to back up that FUD?

Of course I have data, do you think I am pulling benchmarks out of a hat?

But I have not published those benchmarks, if this is what you're asking, the Nintendo thing I am afraid I cannot reproduce easily as I no longer have this devkit on hand.

About the strlen benchmark, this is something I've done a few years ago, that could be easy to run again, but I am not sure this is worth the effort just to convince a random dude on the internet...

> do you think I am pulling benchmarks out of a hat?

Yes. Share the data or you're spreading FUD.

You're being silly. Nothing they said was FUD, it was just a (not particularly controversial) anecdote

Here is the musl stdlib strlen. I do not use glibc.

https://git.musl-libc.org/cgit/musl/plain/src/string/strlen....

   #include <string.h>
   #include <stdint.h>
   #include <limits.h>
   
   #define ALIGN (sizeof(size_t))
   #define ONES ((size_t)-1/UCHAR_MAX)
   #define HIGHS (ONES * (UCHAR_MAX/2+1))
   #define HASZERO(x) ((x)-ONES & ~(x) & HIGHS)
   
   size_t strlen(const char *s)
   {
   const char *a = s;
   #ifdef __GNUC__
   typedef size_t __attribute__((__may_alias__)) word;
   const word *w;
   for (; (uintptr_t)s % ALIGN; s++) if (!*s) return s-a;
   for (w = (const void *)s; !HASZERO(*w); w++);
   s = (const void *)w;
   #endif
   for (; *s; s++);
   return s-a;
   }

There are HN readers besides myself, not to mention hundreds if not thousands of others, who use various programs that use this function, such as

https://cr.yp.to/ucspi-tcp/install.html

https://cr.yp.to/daemontools/install.html

https://cr.yp.to/djbdns/install.html

https://cr.yp.to/clockspeed.html

https://cr.yp.to/software/qmail-1.03.tar.gz

https://smarden.org/runit/install.html

glibc strlen:

https://sourceware.org/git/?p=glibc.git;a=blob_plain;f=strin...

   #include <libc-pointer-arith.h>
   #include <string-fzb.h>
   #include <string-fzc.h>
   #include <string-fzi.h>
   #include <string-shift.h>
   #include <string.h>
   
   #ifdef STRLEN
   # define __strlen STRLEN
   #endif
   
   /* Return the length of the null-terminated string STR.  Scan for
      the null terminator quickly by testing four bytes at a time.  */
   size_t
   __strlen (const char *str)
   {
     /* Align pointer to sizeof op_t.  */
     const uintptr_t s_int = (uintptr_t) str;
     const op_t *word_ptr = (const op_t*) PTR_ALIGN_DOWN (str, sizeof (op_t));
   
     op_t word = *word_ptr;
     find_t mask = shift_find (find_zero_all (word), s_int);
     if (mask != 0)
       return index_first (mask);
   
     do
       word = *++word_ptr;
     while (! has_zero (word));
   
     return ((const char *) word_ptr) + index_first_zero (word) - str;
   }
   #ifndef STRLEN
   weak_alias (__strlen, strlen)
   libc_hidden_builtin_def (strlen)
   #endif

NetBSD common strlen:

https://ftp.netbsd.org/pub/NetBSD/NetBSD-current/src/common/...

   size_t
   strlen(const char *str)
   {
   const char *s;
   
   for (s = str; *s; ++s)
   continue;
   return(s - str);
   }

Apple strlen:

https://opensource.apple.com/source/Libc/Libc-1244.50.9/stri...

Apple strlen comes from FreeBSD. Until 2009, FreeBSD used an unoptimised strlen.

https://svnweb.FreeBSD.org/base/head/lib/libc/string/strlen....

   size_t
   strlen(str)
           const char *str;
   {
           const char *s;
   
           for (s = str; *s; ++s);
           return(s - str);
   }

FreeBSD eventually copied^1 NetBSD's x86_64 strlen.

1. "modeled after", "inspired by", etc.

https://svnweb.FreeBSD.org/base?view=revision&revision=18770...

   #include <sys/cdefs.h>
   __FBSDID("$FreeBSD$");
   
   #include <sys/limits.h>
   #include <sys/types.h>
   #include <string.h>
   
   /*
    * Portable strlen() for 32-bit and 64-bit systems.
    *
    * Rationale: it is generally much more efficient to do word length
    * operations and avoid branches on modern computer systems, as
    * compared to byte-length operations with a lot of branches.
    *
    * The expression:
    *
    *      ((x - 0x01....01) & ~x & 0x80....80)
    *
    * would evaluate to a non-zero value iff any of the bytes in the
    * original word is zero.  However, we can further reduce ~1/3 of
    * time if we consider that strlen() usually operate on 7-bit ASCII
    * by employing the following expression, which allows false positive
    * when high bit of 1 and use the tail case to catch these case:
    *
    *      ((x - 0x01....01) & 0x80....80)
    *
    * This is more than 5.2 times as compared to the raw implementation
    * on Intel T7300 under EM64T mode for strings longer than word length.
    */
   
   /* Magic numbers for the algorithm */
   #if LONG_BIT == 32
   static const unsigned long mask01 = 0x01010101;
   static const unsigned long mask80 = 0x80808080;
   #elif LONG_BIT == 64
   static const unsigned long mask01 = 0x0101010101010101;
   static const unsigned long mask80 = 0x8080808080808080;
   #else
   #error Unsupported word size
   #endif
   
   #define LONGPTR_MASK (sizeof(long) - 1)
   
   /*
    * Helper macro to return string length if we caught the zero
    * byte.
    */
   #define testbyte(x)                             \
           do {                                    \
                   if (p[x] == '\0')               \
                       return (p - str + x);       \
           } while (0)
   
   size_t
   strlen(const char *str)
   {
           const char *p;
           const unsigned long *lp;
   
           /* Skip the first few bytes until we have an aligned p */
           for (p = str; (uintptr_t)p & LONGPTR_MASK; p++)
               if (*p == '\0')
                   return (p - str);
   
           /* Scan the rest of the string using word sized operation */
           for (lp = (const unsigned long *)p; ; lp++)
               if ((*lp - mask01) & mask80) {
                   p = (const char *)(lp);
                   testbyte(0);
                   testbyte(1);
                   testbyte(2);
                   testbyte(3);
   #if (LONG_BIT >= 64)
                   testbyte(4);
                   testbyte(5);
                   testbyte(6);
                   testbyte(7);
   #endif
               }
   
           /* NOTREACHED */
           return 0;
   }

Something worth considering: Sometimes people may use older versions of compilers, e.g., older versions of GCC, for compiling older programs for older hardware. These GCC versions are smaller in size and written to run on less powerful hardware. For example, the gzip'd source tarball for GCC 2.95 from 2001 is 12M while the one for GCC 12.2 from 2022 is 143M, an 11x size increase.

Reply on news.ycombinator.com