Good point. We lose linearly one bit per ~177 characters that way. So zero termination does have linear cost.
1/(log2(256)-log2(255))
Assuming 8-bit char, at 1417 chars, zero terminated string overhead is 2 chars.
1/(log2(256)-log2(255))
Assuming 8-bit char, at 1417 chars, zero terminated string overhead is 2 chars.
For 255 characters you lose 9.4 bits which is bigger than the cost of an 8 bit size field.
For 65535 characters you lose 378 bits which is much bigger than the cost of a 16 bit size field.
On the other hand you have only one string type for arbitrarily sized strings and most strings are short.
What would really be needed to be more space efficient for short and long strings is a size field with a variable length encoding. E.g. 1 byte for string lengths up to 64, 2 bytes for string lengths up to 8k, etc.
It's just decoding that requires pipeline busting branches. Well, I guess it's fine if you take branch mispredict for the large string case.
Maybe 1 byte with highest bit zero, 0-127, 2 bytes for 128-32767, etc?
The irony is of course now we have variable length encoding for the string size...
I think just using size_t instead makes more sense than saving a byte or two per string. Memory is cheap, but CPUs aren't getting much faster.
When size is an issue and nothing else prevents it either, just LZ4 (or similar memcpy order of magnitude speed compression) the string.
Yes, "size_t" should be just about right for everybody except the guys who _need_ to program in C/C++ and Assembler due to the hard requirements of their systems. Not very many people but very important systems.