Hacker News Reader

Comment by Koshkin | Hacker News Reader

Arbitrary or not, the limits had to be there due to small amount of memory computer had back then. I don’t think this is specific to Unix/C culture (unless it was what influenced the design of MS-DOS with its limit on the length of file names, for example).

View on HN

Agreed, I think it was just a matter of being originally developed at a time when memory was tight and CPU cycles were few. That is, if the original UNIX were developed today with otherwise the same development culture, I'm not sure that we'd see fixed length restrictions.

One of GNU's original goals was eliminating arbitrary fixed size limits in UNIX tools, in exchange for using more memory. It's discussed in https://www.gnu.org/gnu/thegnuproject.en.html under "Technical goals".

> But it was natural to apply the known standards of good practice to the work—for example, dynamically allocating data structures to avoid arbitrary fixed size limits, and handling all the possible 8-bit codes wherever that made sense.

> In addition, we rejected the Unix focus on small memory size, by deciding not to support 16-bit machines (it was clear that 32-bit machines would be the norm by the time the GNU system was finished), and to make no effort to reduce memory usage unless it exceeded a megabyte.

you don't need megabytes to allocate your strings dynamically, microsoft basic did it on the altair in 4k

"/bin/sh\0" or "\7\0/bin/sh" is less than 14 bytes, not more, even including a pointer or two

the real issue is that anything you try to allocate dynamically can fail, so you need a failure path, and your failure path needs to reliably free all the other dynamically allocated resources

early unix kind of sucked at handling failures

a fixed-size buffer isn't more efficient but it is simpler

ms-dos used cp/m's filesystem, which had fixed-size filename and ext fields, which i think it copied from rt-11, which i think got them from various pdp-8 systems

fixed-size string buffers make more sense on a word-oriented machine, or arguably in a filesystem, where pointers require some kind of hack

i'd like to blame radix50 but i don't think we can because then the limit would be 9.3 or 6.3 rather than 8.3. there have been some 6.3 filesystems (3 16-bit words per filename with rad50) but cp/m fat (and thus ms-dos) are not among them

Perhaps I wrong, but FAT and CP/M filesystems, aren't the same. CP/M filesystem never had directories. When the original FAT even predates QDOS as was used by Microsoft on his BASIC in 1977

hmm, i thought fat came from cp/m, not basic-80, but it looks like i was wrong about that

thanks for the correction

https://www.informit.com/articles/article.aspx?p=25878&seqNu...

fat didn't have subdirectories in qdos or ms-dos pre-2 either

A great and easy idea, as long as you never need to release anything. Otherwise you end up with holes in your memory and you need to do compaction, and reserve the memory you need to be able to do that.

or you can just have memory fragmentation

compaction doesn't require a lot of memory but it's slow and intrusive; it has to know where all your buffer pointers are

Obviously permanently wasting memory on fragmentation is not an option on a device that typically has 24kb memory. Especially not just so you can specify long filenames for the shell, when the filesystem doesn’t even support those filenames.

the alternative they chose was to permanently waste memory on fixed-size struct fields, so i guess it was actually an option

malloc on the pdp-11 never compacted of course

the filesystem supported 14 characters for the basename, but evidently we're talking about code with a 14-character limit on the pathname, not the basename

You don’t need compaction to reuse fixed size structures. I presume their priorities were with creating a working system, not one that answers to the ‘requirements’ of 50 years into the future.

you don't need compaction to reuse heap-allocated memory either

wasted memory due to heap fragmentation is very similar to wasted memory due to slack space in fixed-size structure fields

I'd argue that in practical cases, resizable arrays require less memory than fixed-size arrays, because the actual length is usually going to be shorter than the arbitrary maximum length. My guess is that C encouraged using arbitrary limits by not having first-class support for resizable arrays (or even a non-hacky type-safe way to implement them as a library).

The other advantage of fixed size buffers is if you want to write the records to disk you can just write it directly. You don't need to serialize or instantiate the strings when reading, you can just copy the bytes straight to or from the file.

Looking at things like man utmp.5 should reveal what their thinking was.

the mode of thinking certainly hasnt changed since hardware improved, however

Yet the computers designed one decade earlier were much better in security than UNIX culture, go figure.

Maybe because they were bigger?

Maybe because UNIX folks had more fun doing their stuff instead of actually do it properly?

One decade older hardware was better than newly released PDP-11...

Thankfully we have encyclopedias.

Oh I bet fun also played a role. But I don't even know what exactly you are comparing against the PDP-11 here or and what point, because UNIX development happened on 11s for at least a decade. UNIX first ran on a PDP-11 in 1971 or so and a decade earlier there was only CTSS perhaps to compare it to, which had more memory than a PDP-11 at the time. Later timesharing systems like Multics and TOPS-10 also ran on bigger machines. And let's not forget it all began on a PDP-7, which was even smaller.

Reply on news.ycombinator.com