The tale of aux.c | Hacker News Reader

The tale of aux.c

heirloom.sourceforge.net

217 pointsuyoakaoma122 comments

So, all that text to tell people he is refusing to rename a single file? :-P

I encountered that issue a few times in the past, most recently when a couple of years ago i was writing the console support for my 3d game engine and made two files called `con.c` and `con.h`, checked them in to version control and then at some point (much) later synced it on the Windows side. Fossil (the VCS i use) simply skipped that file (maybe it complained? i don't remember, but the files weren't created) and when i tried to build the engine it didn't work - i thought that i forgot to add the files, rebooted to Linux, forced a re-add, back to Windows and nothing again.

Eventually it clicked that "con" is a bad word in Windows filesystems, so i simply renamed the files to `csl.c` and `csl.h` :-P. I found this a bit irritating because my naming convention follows a mostly rigid pattern where `xxx_` prefixed functions will always recide on `xxx.c/h` (or `xxx_yyy.c/h` where yyy is a subsystem or specialization) and i really prefer the `con_` prefix to the `csl_` one. So far i've kept the `con_` prefix and made a special note for that in the readme file where i explain the naming conventions, but every time i have to use or work with that part of the engine i feel the irritation :-P.

I found it an interesting and well-written piece of history.

So, you smug Linux guys think it can't happen to you? Well, before you get off my lawn, let me tell you a story...

Early versions of FrameMaker kept all resource files (internationalized strings, dialog box layouts, etc.) in the two subdirectories .../FrameMaker/Resources/{Unix,Core} (the first one for the Unix platform-specific resources, and the second for the cross-platform ones also used by the Mac and Windows ports). We had hundreds of happy customers, and had been shipping for 4 years or more.

But one new customer was calling in a panic. They'd installed the product, and it would work fine, but the next morning, it wouldn't even start up, and they had to re-install it. And then it would break again over night. After much pulling of hair and gnashing of teeth, it turns out that due to some other flakey product they were running, they had previously instituted a nightly cron job that ran around removing any core files it could find (since they were large and wasteful of precious disk space back in the day). And the knucklehead who wrote the script had it do a case-insensitive match on "core" and had it essentially do a "rm -rf" even though no true core file could be a directory. So, every night, our Core resource directory was blown away.

We actually had to change the product, as this was a big, important customer, and they outright refused to fix their cron script. Never had this problem on the Windows and Mac ports.

How is this specific to Linux?

Ok, "unix-derivative" if you prefer. FWIW, it actually happened on SunOS or maybe HP-UX, and it was before Linux existed.

But it could have just as easily been a windows batch file, it's not related to the OS at all.

Well, apart from the fact that UNIX-like systems have traditionally dumped core files in programs' working directories if they crash, so needing a script to clean them up.

Linux still does, though many distros set ulimit or /proc/sys/kernel/core_pattern to disable it.

Windows does this too, except it hides them in it's system folders, it even comes with a built in clean-up script you can run. Not a problem that is worse on *nix.

I'm reminded of when I discovered on a production system that "killall" is very different on Solaris to Linux.

Seriously? They couldn't be bothered to add -type f? Depending on context I can be a big customer but I'd be embarrassed to ask you to work around something like that. My odd-ball only on Wall Street security system I can't do much about but sheesh that's like a couple of minutes to test. They probably couldn't push it everywhere easily?

While an interesting story, this is more of a self-imposed customer screw-up and not quite the same thing, IMO.

A more direct (though fictional) illustration that Unix also has reserved file names that you'd better avoid: "I was writing a novel about digging to the center of the Earth, entitled "core". One day I was running some program that crashed, and my novel turned into a bunch of binary gibberish." Or how about: "My video of my kid's first at-bat, in a file I named a.out, got mysteriously clobbered when I took my first programming class." These are more insidious than the MSDOS examples, which at least fail right away.

They are less insidious by far. You're really reaching for threads here.

The first, `core`, isn't relevant. `/proc/sys/kernel/core_pattern` configures that value, and by default it's named "core.$PID" nowadays.

That's far better than windows' wart since it's configurable, disableable, etc.

Your second example is even more contrived. If you run a tool which writes files, of course it can overwrite files. I don't complain that in windows when I type "copy foo bar" my "bar" file is overwritten by "foo".

In the same way, someone who voluntarily runs "gcc x.c" or "cat > foo" should not be surprised when "a.out" and "foo" respectively are overwritten.

There's a massive difference between an unconfigurable gotcha based on a filename and an explicit action which writes a file.

You don't really put videos into a program's source tree. Also I don't think any modern Unix-like OS would overweite a file with a core dump no matter what is the name, and doubt that ancient versions did so too. Having the same set of device files under every directory is quite a different thing.

Actually, not only did they do exactly that, they could even be induced to do so through a symlimk, with hilarious results.

I'm really curious about them hilarious results now :)

Actually, when I just want to test something silly, I often just go to ~, make a file test.c or something, write the short test program, and run `gcc -o test test.c`. It's not that rare to have videos in your home directory, and if I had one named a.out, and forgot the -o option, it'd get overwritten.

Not saying it's remotely likely, just saying that having video files in a program's source tree isn't the only way it could happen :)

If we're going to criticise Unix file naming, I'd rather talk about how file names with spaces in them break things, or how hard it is do delete a file named "-".

There's something in my environment that keeps creating a file called "-" in my home directory, and it's driving me crazy.

By convention, when a unix program is passed "--" it means there are no more flags being passed. Try the following:

$ rm -- -

This will not work with all rm implementations, in which case you can fall back to: rm ./-

Oh, I can remove it... And a few hours or days later it's back!

Okay, you're gonna want to type $sudo crontab -e...

It is better to write EDITOR=nano sudo crontab ... otherwise instead of edititng the crontab you will be solving the problem of exiting vi.

All the interest encouraged me to dig in and find it, and it turned out to be in one of zsh's many startup scripts.

list the file showing inode number, rm by that particular inode.

Open the parent directory with emacs and you can use it to delete the rogue directory/file. Admittedly, there's easier ways for a "-" named file, but its a good solution to know since you can use it to delete even the most difficult filenames.

  rm -- -

While GNU's not unix... afaik GNU coreutils' rm has done exactly what you expect it to when given a single dash as an argument for who knows how long.

The problem is names that could be interpreted like options, like '-f' or '--help'.

Interestingly, when the shell expands * , it won't do anything to make sure files starting with - are handled properly.

This means, if you're in a directory with the files a, b, -rf, and the directories foo/ and bar/, running "rm * " will not delete all the files and keep all the directories; instead it will be turned into "rm a b bar foo -rf", meaning it will delete all the files and directories except for the file -rf.

I've definitely run into this on bash and zsh (iirc). I'm curious if this glob expansion behaviour is set in posix somewhere.

If you'd like a real world example, I once used the predictably of core dump filenames to (legitimately) gain root access to a busy public nameserver.

https://inopinatus.org/2015/08/24/privilege-escalation/ has the detail.

(^predictability)

>My video of my kid's first at-bat, in a file I named a.out, got mysteriously clobbered when I took my first programming class.

This is a failure to understand GCC's default behavior, not a fundamental flaw with your operating system.

The "core" thing is a little more understandable, but it can all be remedied by choosing reasonable filenames with accurate file extensions. "aux.c" isn't that unreasonable, and if I'm reading it right, he could've called it "aux.foo" and it still would've been broken.

> This is a failure to understand [Program X]'s default behavior, not a fundamental flaw with your operating system.

Ah, yes, the age old Unix trope of, "it's not the system, it's the user's fault".

As a Linux user for many years, that doesn't excuse such boneheaded errors. By default no application should clobber existing files, unless asked to. Avoid data loss and all that.

Heck, I remember back when:

tar myfile.txt myarchive.tar

would result in myfile.txt being truncated to 0 bytes (therefore deleting its contents) and a snarky message that myarchive.tar can't be archived because it can't be found.

For those that don't remember how tar works, the arguments are reversed compared to, say, cp.

At least I think they've fixed it, newer tar version says something like "Cowardly refusing to create an empty archive".

The Unix ecosystem, dragged kicking and screaming towards user-friendliness since 1970.

>Ah, yes, the age old Unix trope of, "it's not the system, it's the user's fault".

Well, if the user had read the manual before using GCC, it wouldn't have happened.

Regardless, I don't blame them or look down my nose at them. It's bad design. If it's anyone's fault, it's GCC's. But it's certainly not a problem with Unix. This makes it incomparable to the system-level issue of having global filenames like DOS did, apparently.

Well, as far as modern design goes, I doubt many people argue that Unix was better designed than DOS. DOS was a single user, single tasking OS meant for very limited hardware.

While Unix was kind of Ok, when coupled with its common userland utilities, the whole often felt like a cactus. Possibly useful, if you don't mind the scars :D

I'm a professional software developer (well, "professional"; I have a part time programmming job next to studies), where I program in C++, compiling with GCC. I have never read GCC's manual. Expecting newcomers to read the 18 thousand lines of man page before trying to make a program isn't reasonable.

Of course, the man page is a great resource for looking things up, but I can't imagine many people sit down with GCC's man page and actually read it through.

> Ah, yes, the age old Unix trope of, "it's not the system, it's the user's fault"

Except that example could also happen on Windows.

Looks like it is time for this again...

> Ah, yes, the age old Unix trope of, "it's not the system, it's the user's fault"

Ah, yes, the age-old complaint that a car driver should be able to run a forklift with no practice.

I don't think Unixen should strive for 'user friendliness'. Hacking in "Do you really want to" dialogs would (a) be contrary to all expectations developed over a long time, (b) break a lot of existing software, (b) annoy the bloody hell out of everyone, and (d) not solve anything, because those dialogs don't prevent mistakes.

Not all systems need to converge, and Unix has been a professional toolchain since the beginning. You're expected to know something about it, just like you don't let kids run plasma cutters.

"My video of my kid's first at-bat, on a DVD in front of my plasma cutter, got mysteriously clobbered when I turned the cutter on."

  > For those that don't remember how tar works, the arguments are reversed compared to, say, cp.

Tar doesn't have positional file arguments; you use the 'f' flag to specify an archive file. You might be thinking of zip or some similar archiver in the DOS heritage.

My bad, I was actually talking about a very common tar idiom which is almost an automatism for me:

tar cvf/czvf/cjvf archive files (or -cvf/-czvf/-cjvf).

I don't think I've ever seen anyone actually using tar by going:

tar -c files -f archive

Online tutorials seem to also use my form: http://www.thegeekstuff.com/2010/04/unix-tar-command-example...

Either way, bad form, I'm glad they fixed it :)

This doesn't really have anything to do with *nix, but rather the customer being an idiot.

Nice story, but unrelated. And I'm not sure why you mention Linux at all.

We had a source directory named "core" in an early version of the Rosegarden sequencer. At the time our build system was a simple Makefile, but in 2000 we switched to Automake and quickly learned that in some circumstances, using the Automake-generated Makefiles, "make clean" would delete the entire directory. Have generally avoided that name since then.

How to delete "~" directory :) https://askubuntu.com/questions/26466/removing-folder-named

I.e. “how do I operate my shell”. You certainly want to understand shell expansion to get stuff done effectively.

I hope that customer paid you enough for you trouble. But obviously it wasn't as much as it would have cost them to fix their crappy cron script.

The lesson here is "suck it up and build software for the systems your customers use, not the ones you wish they used".

On Windows, paths can be accessed with \\?\ prefix:

    mkdir \\?\V:\con
    notepad \\?\V:\con\aux.txt

This avoids reserved filenames as well as 260-character length restriction. But many programs that try to do any path processing fail after seeing an unexpected prefix, even File Open dialog (on Win7 at least).

It was finally fixed on Windows 10.

Length was, maybe. Reserved device names (like CON, AUX, etc) are still reserved.

I was referring only to length.

Not really. Windows 10 made it easier in certain ways for a program to support long file names. But it still requires explicit support, and programs can and did support it previously. And importantly explorer still doesn't support it properly.

It just requires enabling it, no special care from applications.

It is disabled by default due to backwards compatibility.

https://msdn.microsoft.com/en-us/library/windows/desktop/aa3...

The file explorer does not support it on purpose.

https://superuser.com/questions/1114359/windows-10-home-anni...

Once you enable that, it still only works on applications that explicitly declare support in their manifest.

The backwards compatibility here is only in the fine details of how you deal with long paths. The old way and the new way both require application support. The new way is easier, but the old way worked fine even on XP and earlier.

This setting isn't actually future-looking. It was already possible to make these files with certain tools. And they could have let it always be on for programs that support it. This setting exists so you can signal to programs that they shouldn't use long paths, even when they are capable of it.

> The file explorer does not support it on purpose.

Sure, it's on purpose, but it's a good example of how impotent the setting is.

Hell, 8.3 filename creation is still enabled by default on 10. WTF?

https://technet.microsoft.com/en-us/library/cc959352.aspx

Last time I tried I was still unable to create the file '.gitignore' on Windows 10 using the file browser. It insisted that I enter a file name and did not consider '.gitignore' an acceptable file name. It is possible to create that file using other tools such as an editor so it's a user interface shortcoming and not a filesystem shortcoming.

That one has a trick to it! Explorer rejects filenames that are "just extensions", but that's just a strong opinion it has. At the same time, there is a behavior that filenames ending in periods are actually invalid, and get the periods silently stripped by win32. So type ".gitignore." to name a file .gitignore

In 1991 I was working at a place with a Novell Netware network. I decided to make a system that would execute remote commands by writing them to a shared directory and other machines would see the files and execute the commands. I named those files COMXXXXX.TXT where the X's were a random number (yeah, that was a bad idea). Every few days the system would hang and stop working. I finally narrowed it down to the random number being 1-9 that would correspond to the serial ports on the machine and it would then hang.

Lesson: always zero pad.

Should've used left-pad.

00Good Advice

Be careful with that. Using leading zeros (in KSH) treats them as octal and doing things "old" school will break.

To be fair, that was not at all obvious from your tersely worded post.

I’m sorry, I seem to have fat fingered it and replied to the wrong post.

If I had to guess I would bet that none of the commenters in this thread have ever used the Heirloom version of "mailx", and some have never used, or would ever want to use, the BSD mailx either.

It is a shame because I think Mr. Ritter deserves more recognition for his work.

IMHO, some of his versions are real improvements over the BSD ones, e.g., troff (doctools) and nailx.

I for one am very thankful for the Heirloom Project and very glad to see it live on.

I know of him through the version of vi that comes with Arch. I really appreciate that there's somebody keeping a conservative version of vi around. Even if I'm mostly going to use vi-like editors with more features, I'm glad there's a vi to fall back on.

We ran into this issue in Servo too. https://github.com/servo/servo/issues/1226

Me as well while cloning linux on windows

You run into different, but similar, issues cloning Linux on macOS as well, since HFS+ is case-insensitive by default (and looks like the same is true of APFS), but the Linux source has filenames that differ only in case (don't ask me why).

Same thing in Cargo: http://sasheldon.com/blog/2017/05/07/how-i-broke-cargo-for-w... https://github.com/rust-lang/cargo/issues/3982

... and the Linux kernel itself.

https://bugs.eclipse.org/bugs/show_bug.cgi?id=509072

I learned about this when I added cloning the Linux kernel repository to an integration test suite for a cross-platform filesystem. I had to use another project because of that file :)

I ran into it while trying to clone the source for Minix, just to poke around and read the code. (I think it was COM and not AUX but same principle)

The pirate scene occasionally hits this as well (copies of the film "Con Air" which would be named "Con.Air.<foo>" by convention were instead named "Con_.Air" or "Con_Air" etc. All thanks to the legacy CON device.

>But nowadays, people confuse Microsoft Windows, the successor to DOS, with a Unix operating system, and want to run mailx on it.

...huh

>If you want to use mailx, there is the technically and morally sane option of using a free Unix implementation.

I mean it's really not that hard to release a Windows compatible version. You change a single file name?

This just sounds like throwing gas on the flame war that is unix vs windows.

You can even work around it without renaming the file. Cygwin and LXSS for example allow the use of file names such as "aux.c" without any changes to the mailx source.

It was written in 2007. There was much greater tension between Microsoft and free software at that time.

yeah I just noticed that, good point. Although releasing as much OSS/Free software to every OS (regardless of status) would have been a better fighting route than "*uck you."

Is this when I get to complain about operating systems that allow spaces in filenames which also have a shell language that makes it difficult to use those files?

Even if Microsoft keeps this behavior, I'm surprised they don't modify it to check the filename extension. It's one thing to change a device like CON that people might expect to exist, but it seems like not distinguishing CON and con.c is just asking for trouble.

That behavior is intentional and is workaround for programs that does not let user change file extension on saving.

The funny part is, that is not the purpose at all any more. It was probably only for a handful of years that this trick was needed to "trick" programs which thought they were saving a normal file and forcing the correct extension to actually use a printer or serial port. No one has used a printer or serial port by File -> Save As to "lpt.txt" in decades!

Now it's just because some code hidden away under 10s of layers a millions of lines in some obscure line-of-business applications is accidentally dependent on this quirk and "it works now" and no one will ever fix it.

This isn't an issue for contemporary human facing UIs. Somewhere there is guaranteed to be a crusty POS system still in use that is sending receipt data to LPT.TXT or AUX.TXT.

They could remove it in a new version of windows, that crusty POS system is going to be doing a lot of testing before they upgrade the version of windows it's running on. Even more likely is that they'll never upgrade from whatever ancient version they're using so it won't affect anyone.

I think they just keep it because it feeds into the backwards compatibility myth that MS has built up.

Or they could just create a registry flag that enables the backwards compatible behavior. Then all default installs don't have the "bug", while those that need it can enable it.

They could probably make the new behavior opt-in in the application manifest.

Without a manifest, you'll see pre-XP common controls, and GetVersion will return an old version, so every modern application has it anyway.

They could even incorporate this into some sort of... Compatibility Mode.

Batch scripts don't have application manifests (nor, to the sibling comment, a compat mode setting).

I would not be that surprised if even in this year's version of some niche CADs the most sane way to print the drawing in correct scale on DesignJet is to export the document as 'PRN.HPG' or something similar.

Possibly. On the other hand, you're talking about changing behavior that's been in place for 35 years. It's more than a little late to garner any benefit from changing it. And the risk that you'll actually break an existing program somewhere is certainly small but probably not zero.

Or they could have at least removed the behavior when in 64bit mode.

This is an entirely different mechanism, but was bitten not long ago by a surprising feature of the C++ language. It was as surprising to me as I imagine saving a file to aux.c would be.

I was working on a C++ program, editing, compiling, debugging as one does. After a while, my efforts to compile were stymied due to a flood of weird compiler errors that didn't make sense. After rewinding recent changes to a point where I knew it used to compile, it still refused to compile.

Finally I figured it out. During debugging, I created two copies of log file, one called "old" and one called "new". Somewhere in an include file (which probably included other files, etc), there was a statement "#include <new>". Rather that picking up the library source code, it tried to interpolate the contents of the local file "new" during the compile phase.

Every time I read this I have to wonder how many windows users are really interested in running mailx.

See also https://news.ycombinator.com/item?id=12129647 "Git for Windows accidentally creates NTFS alternate data streams"

I came from a BBC Micro background and found MS-DOS to be a step backwards from what I knew. I couldn't believe how naff MS-DOS was and how lazy the coders must have been to make it so primitive. I also found PC vendors to be lazy too, it wasn't as if any of them made an effort to do any code to improve on matters, they just shoved the same old MS-DOS in the box.

What I had not realised before reading this article was that the file system was as flat as Acorn's ADFS, i.e. no hierarchy. How this joke of a product came to take over the world and take us from networked computing to standalone 'personal computing' was a tragedy, holding computing back 'decades' rather than enabling a better world.

ADFS allowed directories, but used "." as the separator, giving you paths like "Letters.Memo1".

Were you thinking of DFS?

Both DFS and ADFS used .

You couldn't do hierarchy though, it was only one deep.

That was true for DFS, but not ADFS.

Does anyone remember Basic-4 systems?

I transferred several programs, in those days and on that system with a maximum of six characters for the name, from one machine to another via magtape. One was named "CR" (customer report or credit report, who knows). It disappeared on the new system because it was mistaken for the Card Reader.

Lol, a huge wall of text because the OP doesn't want to rename a single file.

Jeez.

Most of that text explains the often forgotten issue, it is only the last paragraph that tells his preferences. I didn't even know that CP/M was sane in its naming scheme and thought it was CP/M legacy, not Microsoft's madness. Pretty informative, though not much surprising.

To be fair most of it was quite interesting, but those last two paragraphs felt really out of place.

"{S /con/con" in AOL chat used to BSOD everybody's computers - {s was a directive to play a sound

you could do the same thing to Internet Explorer with simple image link

What happens if you create a file called aux.c under a different OS, and then mount the drive on Windows?

Nothing. Maybe. The magic happens in open() or createfile() (or a little underneath there, precisely). Nothing about the disk format prohibits such names, and they're accessible via other pathnames. But if you have a program that lists files and then tries to open each one automatically, it may have a bad time. I would hope AV software has tested this scenario, but AV software.

This is the type of question that leads to new classes of exploit... well thought!

Except it's one of the first things each of the thousands (likely more) of professional and hobbyist hackers have already tried after learning about this particular OS quirk

I wonder if the restricted names are still with us in ReFS? I don't have one handy to check it on, but it seems like if there were ever a good time to break backwards-compatibiliy, ReFS would be it.

The restriction isn't in the filesystem. You can mount an NTFS filesystem on a Linux box, create a file called "aux.c", then mount that filesystem on Windows.

With default mount options you can even create files/folders named foo\bar.

Reply on news.ycombinator.com