Comment by dspillett | Hacker News Reader

Comment by dspillett | Hacker News Reader

Parent Full thread

> In Windows, folders are entirely free in terms of disk space! For proof, create say 352,449 folders and get properties on it.

To be that guy for a moment: well, hackchewally…¹

Directory entries do take up space in the MFT, but that doesn't show up in explorer which is only counting allocated blocks elsewhere. You will eventually hit a space issue creating empty directories as the MTF grows to accept their allocation.

You can do similar tricks with small files. Create an empty text file and check, it will show 0 bytes size and 0 bytes on disk. Put in ~400 bytes of text and check again: explorer will show 400 bytes in length but 0 size on disk because the data is in the directory entry in the pre-allocated MFT. Double up that data, and it will be big enough that a block on the disk is allocated: in properties in Explorer you'll now see 800 bytes length and 4,096 bytes (one block) on disk. Drop it back to 400 bytes and it won't move the data back into the MFT, you'll now see 400 bytes length, 4096 bytes consumed on disk.

--

[1] though don't let this put you off enjoying the splendid thing overall!

NTFS is very much an old school file system design, where most/all metadata is stuffed into predefined regions on the disk. NTFS's MFT, in particular, can and does grow as needed (though it never shrinks!), but as entries in the MFT vanish, that space is just waiting for future allocations. It makes it a bit difficult to tell what an empty directory or file "really" takes, because when it's just part of the MFT, there's (usually) no change in used/free disk space just by creating them.

In the Unix land, the same sort of thing exists with file systems such as ext4 and UFS2: they also depend on predefined regions for metadata. If you'd like to venture into ZFS, however, (almost) all metadata is dynamically created and destroyed on an as-needed basis, and as such, ZFS always reports the on-disk usage by counting both user data and metadata. It's easy to see a directory grow to many megabytes by just creating a lot of empty files inside of it.

interesting! does that mean a windows exe could repeatedly create empty files, stuff a bunch of metadata in them, and eventually eat up all the disk space in a way that can't be fixed without re-installing windows? have viruses been known to exploit this?

> and eventually eat up all the disk space in a way that can't be fixed without re-installing windows
Not quite, as the MFT can't consume all the device, so you can't fill the whole lot that way, but you could cause significant inconvenience that is not undoable without migrating to another filesystem. That migration might be possible without a reinstall as such, though it would take a lot of manual jigger-pokery (and I don't know of any tools that would help, it isn't something I expect someone has felt the need to write tools for) so the reinstall would likely be easier.

> have viruses been known to exploit this?*

I doubt it. If the virus is intended to extract something from the user (exfiltrating data, ransom, making them part of a bot-net, etc.) then it wants to work in the background not causing inconvenience like that (until it fires in the case of encryption & ransom, but that is a different sort of inconvenience), and if the goal of the virus is just to cause a mess then there are more effective methods of doing so.

Remember that esolangs are arguably the "purest" medium of artistic expression for programming, and that artworks often are about challenging or confronting implicit assumptions. Or to put it differently: yes, that is indeed the joke :).

Thank you for elaborating the technical details though (plus I did not know the small file trick, that's a neat bit of trivia).

Also MFT can also grow in size if you exceed the current allocated size. Depending on your version of windows that growth rate is different. Also once the MFT grows it will not shrink. The tools for MFT cleanup are rather poor. With the usual recommendation of 'just format a new drive and start over'.

Apple finally managed to switch its file system and escape the pains of late 80s technology. When will Microsoft replace NTFS?

In theory ReFS is the replacement. It hasn’t seen large uptake though.

"Don't touch if it works" (c) /s

Master File Table

"Disk manufacturers hate this simple trick for free data storage."

Oh, that might make a great April 1st release: a mirror filesystem module for WinFSP that splits files into 500 byte chunks on disk. “See, we saved that 4Mbyte photo to the new filesystem, and it, using the NTFS infinite space for small files trick, made it take absolutely zero disk space! Now we'll make a sub-folder and drop a few more files in that, and look, they show as taking no space in the magic backing store but can be properly opened as normal!”.

Drop it on youtube or tt, get your popular friends (this might scupper me, if anyone I know is an online influenza they have the good sense not to let me know about such proclivities!) to make a review of it, and see how far and wide it spreads with people either in on the joke or idiots just parroting it for views.

Back in the days of dos, I use one of the disK compression utilities to store 20 MB on a regular 3.5-in floppy. The entire 20 MB consisted of metadata for the compression system!

Reply on news.ycombinator.com