Hacker News Reader

Comment by thom | Hacker News Reader

What’s the state of the art for file systems in the age of M2 drives that are bonkers fast? I dunno about everyone else, but knowing hierarchically where my disk space is going is a really common concern.

View on HN

I would imagine, the “m2 fast” is visible more in reading medium sized files than 100k tiny reads (when recursively querying directories like this case GP is taking about)

To make GP’s use case better, FS really needs an custom index or require that every write also update all hierarchy. Unless of course they work like indexers and just work with delayed data

Yes, main speed improvement with SSDs come now via IOP/s, for metadata, small file ops like this.

Good point but likewise one has to wonder why the need for folder hierarchies anyway. With bonkers fast disks and cpus the organising of files should be able to be dynamic and driven by some form of metadata and tags rather than static tree structures. The web doesn’t first require you to define a structure before search so why should a desktop?

This is exactly what Steve Jobs would tell me every week when he would look at the latest Finder build. Of course, we needed multiple pieces of infrastructure to get there; a journaled and indexed filesystem, kernel-level, file system modification notifications, metadata indexing systems, etc. All of the pieces did appear, but there are also those darn stubborn users who kept insisting on the ability to navigate a physical file system to an actual file.

Many of the features of OSX (which I rarely use, but worked tirelessly on) came as a result of Steve's dislike of having to know where files were and having to navigate to them. The Dock, Spotlight, Mission Control, Expose, even Time Machine all came out of Steve's hard-to-pin-down concept of what a modern user interface should be.

I suspect it may be possible to create a fully "Smart Folder" query driven Finder experience. Your sidebar could be populated entirely of saved queries. This might be a fun experiment!

So I'd have to tag every one of the umpteen files I create (in some session doing whatever) with some spur-of-the-moment searchword I'll never remember later, in stead of just cd-ing to some directory and that's where they'll all be?

It's funny how all these anti-directory tag-and-search proponents seem to take for granted that their hobbyhorse is obviously superior. I've never seen any proof that it is, and I don't think it is.

> "tag every... file I create ... with some spur-of-the-moment searchword"

isn't that kind of what we are already doing with the names used in the directory hierarchy + the filename?

No, I just create files. I don't have to do any tagging at all.

Or: Yes, in a way that's what the hierarchical directory structure already does for us.

In either case: So what's the use of ripping out the hierarchical directory structure and replacing it with some (other) "tagging" system???

(We already have a universal "tagging" system that automatically uses every single word in every single file as a searchable "tag"; it's called "grep"...)

Because spacetime is hierarchical in that way too. I can put a thing in a box in a room, but I can't put the thing in multiple boxes in various rooms. The filesystem thus makes intuitive sense: we store physical objects in exactly the same way.

Now sure, it'd be nice to grep through my house, but I still couldn't hang the same painting in two places at the same time.

On a related note, why a filesystem and not a database? I get the feeling that a filesystem ultimately is just a .. poor man's database. Except that it's quite crappy and there are no proper transactions, many things are impossible without TOCTOUs, many a bug (including security ones) have been due to race conditions around filesystem operations.

I guess path->{metadata,data} with no transactions is a simple abstraction, as with so many other simple abstractions, it just bites you when you try to build anything nontrivial. Then you need to switch to a real solution. Just as people sometimes start with a bunch of shell scripts before they recoil in horror and realize that they should've started with a real programming language.

Afaik, it's been tried and abandoned for some reason.

see WinFS https://en.wikipedia.org/wiki/WinFS

Not fully abandoned, mainframes and microcomputers use database like filesystems.

Like beOS/Haiku does! Metadata and dynamic grouping of files + search is an amazing concept

Interesting. Sounds like Gmail’s labels vs everyone else’s folders. I wonder if it is a patent holding things back.

ProtonMail launched with labels only. They later added folders for those of us who like to file messages under "organized now, don't want to think about this again".

No, Fastmail uses labels, and is also the org behind JMAP, which uses them natively. Folders are so common in email because that's built-in to IMAP.

Folders are so common still in part because many of us want them to the point that both Fastmail and Gmail lets you use tags in ways that behaves almost entirely like folders, including a hierarchy.

I'd never consider an e-mail system (or a filesystem) that doesn't allow me to at least very closely approximate that structure.

Reply on news.ycombinator.com