Building the next generation file system for Windows: ReFS
blogs.msdn.com
blogs.msdn.com
Of these, I'm sorry to see the demise of sparse files. This was, IMHO, the single most under-utilized feature of NTFS, and I was able to integrate support for sparse files into a number of clients' applications (I'm a low-level consultant and developer) to great effect. While the increasing size of volumes along with the sub-par utilization of this feature makes it an obvious victim when creating a new filesystem and looking for features to drop, sparse files can be amazing for other reasons.
One of the advantages of sparse files is that they can be used to naively support certain seek-related behaviors. If you create the file right, you can save yourself a lot of code and complexity in any applications consuming that data.
The biggest advantage of sparse files though is speed. For instance, you can create a container file of X size filled with zero bytes, and only use as much space as the end application requests (for example, creating a virtual disk of 2TB that only takes up 100MB on disk).
I, for one, am sad to see this feature go. For anyone interested in this amazing feature, have a read here: http://www.flexhex.com/docs/articles/sparse-files.phtml
Honestly, any serious application that doesn't take symlinks into account in 2010s is a joke. Unfortunately, in the Windows world, there's a lot of them. Even hard-core backup applications (I've consulted for a few backup companies) mess this one up.
Hardlinks were the precursor to softlinks, in this day and age their only purpose is to let you say "I've given up on the software I use handling soft links properly," and while we can wish for such a feature, I don't think it's that bad of a decision to drop them.
I agree, but it doesn't stop me from having to use these programs where they clearly kill their competitors in terms of features or usability.
Not to start a fight but windows programs take a serious step back in ease of use (for me) vs their linux counterparts. I work from home so my home PC = my work PC.
Unfortunately I'm a massive fan of multiple displays (5 currently) in various orientations and linux epicly fails at this. I used linux as my primary OS for 7 years but finally gave up over this singular issue.
Failing to support sparse files, though . . . man, that's just insane. That would relegate ReFS to the status of a toy in most filesystem developers' minds, even before you consider their increasing usefulness when storing virtual-machine disk images in a shared filesystem to support migration, etc. It's hard to imagine that none of the many people who must be involved in this at MS raised the red flag. What seems more likely, from what I know of MS culture, is that some people did raise it but then some idiot dictator with a reputation built on some long-irrelevant project ignored or dismissed their objections.
Since ReFS doesn't look to be in the client version of Windows 8, though, i don't think it'll make much difference to applications - and i expect they'll add some of these features back before Windows 9. Hopefully they have a more transparent versioning system than NTFS this time around
Perhaps Microsoft is making a decision to focus the filesystem on being a simple storage engine and moving features into other modules (primarily) above or even below it?
- named streams are out => it becomes unlikely that we will see these become popular on any OS (because being incompatible with the market leader is problematic; see Mac OS X, .DS_Store). I find that a pity.
- I guess quotas are out because there will be something else replacing it?
- Can anyone explain why a modern filesystem should have a limitation on path length? For APIs I can understand it because the standard C library thinks paths are fixed-length, but for file systems? I would think this complicates the implementation, as every directory would need to know the length of the deepest path below it (in case one attempts to rename it). Aggregating that info upwards whenever a file is created or renamed (let alone deleted) cannot come for free, can it?
Not free, but it's not asymptotically significant.
If it works like ZFS, then you will be able to create as many filesystem volumes as you want on top of a storage pool, each with their own size limit. You could create separate file systems for each user.
> Can anyone explain why a modern filesystem should have a limitation on path length?
The 32K limit doesn't really matter, because we're effectively stuck forever with the much lower and older limit of 260 characters. Longer paths would crash existing applications who use 260 byte buffers to store file system paths.
http://serverfault.com/questions/163419/window-256-character...
Performance and verbosity of code. Having a limit allows the creation of fixed-sized structures, which makes the code much simpler (remember, code has bugs, so more code is more bugs). It also makes the generated machine code much simpler, leading to better performance.
"The NTFS features we have chosen to not support in ReFS are: named streams, object IDs, short names, compression, file level encryption (EFS), user data transactions, sparse, hard-links, extended attributes, and quotas."
I wonder if/when this will actually take off - most Microsoft "edge case" solutions have trouble gaining adopters. If it gets boot off mirror support, it has a chance.
I don't think that's true at all. As others have mentioned, it appears they are matching the state of art achieved by ZFS.
The 'Linux folks' have been working on this for quite awhile, if it weren't for licensing incompatibilities with ZFS, they'd likely already have it.
If you'll pardon my saying so, that's because it's not 'the Linux folks' that have been working on it - it's other non-'Linux folk' companies (cough RIP, Sun cough) that took it upon themselves to make a better filesystem for their (coincidentally, and nothing to do with Linux) open source operating systems. The closest the 'Linux folks' got was ReiserFS under Hans Reieser, who's work was largely rejected by the mainstream 'Linux folks' working on the kernel... until the months just before his arrest and conviction for the murder of his wife.
ZFS has as much to do with Linux as NTFS has to do with Linux - developed wholly outside of the Linux community by people not in the Linux community nor associated with the Linux community, with implementations available for Linux that are not redistributable with the kernel for patent- or licensing-related reasons.
But, yes, BtrFS (developed by Oracle) is indeed a 'Linux folk' attempt at creating a modern filesystem. And BtrFS does indeed predate ReFS.
There's also a native port: http://zfsonlinux.org/
So it will be a bootable FS eventually.
Now, seriously, if I got a dollar for every new Windows filesystem announced for every next version of Windows and canned before launch, I'd be at least five dollars richer. By the time they deliver it, IF they deliver it, BtrFS will be widely used in Windows servers. ZFS already is way more advanced than what they propose.
The only major change I saw was when Microsoft ditched HPFS to go with NTFS.
No. WinFS was not a filesystem, it was an application layer on top of NTFS (and SQL).
In fact the changes to the underlying filesystem originally motivated by WinFS (i.e. TxF) did make it into Vista.
BTRFS on Windows?
And yes, ZFS is more advanced, at least from what can be deduced from this article.
Sorry. Editing accident. Can't correct it anymore. I meant Linux servers, of course.