Vboxsf fixes for 5.14-1
lore.kernel.org
lore.kernel.org
As the person who tested the latest ntfs3 patchset, and had tested
many of those iterations in the past, I would really like to see
this *finally* land in Linux 5.14.
However, I get the feeling it's not going to make it for 5.14 *or*
5.15, and it seems like Paragon became discouraged by the lack of
feedback on the latest revision.
I know that compared to all you awesome folks, I'm just a lowly
user, but it's been frustrating to see nothing happen for months
with something that has a seriously high impact for a lot of people.
It's a shame, because the ntfs3 driver is miles better than the
current ntfs one, and is a solid replacement for the unmaintained
ntfs-3g FUSE implementation.
I thought it was FAT32. Do Macs support NTFS?
EDIT: It seems they don't out of the box: <https://www.howtogeek.com/236055/HOW-TO-WRITE-TO-NTFS-DRIVES...>
EDIT 2: Since many comments replied with the same thing, I'll reply here: I wouldn't consider read-only support as support for portable data storage, since that allows you only to send data in one direction. When you want to copy a file from a Mac to a Windows computer, NTFS is not a solution now (out of the box at least), therefore falls pretty short wrt "portable".
Trouble is , FAT32 has more OS compatibility.
Would be nice if Linux did have first class support for NTFS as it’s superior. But right now for maximum compatibility you need FAT32
it doesn't in spec, that's just the win32 API
NTFS filename length is limited to 255 Unicode chars.
You may be thinking of full path lengths of 32,767 Unicode chars but the filename component is still limited to 255.
If you look at the NTFS data structure "$FILE_NAME (0x30)" at byte offset 0x40: https://www.google.com/search?q=ntfs+%22%24FILE_NAME%22+byte
... there's only 1 byte used to describe the filename length. Hence max of 255 chars.
Wikipedia page about NTFS also mentions 255 max length for filename: https://en.wikipedia.org/wiki/NTFS
FAT16 is limited to 2GB, FAT32 can extend to 2TB (but the max file size for a single file is 4GB, as the parent stated)
1- download iso
2- format usb with fat32
3- copy the data
Apparently you can't just dd the ISO file into the USB block device like every other Linux or BSD ISO. It need to be FAT32 to be able to boot through UEFI.
The issue however is that the .wim file containing the OS installation source is bigger than 4GB. I think its sources.wim but I'm not sure. I had to install some command line utility from brew that did the job of "optimizing" the WIM file and managed to reduce it to a file bellow 4GB.
What does optimizing means? Better compression? Cleaning up crap inside the image? If their ISO tool probably does this without user interaction then why include a file bigger than what the FS can handle in the first place?
I don't know if this counts as some kind of dark pattern or MS being really antiquated in terms of bootable flash drivers.
I not sure of Android compatibility though.
Whilst at the same time missing a very important feature for portable storage - a journal.
I prefer having to replace a cheap flash drive more frequently than to lose data altogether.
Of course, everybody's use case is different and tradeoffs need to be considered.
Worked one day, then few days later plugged it in and it was all gone.
I kid, this is what I do. I've recently been backing up my family photos/videos to various media and I've been hashing each file and storing a hash file alongside the media to compare to in the future.
ExFAT is much more complex and AFAICT it does not have a 2nd copy of all of its data. [2]
Anecdotal evidence is that I personally lost data on ExFAT file systems, but also have had many customers with problems when they figured to use ExFAT for my backup products. If I see ExFAT in a support bundle than -by default- my suggestion for fixing issues is to reformat to a different file system. More often than not, that's the end of the open support case.
Still it is a shame that we're kept back literally decades when it comes to file exchange due to corporate culture polluting every corner of IT. Ensuring file export options plus file formats full documentation, should be among the most important requirements any software should satisfy to be even considered for professional use.
Alas you are right in that this culture is all over Corporate World (yes not just America). Vendor lock-in is something that's been with us since forever and I doubt it will ever go away. They might have gotten better at hiding it from obviously being spotted but that's about it and in some other cases it's still just as clear as always but the marketing people are still very good at selling it to customers.
Unsure about Android and ChromeOS compatibility.
I do know that FUSE MacOS stuff seems to be disabled in brew.
It's read only.
why?
As a real world example, I’ve had to help recover my aunt’s 5-year-old 2TB HDD that she dropped several times and had no backup of, and by helping I mean confirming she probably lost a terabyte plus of data unless she wants to pay five digits USD to pay to a recovery company for a way less than 100% recovery. She is a highly educated professional (a MD) and yet, it did not even occur to her that the drive will not only might fail, but rather, will fail.
To quite a lot of non-technical people an external drive appears to be the safest form of storage, considering its being physically visible and movable. The closest analogy is storing money under the pillow versus a in bank, in all the good and bad ways.
Plug an existing NTFS drive from one Windows system into another Windows system and you get a similar issue where opening a home directory requires you to take ownership.
The lack of permissions is why it works as a portable file system. If you've ever tried to use, say, ext for this it becomes clear why nobody does that.
NTFS is pretty much limited to internal drives on Windows PCs.
ExFat is great cross-option, though, without the limitations of fat32.
The camera world has been using exFAT since quite a while.
UDF as I stated works fine with udfclient under OpenBSD and it could be "mountable" even with base tools (read /usr/local/share/doc/pkg-readmes/udfclient in order to correctly format it with udfclient).
I copied files back and forth on USB drives written in under OpenBSD and the volumes mounted like magic under Linux. And the performance is not bad at all.
That would be either FAT32 or exFAT
I've never seen it used that way, guess I'm weird.
One hopes that the people who do use it will work on it, but given the patent fiasco with exFAT, I wouldn't either look at or use the code, personally.
Maybe things have changed now but NTFS was the only reliable option for that last drive when I first set things up this way a few years ago.
FAT32 needs to be used if you want to maximise portability
I suppose exFAT might be a viable option for larger external drives as well due to its higher file size limit. None of the other commonly used file systems (including native Linux file systems) seemed easy to use from both Linux and Windows the last time I looked into it.
are you high right now? 'cause you sound like you're on drugs when you say things like that.
If you wouldn't mind reviewing https://news.ycombinator.com/newsguidelines.html and sticking to the rules when posting here, we'd be grateful.
Computers are easier to deal with, there might be obscure programs that can read and write any known file system format despite what the kernel supports.
The kernel group has different priorities to end users
TDLR: Needful, worthy improvements require champions. A thankless role. And few of us have the resources and tenacity to serve the champion role for the long haul.
In trying to understand organizational psychology and apparent apathy, I keep thinking of Stephen Jay Gould's theory of "punctuated equilibrium" meets the attention economy.
Metaphorically, organizations (and people) are information processors. Attention is our scarcest resource. We're all doing triage, at every level. Just trying to cope with the torrent of information, discerning signal from noise. Trying to prioritize efforts.
So it's almost inevitable that needful, useful, worthy things remain neglected, undone. Even in the absence of pushback.
Until some kind of crisis. When the neglected thing suddenly becomes very important.
Cue the fire drill. Something must be done.
The champion must be watchful, ready, proactive. Seize that very brief opportunity to sneak in their needful improvement. Ahead of the all the spazs pitching their weird and wrong and ignorant reactive proposals.
The only winning strategy is to just keep pushing. Patiently, diplomatically. Be that useful thing that everyone's dimly aware of.
"Oh, wasn't Con Kolivas working on that mouse jitter thing? Ya, I heard it's pretty good. Ah, it's ready to go? Cool, let's do it."
If your needful thing is truly worthy, be confident that eventually it'll percolate to the top of the triage stack.
This game truly sucks. But I don't see how it could be any other way with our current decision making structures.
Do they? I don't think egress/ingress from external drives is an essential feature for consumer-grade NAS.
Internal drives are typically formatted with a Linux-native filesystem and accessed through SMB/Samba from Windows machines.
Is it not? I think most consumer NASes on the market support NTFS-formatted external drives (be that USB or eSATA) and need a r/w capable driver.
No, that would be exFAT and FAT32. Some form of FAT is basically supported by everything, which is why if anything is the "de facto" for portability, it's FAT. FAT is the netBSD of FS.
I guess politics gets in the way?
Is it? The kernel has a billion users who pay $0 for it. Most of the work is done by volunteers still. Oracle and RedHat have a few people looking at it full-time, and then Linus himself. This patchset is maintained by a third-party, Paragon, who also have an NTFS-for-Macs solution.
Hence:
>> We simply don't have anybody to funnel new filesystems
Also I see the new polite kernel is going well: https://lore.kernel.org/lkml/YO8LOKR%2FvRUgggTx@casper.infra...
(I don't have a clue what does Android actually use for filesystem drivers, but I would _guess_ it's just Linux down there.)
Yes, and almost all consumer routers and quite a lot of other random pieces of hardware. Hence the billions of users.
How many of those pay for upstream maintainers?
> I don't have a clue what does Android actually use for filesystem drivers
"Yaffs" or "sparse" (which is just fancy ext2). https://source.android.com/devices/bootloader/images ; most Android kernels will also have exfat so they can use SD cards and USB sticks.
Paragon, for better or worse, is just yet another corporation. They shouldn't really expect other corporations or volunteers to care much about their code, other than the fact that their code may break others'.
Now that they have Linus' blessing, they can just send a pull request. The work paid off in the end.
> FYI: There is also another cross-platform filesystem (Linux kernel, Windows NT kernel, Mac OS X kernel) suitable for hard disks too with POSIX permissions about which people do not know too much. It is UDF.
As I was struggling to find a way to share data between Linux/Windows with my gpu-passthrough setup, I'm excited to use this.
I think I've read something about 9p support in the latest Windows versions and I think QEMU supports 9p so that might become a good solution, too.
macOS will only access a UDF filesystem created on a disk device. Windows will only access one created inside a partition.
Linux, naturally, has no problem handling both. ;)
> Although an implementation may use a journal to protect metadata integrity, this does not guarantee interoperability between platforms since it is not part of the standard.
it seems it's left up to the implementation.
Obviously that needs to be enforced with some kind of "do not mount" option so it isn't accidentally used on another OS.
I usually use LUKS and EXT4 for all my drives but lately I've needed to move files to and from Windows systems on occasion, and it kills me that (from what I can tell) there are absolutely no solutions that even attempt to do something like this.
Does everyone just walk around with unencrypted portable media? I don't really want to float sensitive data on unencrypted drives, even temporarily, since that makes the data vulnerable to things like wear leveling analysis etc.
There is also --bootarea=mbr which sounds similar to what this script is doing but I'm not sure why it's necessary; maybe for older versions of Windows.
In plan 9 files are served by file servers, programs which speak 9p. The plan 9 kernel knows nothing of disks or file systems living on those disks. In fact, you don't mount disks in plan 9, you mount 9p connections. So to access files on a Fat formatted disk you need a file server program called dossrv(4) which opens the disk file and listens for 9p messages and performs the necessary Fat operations.
Fun fact: A common plan 9 newbie mistake is to mount a disk file directly as in Unix. This causes the kernel to write a 9p T_attach message to the beginning of the disk overwriting the boot sector. http://man.postnix.pw/9front/5/attach
I vaguely remember having to use fuse to write to NTFS partitions in the past.
ntfs-3g is a FUSE driver with read/write support, but it's FUSE, and not bundled with the kernel.
This new driver is (based on?) the commercial NTFS driver Paragon sold, and does not suffer the limitations of either, AIUI, and I gather the objections have mostly been around the state of the code rather than functionality.
edit: Having looked, the comment in the Kconfig makes it clear that they believe the limited write support is quite limited, but quite safe.
https://jp-andre.pagesperso-orange.fr/changelog.html doesn't seem totally dead, though the github repo hasn't seen much since March (but I don't think "since March" is what you meant by "last few years").