Does anyone know why NFS is such a pain? In the past (10 years ago) I just assumed I was doing it wrong and stopped using it, and have not used it since.
Towards the end of my tenure there, I gave a Linux desktop a try. The NFS experience was amazingly bad by comparison; lots of issues with locking, with becoming disconnected (often until a reboot) from NFS servers, odd performance issues, reliability issues with the automounter, etc.
In the last few months I have tried the NFS client on my current Linux desktop again, thinking things might have improved -- they have, I guess, but not by much. It's still pretty easy for the client to get into a hung state if there's too much packet loss, or if the file server reboots, or whatever. I have to imagine that not enough people are really using Linux NFS clients in anger to drive fixing the issues with it. There is often no escape from the Quality Death Spiral.
Depends on what you're going to do with it. For something like sharing home directories, it works well enough.
The defaults are usually pretty decent. There's unfortunately a lot of obsolete NFS tuning advice hanging around on the internet that seems to get cargo culted over and over again.
Like the advice to set some specific rsize/wsize settings because the default is too small, oblivious to the fact that the NFS protocol allows the client and server to negotiate maximum sizes, and at least the Linux client and server have taken advantage of this negotiation mechanism for the past 2 decades or so.
How informative.
> Having worked in high volume, highly available environments for years soho scenarios are not good examples.
FWIW, I wasn't talking about SOHO. At least in my experience, defaults work well for home & shared work dirs for O(10k) users (not all simultaneously active, though). HA is a pain, though, if you want to DIY, I'll grant you that.
> Real world issues with complex NFS environments (mixed nfs3/4 + krb5p and multiple OS'es + automounters)
Complex? Sounds like a pretty standard NFS environment.
> or pNFS and gluster require more than tuning mount options.
Yeah, no personal experience there. What did you have to do there?
We did have a clustered NFS appliance for HPC use a decade or so ago. People like to complain how Lustre is a beast to run, but IME Lustre has been smooth sailing compared to the grief that POS gave us. But that wasn't really the fault of the NFS protocol per se, it was just the architecture as well as the implementation of that appliance was crap, particularly so for HPC.
Trying to do anything like a database (and the 'git gc' process described is exactly that, a tiny database) over NFS requires the use of very specific techniques to get right.
Sibling commentator has it right - for unreliable WAN networks S3 offers far better semantics, because it's not quite a filesystem.
Firstly NFS only really works reliably when your network has harmonised UIG/GIDs. Thats the first pain point. This normally means LDAP/AD or shipping /etc/passwd (_shudders_) Also you need to squash root, otherwise people who are local root can do lots of naughty things.
Then you have to make sure that your mountpoint doesn't go away, because stale file handles are a pain in the arse.
Then you have file locking, which causes loads of other pain aswell. Most people turn that off.
After that its mostly alright.
nfsv4 has certain things that are good (pNFS, kerberos, etc) but support was not that great.