I've personally seen read/write rates exceeding 800MByte/sec on more or less white-box hardware, at which point it was limited by the underlying storage infrastructure (8Gbit fiber), not the NFS protocols.
Dell has a 2013 white paper (I'm not affiliated with them, fwiw) about their fairly white box setup that achieved > 100,000 iops, 2.5GBbyte/s sequential write, and 3.5GBbyte/s sequential read: http://en.community.dell.com/techcenter/high-performance-com...
My only test case has been a VMWare virtual machine, mounting an nfs share from the host so I could work on my local filesystem and execute within the VM. Switched to a filesystem wather + rsync combo after struggling with poor random read performance. Maybe it was due to bad configuration but always thought it would be a poor choice for anything serius.
I've found nfs to be much faster and more reliable than sshfs or smbfs for VMs, using either qemu-kvm on linux or virtualbox on OS X.
Not sure how it would ever be technically possible for a networked filesystem to get even near directly attached storage. But, for sure, the typical carrier grade EMC or Netapp is MUCH slower then a good SAN. I'm talking about petabytes of very small (average maybe 20kB) files with lots of _random_ sync writes and reads. NFS has a lot of other benefits, but it surely is not super high performance in every usecase. Regardless of what a theoretical marketing whitepaper has shown in some lab setup.
Someone who thinks that you can put a network protocol around a filesystem without _any_ performance impact is nuts.
BUT if your usecase fits NFS you might as well get very good performance out of it. As always, pick the right technology for your specific case.
I think you might want to use your filesystem more effectively.
If you're using modern servers and clients, it is as fast as you can imagine a cluster of ssd nfs servers to be.
I've used NFS at home and have had NFS file handle problems but IIRC that was only when there were problems like kernel faults or network partitions.
However several of my colleagues at work have many NFS horror stories and are adamant that NFS does not scale well.
Is NFS stability at scale simply a function of your underlying network and infrastructure stability in your opinion?
Hard coding the fsid's on the server side can help if you change your exports a lot.
Using the automounter can help, although it does not scale to large numbers of mounts well.
Ordinarily, even if you unlink a file, the operating system keeps the inode around until the last filehandle referencing it goes away. But an NFS mount cannot know when all filehandles on all networked systems have closed. When you attempt to read from an NFS file handle whose underlying file has been deleted out from under you, BOOM -- `ESTALE`.
The solution is typically to guard against file deletion using read locks... which are extremely annoying to implement on NFS because of portability issues and cache coherency problems.
I'm not sure I'd describe that as a "scaling problem" per se, because it gets bad quickly and stays bad. It's more of a severe limitation on how applications and libraries can design their interaction with the file system.
I think that's limited to v2/v3 and not fully general or reliable.
Small files were much worse because they require a server round-trip every time something calls stat() unless you know that all of the software in use reliably uses Maildir-style practices to avoid contention. That meant that e.g. /var/mail could be mounted with the various attribute-cache values (see acregmin / acdirmin in http://linux.die.net/man/5/nfs) but general purpose volumes had to be safe and slow.
If you read through the somewhat ponderous NFSv4 docs, there are a number of design decisions which are clearly aimed at making that use-case less painful. I haven't done benchmarks in years but I'd assume it's improved significantly.
Oracle supports this- and they even wrote a user-space NFS client to "get the highest level of performance" (because they thought the kernel NFS implementation sucked).
The important bit is to ensure the NFS client and server implementation handle whatever POSIX features are required by the DB server.