Not resilient on a system level, but refilling the cache is cheap.
ZFS was generally pleasant from an operability viewpoint once we ironed out the quirks, but the perf hit from no sendfile was too much.
It’d be really nice to see that fixed like the recent DIRECT_IO additions.
(I'll just point of that using sendfile means that traffic is unencrypted... which is probably fine on an internal network, but I've started adopting the stance that even internal network traffic should be encrypted unless there's a very good reason not to do that. An absolute requirement for performance might be a good reason.)
Now I’m more curious about the actual threshold where not having sendfile begins causing noticeable performance problems… at what point before you become Netflix?
Of course sharing resources between a couple services would be good, as NICs and switch ports are sill a way from free.