We did NBD in preference to everything else because NBD is so simple, and is the kernel's de facto answer to "abstract a block device away from the hardware". Shaun ran into reliability issues with it (which, for all I know, were just bugs fixed in more recent kernels) and iSCSI was within arm's reach, so it wound up being iSCSI.

We could do a bunch of testing and find the performance gains of NVMe-TCP worth the switch-out, and that project would be tenable (though: big enough we wouldn't do it unless the win was really big).

A serious NVMe-OF deployment would be tantamount to us building an EBS-style SAN scheme, which almost certainly would require us to single out specific regions to get the "better"/"newer" disk storage in, which is something we haven't had to do yet. I think in the next year or two we're more interested in seeing how LSVD, and especially Tigris LSVD, plays out.