If there are any reasonably-priced ways of doing this especially with more PCIe lanes please let me know. In my experience it's often easier to just buy more motherboards and CPUs than invest in PCIe expansion.
https://www.microsemi.com/products/drivers-interfaces-and-pc...
The Dolphin systems are tuned better for computation: http://www.dolphinics.com/products/IXS600.html
I expect small PCIe NVMe external storage systems to become quite common in the near future, because enterprise systems need multipath storage for reliability; 8GB/s FC is too slow for SSDs, let alone NVMe, same with SAS bus expanders.
https://events.linuxfoundation.org/sites/events/files/slides...
Would require some modding to use more up-to-date GPUs (there are some mounting studs on the chassis floor that need trimming) but the forced airflow means that passive cards like the original Xeon Phis are also a possibility.