Is this going to be any better than using an SSD with UAS which is what I'm doing already? (https://rwmj.wordpress.com/2020/09/24/raspberry-pi-4-running...)
There's two modest perks to NVMe over SATA. First, the drives tend to be faster. For large reads/writes, this probably wont help, as you'll be bus limited to USB3 speeds. But for smaller reads & writes, the NVMe drive can maybe probably do more iops, and you may see wins & better latency on workloads. NVMe also has a bunch of under the hood improvements. There's a quite in depth comparison of the two[1]. Larger queues, multiple queues, less round trips to issue commands, better completion model. I am very much unsure of how many of these advantages are exposed & would carry over when the host-to-device connection is still UAS based, which is what adapters like the one in this article still feature, same as if it were SATA. But some of these wins will help latency & iops again vs a SATA drive.
[1] https://sata-io.org/system/files/member-downloads/NVMe%20and...
It's a straightforward technology to implement and I believe it is now royalty free. The only implementations are Intel's though and I'm sure they're expensive.
Expansion is a complicated business model. Apple doesn't even ship headphone ports. The AirPods are smash hit, not least because when you sell a $1,100 device a $100 accessory suddenly seems really well priced. PCs are similarly pretty high ticket items, that market supports a situation where spending $100 on custom internal PSU cabling is not that bad.
The Raspberry Pi is tough. Accessories cost more than 2x the device. It's a model that doesn't really make sense. You could say that people are buying these accessories, but really its greatest limiting factor is that it's too cheap, it's hard to psychologically deliver on the big ticket item + accessories shopping cart for it.
Current Intel host-adapters seem to be ~$10 for the dual-port chip itself[1] (JHL8540). Not sure how much else it adds to the bill-of-material beyond a regular USB port! Adding it direct to the CPU would be a wonderful way to save space, & significantly reduce how hard it is to implement, since one would no longer need to route USB and PCIe and power to this extra chip.
I wouldn't expect Thunderbolt support on the low end for a long time. But I really do think it's really time that cpu makers begin offering it onboard. They already need to start moving towards USB4. Adding PCIe as well would be a courtesy, & a big enabler, for their customers. Hopefully we see these great awesome futures begin arriving in a more well-distributed manner.
[1] https://www.mouser.com/new/intel/intel-8000-thunderbolt-4-co...