One data point about btrfs, FWIW: I've been using it since 2016, the only "issue" I had is that it forced a failing drive into read-only, while other filesystems didn't detect the upcoming failure, thank checksuming for this.
The main missing feature for me is caching on faster disks, though there are some ways to get that. No mention of LVM nor LUKS either, I guess the understanding is either still a bit superficial (everyone has their own areas of interest), or they didn't want to complexify the setup too much.
For self-hosting, I can recommend https://yunohost.org/ which makes easy some traditionally hard parts of self-hosting, though it is quite opinionated (which isn't always a bad thing). Certainly saved me some time on maintenance, though I do see the limitations.
The holy grail of a "perfect homeserver" for me would be to run everything in GUIX, with a declarative configuration I can locally test before deploying, and rolling back. More packaging efforts are needed before this can become reality. I'd also like the option to have simple distributed storage, trough something like hypercore. This would help for backups, and for overcoming low-bandwidth links.
But maybe it's the Internet that needs to be fixed, with public key routing and content-based addressing, plus "heavy" clients and light, dumb servers instead of pushing everything to the browser.