When restoring from backup I went with Rook (which is a wrapper on ceph) instead and it's been much more stable, even able to recover (albeit with some manual intervention needed) from a total node hardware failure.
So far things are running well but I can't shake this fear that I am in for a rude awakening and I loose everything. I backups but the recovery will be painful if I have to do it.
I will have to take a look at rook since I am not quite committed enough yet (only moved over 2 things) to switch.
I have a 15TB volume for video storage, and it can't complete any replica rebuilds. It always fails at some point and then tries to restart.
I think I am likely keeping most of my storage just setup with a storage class that uses my NFS as storage. But longhorn will be used for the things that need to be faster like the databases. I moved jellyfin over to Longhorn and it went from being borderline unusable while metadata was grabbed to actually working well.
I can't imagine my biggest volume being more than 100gb, and even that is likely a major over estimation on my part.