I run ZFS on Ubuntu (ZFS is superb and one of the best things I've learnt in years) with a RAID 10 setup (2x2TB mirrored pair, 2x4TB mirrored pair). I'm actually just resilvering one of the 4TB drives right now as I finish migrating to them from a couple of old 1TB drives. RAM usage was high when I used dedup but now I've ditched that it's running much lower. I think the hardware will start to struggle once I get up over 10TB of usable space, but considering the cost of the actual machine I really don't feel like I've wasted any money at all.
I picked RAID 10 for this because I feel like with only 4 drives available it makes the most sense, weighing up resilver speed vs slightly better failure recovery if it was RAIDZ. On bigger servers I tend to go for RAIDZ3 instead.
For backups I use syncthing on various machines to copy data to the server, and then rsync the stuff I care about every hour (with a lockfile to stop overlap) to rsync.net, which then performs all the snapshots. I used to run borg but I swapped back to plain rsync because I felt more confident in being able to restore from it. rsync.net is fantastic. I'm currently storing about 1TB with them, and although it looks more expensive than other options for home backup, I feel like the lack of hassle is really worth it for me.
I'm not usually a fan of HP btw, but these little microservers are really well built and I actually ended up using a ton of them at work as well; we only ever had one failure (which I believe was due to PoE somehow being delivered to it, which was really fun to debug :|). When this one eventually can't keep up I suspect I'll just go for a newer model of it. The form factor is very convenient for the home, especially if you don't have the convenience of a spare room in which to put a rack.