I know, I'm a sysadmin. I care for hardware, mend it, heal it, and sometimes donate, cann-bird or bury it. I'm used to it.
> worst case scenario is an host reimage...
While hosting PBs of data on it? No, thanks.
> Would I build a file storage system around btrfs? No - without proper redundancy at least.
Everything is easy for small n. When you store 20TB on 4x5TB drives, everything can be done. When you have a >5PB of storage on racks, you need at least a copy of that system running hot-standby. That's not cheap in any sense.
Instead, I'd use ZFS, Lustre, anything, but not BTRFS.
> I can live with that - applications will generally break production ten billion times before btrfs does.
In our case, no. Our systems doesn't stop because a daemon decided to stop because a server among many fried itself.