Power is provided either using official RPi power supplies, or a couple of multi-port Anker USB power supplies that I had previously. A limit of 2.5 amps does not seem to cause any issues.
Currently everything is connected to a single switch, but I move things around my office sometimes, and sometimes have the RPis connected to two different switches.
Right now, everything including the 1 switch is connected to a single APC UPS, and that thing is super old, so that's another SPOF.
My clients currently are a few wired desktops and laptops over wifi, all connecting via CephFS. I haven't tested with librbd or krbd, I imagine it wouldn't be fast.
The RPis are mostly 8GB, but I do have a couple 4GB, and one RPi 400, which is kind of hilarious.
Everything is running Ubuntu 20.04, Ceph Pacific, and deployed from the first node with cephadm.
I use only Samsung microSD cards, either 32GB or 64GB. I don't think it matters what kind, but getting bigger cards makes me feel like they'll last longer. Most of the nodes have the var partition on the external drives (on a small partition at the beginning of the drive), but I do have a few where I didn't set that up early on, and haven't gotten around to redoing it.
I partition the drives and put LVM on manually, and tell cephadm to use the specific LV instead of the bare drive.
If you want any kind of performance, definitely set your expectations very low, but for me this works. I can stream at least a pair of 4K movies off this simultaneously, and I also run an instance of Paperless-NG off this over a CephFS mount and haven't had any issues.