First, the overall footprint of an RPi5 is way smaller, plus RPi is easier to cool passively than an N100, also it allows RPi5 to be lower maintenance on the longer run.
Most N100 are noisier, hotter and bigger than a Pi5 (I have a smaller N100, but it's still actively cooled).
When paired with a decent A2 card (e.g.: Kingston Canvas Go), the SD-card related lag disappears. Add a good external SSD, you have tons of solid state storage. At that point you can just forget it.
I like my N100, and companies like GMKTec routes 2 PCIe lanes to NVMe slots allowing ~1.5GB of throughput, but for most home use, that's overkill. I rather use Pi as a server and N100 as a desktop (which is exactly what I do).