I've been using them for light to moderate desktop use for years now. The important things in desktopping a Pi, as far as I'm concerned:
- Fix the storage. The SD card is fine for a toy. It will not withstand actual daily use, either in "delivering sane performance" or "handling a lot of random writes without dying in a year." Get a USB to SSD adapter and a cheap 32GB or 64GB SSD from eBay - doesn't have to be fast, doesn't have to be new. It's radically better than an SD card in any use case that resembles desktop use. You can either boot straight from USB, or, to improve compatibility with weird USB to SSD adapters, boot from the SD card, mount root from the SSD, and use the SD card for swap or something (see below).
- Enable zswap if you're not on a 4GB or 8GB Pi4. It should be a module included in the Pi kernel now (I had to argue for a while to get it included). This is not zram - this is zswap. There's a huge difference. It's a compressed swap system that can flush out old or poorly compressible pages to the actual swapfile - which, since you're now fronting it with a compressed swap region in RAM, can be over on the SD card. The bulk of the stuff written to it will never be used again, unless you're just massively overcommitting RAM, at which point nothing really will help. I think around the 5.x kernels, z3fold starts working, though I've not seen a real practical difference between z3fold (up to 3 compressed pages per page of RAM) and zbud (only two compressed pages), as long as same filled pages are enabled - that will crunch a page of 0s down into a "Got it, all 0s, done!" note and not bother wasting space compressing it.
- Set the governor to performance. 'sudo cpufreq-set -g performance' The stock governor is really bad about ramping up, and leads to lag in typing, especially if you're using the atrocities that are modern "desktop" apps (Electron). Eventually, the system will spin up and do something useful, but you improve the performance and responsiveness rather substantially by just pinning the cores to their fastest speed and letting them stay there. I'm sure there's some measurable power difference doing this if you're concerned about the absolute lowest power, but it's not substantial, and "race to idle" solves a lot. If you're using a Pi as a desktop, try it - you'll like the change.
- Use adblockers. It's amazing how much CPU time the internet spends on stuff that is advertising, tracking, and generally evil. Most websites behave a lot better once you remove that garbage.
Otherwise... once you've done that, they're actually quite capable little systems! The Pi4 with 4GB or 8GB is the best option right now, and for most people I'm not sure the 8GB really gains you much - I have one, use it fairly heavily, and rarely see any RAM pressure past about 4GB. But it does make up for some slow disk.
I've written about some of this more extensively over on my blog over the years, if anyone wants more details - link is in my profile.