Direct correlation: imap use and tearing my eyes out

What has been your experience with using imap (e.g. gmail and outlook)? I gave it a shot and was trying to be smart about it (e.g. not downloading spam folder). However, I found that performance was very slow. I'm thinking of going back to pop or just using the gmail client.

Looking to understand other's experiences and maybe learn how to improve performance or other approaches to utilizing IMAP. have a good one.