This is absolutely true -- I'm on a nexus 5/android 6.
There's no native caldav. I use a paid client called caldav-sync which is mediocre. For a while on android 5/lollipop, you had to install a second app (free from the same author) to prevent settings being wiped during OS upgrades/patches. When I upgraded to android 6 I couldn't figure out why calendars weren't syncing until I discovered caldav-sync decided to sync every 6 hours instead of every 5 minutes. The whole thing feels and works like a hack.
Using the gmail app as an imap client for fastmail is crap, particularly if you access the email account from a desktop. The Gmail app regularly desyncs from the state of your email. It has recently decided to announce old emails as new. If you move/delete an email in fastmail from the web client on your desktop, you have to manually tell gmail to sync or it won't notice, even hours after the fact.
I was hoping for a cheap nexus replacement and was also looking at the OnePlus 3. Dual sims are nice too. That's too bad that you don't like it...