Just wondering: why don't you use your phone's native email app and IMAP?
Another reason is the lack of push notification (not sure if i'm using the right term) when using the built in gmail app for fastmail. It only gives me the option to poll every 15 minutes. The gmail app also doesn't support actions like snooze, swipe to archive etc on non-gmail accounts and I like having that in the fast mail app.
I'm doing exactly this from Android with Aquamail.
said android app supports generic IMAP accounts and doesnt need to be paired to a google account to work with them.
It's not usable for real work, and I'm pretty sure that's how Google likes it.
Fastmail's iOS app does support this functionality.