Sorry I should clarify, by "it" I meant any sort of ping must go through Firebase Cloud Messaging, not that the message content itself goes through Firebase.
Looks like there is a way to bypass Firebase by using something like UnifiedPush which runs a perpetual background process that acts similar to Google Play Services to pick up notifications from the server and calls the local notification API.