If I understand correctly, Google and Apple have a way for clients to request push tokens, but in order to use the token, you have to authenticate as the app owner, so a client unaffiliated with the server can't simply send a push token to a server and have that be used ... Instead you'd need that push proxy server.
That proxy server comes with concerns about reliability and privacy and etc, but if you do pushes without cleartext content (and you should!), the footprint is minimal and you gotta do what you gotta do.
I don't follow xmpp/jabber, but I'd hope there's already an XEP for this. It's a pretty apparent need for the last I dunno 15 years and it has a clear solution.
With the right agitation, maybe you could get sharable platform push tokens and skip the intermediary app developer pusg server, but that seems unlikely.
You probably also want something in the push system to help trace push problems. For small chat servers and smallish client push proxy servers, I expect minimal operations staffing, but when messages are being delayed because of push problems, you want to be able to let affected clients know that they need to be more agressive with periodic/background connections until push works again. You may also want to let users or the administrators of the proxy push servers know as well. Pushes might be initiated late on the origin, delayed between the origin and the proxy, queued for a long time at the proxy, delayed between the proxy and platform push, or delayed within platform push (including on device delays from Doze, etc).
You've got to be able to detect that so you can make adjustments and reduce delay. Messaging delay reduces user satisfaction.