Gotify – a self-hosted push-notifications service | Hacker News Reader

Gotify – a self-hosted push-notifications service

github.com

490 pointsadraenwan96 comments

It is worth noting that this will use a noticeable amount of power more than Google cloud messaging. The basic idea of how is works is about the same, but the devil is in the details. If you look at an idle connection over LTE, you will notice that after a while your carrier will close it. You need keepalives. If you look at the connection for Google cloud messaging, you might notice that it needs much fewer keep alives than YOU need to keep your connection alive. This means fewer wakes from sleep mode and lower power.

Is that because carriers have put GCM/FCM on some sort of white-list?

Probably because GCM/FCM has been very well optimized.

While it's likely true, GCM/FCM is handled by Google Play Services, which is whitelisted and has I believe system-level permissions. It has less restrictions than a regular app exempt from battery optimisations, as it's supposed to be the only app that needs to run in order to coordinate things like push messages or location updates across all apps on the device

edit: I missed that the original post was talking about keep-alives and you were probably referring to them. But how app-level optimisations can help with keep-alives?

TCP connection can be open and idle for hours without keep alives. This can easily be configured with kernel parameters:

https://www.tldp.org/HOWTO/TCP-Keepalive-HOWTO/usingkeepaliv...

I suspect the hard bit for normal apps is keeping the app active on the device while it is in the background. Google has their FCM receiver running as a low level service so can make sure it is always running.

You can configure it locally in your kernel, but your ISP will most likely close your connections for you, especially if you're behind a CGNAT.

Even if you have native IPv6, some carriers do connection tracking anyway to prevent random port scans from the internet waking up your phone or trying various exploits.

Yes. Google/Apple have special deals with carriers.

https://0xacab.org/leap/bitmask_android/issues/2226#note_921...

https://news.ycombinator.com/item?id=17946320

Not only that, but for many manufacturers this will straight up either not work after the app has been in the background for some (not long) time, or will deliver (that is, app will be able to connect and receive) push notifications with significant delay. Sadly, most reliable way I know to trigger push notifications when app is closed is using FCM - I'd love hear if there's an alternative, though.

Doze mode nukes network connections that are not FCM.

In my experience it's not bad enough to worry about.

I have:

- Signal - Whatsapp - Etesync - Conversations.im - Syncthing

all whitelisted from battery optimisation, and I've not noticed any huge drain (I expected it to be unusable and have to go back to google play services or micro-g).

Signal uses gcm to push to the phone thus waking it so it can download any messages. Not sure about the others though.

If you don't have Google Play Services, it doesn't use GCM.

Awesome! I've been working on a collection of news source parsers [1] (a combination of RSS and HTML scrapers) that this will work perfect with to replace my native news app. All I have to do is pipe the lines from my parser through a Python script issuing these message POST requests and it just works out of the box.

[1] https://github.com/wybiral/stream-sources

I gave it a quick try and "it just worked" with minimal configuration. Really easy to get started.

Unfortunately messages are only plaintext.

Use JSON

Same. I wasted more time this evening setting up octobercms behind a reverse proxy (hours, not working) than getting that server handling notifications from a VPS running many small services.

Although I've been using Pushover[0], a paid service, instead of Gotify, I found a really cool project to pair a push-notification service with is huginn[1], which allows you to program bots to do things on cron jobs, among other really powerful actions.

[0] https://pushover.net/ [1] https://github.com/huginn/huginn

Just to clarify, Pushover is not a "paid service" in that you need to pay for a subscription. You only have to pay once to use it past the 7-day free trial.

https://gotify.net/

I would like to point out that there exists solutions tested in the field in the last 20 years to solve the same problem.

In particular I would suggest to get to know MQTT an open protocol with several (open and close) implementation that is know for its flexibility and scalability (if nothing has changed it is used to power the Messager Facebook platform, but don't quote me on that).

verneMQ is the server implementation I would suggest if you are interested in such topic.

MQTT is just a protocol, not a service. I don't think it was the developer's intention to provide an alternative to MQTT, but to offer a solution for push-notifications, which just happens to be built on top of WebSocket without MQTT. Also keep in mind that MQTT will usually get wrapped by WebSocket to avoid issues with firewalls or to easily get TLS via a proxy server like NGINX, or simply to add web browser support. VerneMQ is not the only server implementation for MQTT.

Depends on the exact problem definition. This is a small, single-binary server, offering a built-in web UI & API, requiring very little configuration, working with a mobile app for Android phones. Took me 5 minutes to start getting some notifications from things around my house.

Sometimes you don't need a cargo van or a jumbo jet to get the job done when a skateboard will do.

Interesting project. Is this intended to replace Firebase Cloud Messaging on android or be a wrapper in front of it?

I only saw android support, anything for iOS in the works?

From what I can tell it appears to be a replacement FCM/APNS. The client subscribes directly via websocket to a self-hosted server (Go binary).

New messages in the queue are then notified (gotified) via the websocket. https://gotify.net/api-docs#/message/streamMessages

The payload can be picked up and managed via REST API as well.

I'm building a similar solution for a slightly different usecase, so wanted to have a look under the hood. There's nothing I can see that would stop you using this for iOS so maybe they just don't have a iOS dev on the project.

Edit: Ah, here's why. https://github.com/gotify/server/issues/87

Thanks for clarifying. This could be a good solution for a non Google play type phone, or just someone who doesnt want t rely on Google servers. It would just mean in the latter case there would always be 2 open connections listening for messages...

Those open connections would be very hard to keep open indefinitely in an Android app that gets optimized or goes into sleep / doze mode.

So, their conclusion in the end that it is technically impossible to build a self-hosted notification solution on iOS due to Apple's restrictions. Shame because I would love to be able to deploy something like this internally at work, but of course, half the business is walking around with iPhones. I'm kind of curious though why it doesn't ruin every single XMPP client out there - there seem to be a few on the App store.

Either the server is sending pushes through APNS or the client is simply not getting messages reliably when the screen is locked or the app backgrounded, as the network restrictions very much apply to those clients as well (and to everything, to save power by sleeping the NICs).

Funny, I was just looking for an alternative to pushover that doesn't use google services last week. Tested this and it worked quite well.

I actually wound up just using an SMS gateway provider with a simple API you can just fire a get request to.

Sending a text message uses the GET verb? Yuck.

I get why that's "yuck" but if I'd hedge a bet...it's probably done because it's easier to make a GET than it is a POST or PUT request and the length of a text message will never exceed the query string uri length limits.

Not saying I disagree, but i don't know it changes a whole lot in this case.

In this case maybe, but generally GET should be avoided for any creative/transactional action because many systems will replay/cache GET requests, assuming that they’re idempotent.

Yeah the caching issue is a great reason not to use a GET, i was more just suggesting if you're being lazy and pasting a request into a browser v's using a command line (which obviously you should) then it's easy to get going quickly to kick the tires.

just to point out: there are no uri length limits. browsers typically have one (I haven't checked lately tho), but other tools generally don't (except by mistake / size-capped buffers).

Not sure why you believe this so strongly, but every single HTTP client impose some limit.

Modern browsers are typically around 100-200k limit, command line tools such as curl and wget also has their limits.

IE was famously limiting GET requests to about 2k bytes.

the spec does not define a limit, and the widely varying implementation-specific limits are soft evidence of that.

even wget and curl are probably more limited by your CLI arg length limits than the bins themselves. e.g. I can craft a multi-megabyte GET request in a file and pass it to `curl -K` and it works just fine (I just did so to verify, a bit over 3MB. google complained a bit, but it responded). even if I screwed that up somehow / it silently truncated, I can absolutely do something with netcat and know it won't truncate.

How is a GET request easier? I assume you need to include authentication somehow, so it isn't like you are triggering it by typing the url in a browser.

If you really want to, you can in fact issue a POST with a query string and no body. It would admittedly be rather unusual.

Also, application/x-www-form-urlencoded bodies are literally identical to query strings anyway, so if you can construct a query string, you can easily construct a POST body using that Content-Type too.

I had exactly the same problem, and ended up using Signal. I like it more than normal push notifications because you can reply and have a full chatbot running.

I liked it so much that I even wrote a small Python library for it: https://gitlab.com/stavros/pysignald/

That's pretty neat, but could you add an Open Source license to it? It's gonna make folks hesitant to use it.

Thanks for reminding me, I forgot to add the file. The license is actually MIT, as you can see on the PyPI page (https://pypi.org/project/pysignald/).

https://pushpin.org/ is another alternative that has some history, has support for more protocols and is used in production by its creators

FCM is integrated in kernel, so it cannot be killed. And it starts a persistent TCP connection to Google server. If all apps uses FCM, then all notification would route through this connection, and this significantly reduce battery consumption than the alternative, where each app uses its own web socket server.

So, assume using a degoogled android, can this notification replaced FCM if more apps use this notification server? This server uses web socket, so what does FCM use for its connection? Does web socket battery efficient enough?

>FCM is integrated in kernel, so it cannot be killed.

What do you mean it's "integrated in kernel"? Are you saying google play services includes a kernel module?

It does not.

It's a plain-old TCP connection in GMSCore, but has special permissions to ignore various power save modes.

I personally wish Google would work with mobile networks to replace it with something lower level (eg. Based on the same thing used to initiate phone calls), because a hanging TCP connection open for many hours gets unreliable as NAT drops it and as the phone migrates from one mobile network to another (eg. Across country boundaries)

That was how BlackBerry worked wasn't it?

It's handled by Play services which is whitelisted to allow network access when the device is in light/deep doze mode.

Play services can also instruct Google's servers to stop sending notifications when the device is in deep doze and re-open the connection when the device is active again -- thus saving power (messages are queued at Google's servers)

FCM uses XMPP

https://firebase.google.com/docs/cloud-messaging/xmpp-server...

I really like the idea of Google providing a single, efficient connection for push-notifications, but having it bundled with Firebase (and therefore with Google Play Services) is an unfriendly approach.

XMPP is only one option for communicating with the FCM server. The most popular method is using HTTP/REST. XMPP is only needed when you have bidirectional messages - server to client app (downstream) and client app to server (upstream).

The documentation states that End2End tests are included. I didn't find them in the repository. I want to learn from the tests - Can somebody please help me and point me to the End2End tests?

There are more repos on the parent account. They may be in one of those.

You can find the ui end2end tests here: https://github.com/gotify/server/tree/master/ui/src/tests

Is there a standard understanding of what are push notifications? Are they the same for android and iOS? I’m not sure what this project is about!

https://en.wikipedia.org/wiki/Push_technology

Right but as the article shows, there are many "push technologies"... what is this project about? Someone subscribed for notifications and then this server sends them?

iOS does its own thing, FWIW: APNS (https://en.wikipedia.org/wiki/Apple_Push_Notification_Servic...)

Thank you!

> what is this project about? Someone subscribed for notifications and then this server sends them?

Exactly that. A device registers with the server and then there's a REST API or CLI binary that you can use to send it push notification messages.

I think Push API is an industry standard but there is this W3 draft https://www.w3.org/TR/push-api/

But yes, it's essentially implemented the same across iOS/Android and browsers like Chrome, FF, Edge etc. There are services such as Amazon SNS or Google Firebase that provide a single interface which can talk to all the slightly different implementations.

In a nutshell the device registers to the server and receives an identifier token. Then a server can send messages to that identifier and the message will be displayed on the screen. Some providers allow for Icon, Audio and some Actions to also be sent as part of the message payload.

Thank you! I’ve never actually implemented push notifications. Will soon so I was wondering...

I like the idea of the project, but I'm curious.

Why would you need a self-hosted (or really, any other push-notif service) other than the official one from Apple/Goggle?

They're optimized (for each respective platform), no need to maintain a server, and free. If you need "cross-platform", you can use Firebase(?) that abstracts that away.

Privacy? Keep in mind that not everyone intends to build a user-supporting platform on this tech. I like the concept because it's an easy way for me to push notifications from my homebrew IoT stuff and news streams without it flowing through ad companies.

You can encrypt notifications that go through Google's FCM and Apple's APNS push notification servers. This is how Pushover works - each notification is encrypted with a device-specific key, sent through the notification servers as an encrypted blob, then the app on the device is woken up and receives the blob, decrypts it, and posts it as a normal notification.

You could. But if you don't need that kind of added complexity and scale (and I don't for personal news updates or IoT device alerts) then why jump through those hoops?

I think the major reason this was created was for apps on the F-Droid store and for people who prefer not to have Google Play Services installed on their phone.

Consider that some countries do not have a reliable access to Google backend. The problem remains when trying to use notifications with a PWA. AFAICT, there's no way to set your own notification service in Chrome/Firefox/Safari.

Is it true that GCM is centralized and battery efficient, one push service per app will drain phones quickly? Or is that less of a concern nowadays?

It's still a concern and this service will increase battery drain - this is why all the users of apps using this service will have to explicitly exclude the apps from battery optimizations.

I don't see Google Play allowing these kind of apps in massive numbers though - applications abusing background processing to poll for messages were one of the main reasons why Android got blamed for being worse than iOS in standby battery consumption. Warranted or not, Google/Android gets blamed for abuse of power saving at the end.

Bah... no linux 32 bit binary for my old home-automation server.

just build it yourself :)

$ git clone https://github.com/gotify/server

$ cd server

$ env GOARCH=386 go build

This is what home assistant needs! The only really lacking thing about HA is the lack of native Android app, but the most important thing is notifications.

Neat. I could definitely use this for some home infrastructure I'm playing around with at the moment - will definitely give it a shot!

Have anyone used Nats as a push notification service? We are looking for alternatives to Google Play Services.

Hmm, I didn't see anything about security while skimming the docs?

Awesome! This opens the door to so many possibilities!

Logo reminds me of traefik. Is it related ?

That's the Go Gopher: https://blog.golang.org/gopher

Thanks ! That mascot is super cool.

I cannot tell is this is a sarcasm.

It’s the go gopher... gophing about.

It's not :).

Useless without iOS support. What’s the point of an abstraction that has only a single implementation?

I'm not sure, but I don't think iOS lets apps auto-start and run in the background forever. This would be needed for any alternative push notification app, else it would be too easy to miss notifications.

As stated previously: this is useless.

What is the point of building alternative push notification services that can’t even properly rise above the bar of technical feasibility?

It doesn't, but iOS will wake your app up when it receives push notifications.

just write a Twilio or Slack plug-in?

I'm not up to date with Android, but why does displaying notifications require an entire server or a paid service? Is it not just a matter of installing a client program and sending a curl to an endpoint?

It is not needed for displaying the notification but specifically targeting users. You would usually call an FCM function to get a token, which can be used to specifically target that user and is stored in the database. To send a notification to your customised target audience based on user-generated events, it becomes essential to use web-service that does it for you.

For example- Let's say you wish to send a notification to user A when user B sends him a message. In this scenario, you can call another FCM fn, using the token user A had generated. Of course you can use the web dashboard to send the notifications, but in this case, you would not be able to send event-based notification reliability and in real-time. Other services also work in a similar manner.

Of course FCM also has notion of channels where you can broadcast to certain topics (which apps listen to) without having to track tokens on application server side.

You need a way to tell the mobile device that a new message has been received. Since you don't have a fixed IP or anything there is no way to directly send the message.

An alternative is that the mobile device pulls from time to time, but that's quite power intensive or, when using a long interval, is slow.

Instead one has an idle connection open to which the server can send messages.

Sending a curl to an endpoint from the client is not push but pull. The idea of push notifications is that they arrive immediately.

> I'm not up to date with Android, but why does displaying notifications require an entire server or a paid service? Is it not just a matter of installing a client program and sending a curl to an endpoint?

Imagine if every program on the users phone had their own schedule for checking for updates against remote hosts. That sounds terribly inefficient and wasteful of power, as the applications would need to "wake up" (as in, consume processor cycles) to initiate and perform a check.

Google's solution - Firebase Cloud Messaging is for the most part free and fairly easy to setup. Yes you can use curl to call their HTTP/REST endpoint to send a notification I believe.

If you get to the stage of sending millions of notifications per day then you'll need to start paying,

Off-topic

I got so scared when I looked at the swagger. The authentication uses the exact same variables as mine and is somewhat similar. The name of the endpoints are also very similar and the project has resemblance to mine.

That was really really akward for a moment.

I'll use the project though ( different use-case), looks really interesting. Coincidentally it's for the same project as I mentioned.

Reply on news.ycombinator.com