Push Notifications for Decentralized Services
unifiedpush.org
unifiedpush.org
Now, I actually think this is a little too easy. What if I want to keep my ntfy server private for certain at-home uses? But still, very impressive. Element does it right, it at least asks you when you open the app (or it's in a setting or something). But I think UnifiedPush or ntfy instructions should make these steps more clear (or I failed to find it), even if just to say "it's actually that easy" to save our time looking.
BTW I'm working on porting ntfy to Sandstorm. Already works, I just have a handful of loose ends.
Oh this would be really handy, I'd love to run it on my Sandstorm server!
One major caveat is that because of Sandstorm's draconian security (I mean this in a good way), a handful of ntfy's features won't work because they use special headers. Mastodon works so I'm assuming they use the json API. One of my tasks is to go through major services and test them.
Another one is protected topics. "Users" use auth headers (there's a GET string option, but the Android app source code indicates that it opts for headers). Sandstorm does give you the benefit of randomized API URLs that you can revoke, which protects you from random actors, but any service (Mastodon, Matrix, etc) you connect to will need to see that URL so they may in principle snoop on easy-to-guess topics.
If you're curious you can see my tangled mess of notes: https://github.com/orblivion/ntfy/blob/sandstorm/.sandstorm/... Don't get too invested in understanding it if it's confusing, it's a fucking mess.
Is it about hosting apps on different domains and using a powerbox between them? Or is it something more? Why does sandstorm use capnproto for instance?
In our framework for instance, instead of capabilities, we implemented "invites", which people can send to each other. If an invite grants you readLevel access to a document, you'll be able to see it even before you log in and create a user account. But if you want to get the full access to write etc, you'll have to create a user account and accept the invite as that user.
https://github.com/Qbix/Streams/blob/main/classes/Streams/St...
I'm not an expert, but my understanding is that sandstorm takes all of the system calls from your apps, and uses capn proto to serialize those calls to the powerbox. The powerbox then checks to see if that operation is allowed for that user, and if so it executes the call, and serializes it back to the app (which then displays it to the user, for example).
Sandstorm uses CapnP because it's technically sound, battle tested, and efficient.
You could just make http requests back and forth (that's kinda if the default for web apps) but there is latency overhead and way more surface area with http.
My favorite explainer on the object capability model is... the first half of this exceptionally long page: http://habitatchronicles.com/2017/05/what-are-capabilities/ It's worth the read, but read it when you have time to sit down.
So access to apps or documents by users, as well as those apps and documents being able to reach out to other resources, is all generally wrapped in capabilities.
I think as a self-hosting platform, the biggest advantages of Sandstorm is that we force developers to provide a no-config-file, no-sysadmin-required experience, so Sandstorm is probably the only self-hosting platform currently intended to be used by people who are not tech people. This naturally flows into some other things: If you don't have a sysadmin running it, it needs to be really secure (capabilities) and very good at self-managing it's own performance.
So Sandstorm apps do not consume any resources unless you are accessing them. Everything is spun up and closed down on demand. And the platform's security model not only means vulnerabilities in apps mostly aren't exploitable and apps don't need to be patched, but Sandstorm strives to safely allow non-technical users to install and run potentially even malicious apps without risk.
We are working on the documentation: https://codeberg.org/UnifiedPush/documentation/pulls/16
What is old is new again (Google used XMPP for Android Push Notifications before they switched to something else).
This is a key point right at the very end. It has always been much easier to implement push notifications to Android devices, even when only Web Push is implemented. Google has been the main force behind the adoption of PWAs as Native apps on devices, even by immediately showing a "Install this app?" popup when navigating to a PWA-enabled website/web app.
iOS permits "Installing to home screen" but there is no popup shown natively, nor any indication from the OS that the website you're visiting is a PWA. The user has to go through several steps to add the app to his device.
Once it is added, it's possible for the device to receive web push notifications and show them as native push notifications (IIRC), even with the app is closed, however the reliability is inconsistent. Sometimes you get them, sometimes you don't. At least that was my experience still about 6 months ago when I tried implementing web push with PWA for iOS.
I gave up trying to go the pure PWA route and instead wrapped my web app with PWABuilder into a native app, and went through the whole process of submitting to the App Store, getting APN, registering the web app in Firebase, then sending the push notifications to the native app via Firebase Messaging. Works great then, but it's a lot more work than keeping everything simply as a web app and using web push.
Web push is basically useless because Google hasn't prioritized fixing the issues causing Android to delay the notifications by 10 minutes or more. They are basically only useful for "re-engagement" spam, nothing time critical.
And while I'm not fun of heavy-handed government intervention, I hope the EU compels both companies to adopt a more open approach; provided that security remains uncompromised.
The post says it is about Android. But I'm online searches turns up documents about VAPID at apple/ios. Will it work there too? What about other platforms (eg non-google android-likes)?
On the phone, the Play Services are exempted from the battery optimizations, so the socket connection stays up. And once a notification is received, Play Services unmarshall it, and route it to the correct application ("activity"), possibly waking it up.
From the notifier's side, you need to have the credentials for Firebase interaction and the device's token (that Google Play Services make available to the application during the notification setup).
iOS works in an exactly similar fashion, except that the endpoint is controlled by Apple. To send it, you need to set up an "APN".
What prevents someone else from impersonating my app or other users on my app?
The application may embed the VAPID public key while the VAPID private key is kept secret by the app developer. That way only the app developer can send valid push notifications. This approach doesn't work when the app running on an untrusted device sends push notifications directly though?
I guess the trick is for the app to treat the push notification purely as a hint to go fetch the latest state from the app server. Do not trust anything in the push notification message. Then it doesn't matter whether the messages are spoofed.
You linked to some Android Intents code in the firebase-message code. I guess that is related to preventing Intent spoofing, but I'm not sure?
I have a Octoprint integration which lets me know when 3d prints have completed.
You can also modify the priority of a message, so I can send "critical" events if I need to, although I use that rarely.
What is a "push notification" exactly?
They’re “notifications” in common speech but there’s also stuff like “now playing” and live activities and rich content that are also “notifications” but do require the app to run.
From what you're saying, it doesn't sound like the OS-based polling would be more power efficient, as someone in this chain suggested, just more reliable. Though, I think you can just turn off battery optimization for the ntfy app, in which case I'd expect them to be on par.
If that's right, it seems like ntfy with battery optimization turned off is effectively a "real" push notification system.
The "distributor" receives notifications onto the device and sends them to the apps via a standardized API.
The apps themselves then post the notifications to the user via the android on-device notification API.
The "gateway" receives the notifications from the services over a standardized API and sends them to the distributor.
How the gateways actually distribute the notifications is actually completely up to the gateways. The main gateway (ntfy) uses websocket, another (nextpush) piggybacks off of nextcloud, and the new kid on the block (sunup) uses the Mozilla Push Service behind the scenes.
Nothing in this is restricted or locked in to a centralised platform.
Just use or improve upon email or RSS (recommended).
It's anonymous, fast, privacy friendly (no device registrations or whatever), open and already decentralised so no need for anything else.
This exists to improve battery life and decrease data usage for android users who don't use google services or prefer to host their own. And generally it exists to provide a standardised, independent, privacy preserving option for users and app developers.
Neither RSS nor email solve this problem and actually this specific service would be the exact type of service you'd use to notify the user's device that the apps that use RSS or email have new content to fetch.
If it's using intermittent/timed fetch, that runs into the latency problem (taking longer to receive notifications). Push notifications exist to solve that problem.
If it's using IMAP's IDLE, MAPI's push, or JMAP's push, that's just another long-running HTTP or websocket connection.
And this protocol (along with the proprietary push notif aggregation services it is aiming to replace) exists to deduplicate those many long-running connections into a single connection out to some hardwired server that manages the many services/connections for it.
Local notifications?
I don't see the problem here with email, it seems to work well that it is great for my use case.
When you're offline with push notifications only the latest notification gets sent, not the previous notifications. This seems by design for notifications.
At least with E-mail I can see a history of emails that was sent to me.
I just see push notifications as a flawed concept that doesn't make sense for decentralised services.
And here's your issue: your app that has nothing to do with Google relies on a centralised Google service, and depends on a proprietary library that speaks to some proprietary code running on your device.
Alternatively, your app isn't doing this, but then it risks being unreliable and stop notifying you if the system kills your app because it's been inactive for some time to save battery, or to reclaim memory, and/or it uses more battery than it could.
This post is not about a way to have more notifications: it is an answer to these issues.
Except it doesn't?
The Mail app on my iPhone doesn't use push notifications, fetch works well and I still get local notifications. No external push service at all.
> Alternatively, your app isn't doing this, but then it risks being unreliable and stop notifying you if the system kills your app because it's been inactive for some time to save battery, or to reclaim memory, and/or it uses more battery than it could.
You're saying a bunch of "issues" that never actually happened with me or anyone really, there is actually no issues here.
E-Mail is fine.
See how I introduced your case later with "Alternatively"? You saw it since you quoted me.
Fine, your email app works well in fetch mode. It took you three roundtrips to mention your actual setup despite having been asked about it quite early and despite it being a critical element in this discussion. Omitting to mention that doing this choice, your notifications are delayed.
Fetch mode is totally a valid choice, email doesn't need to be real time for every one. It will actually save battery life from what I can read: you don't need constant polling. Of course, push notifications are for real time stuff. What costs battery life or requires a push notification service on current widespread mobile devices is timely notifications. I can't help you if you fail to see how this can be useful, even if you don't personally need this or are against the idea.
Despite a thread that started well (yeah, notifications via emails, a decentralized, open standard, why not? It's terrible for many reasons including deliverability and the fact that you need to be notified of new mails so it's a bit of a chicken and egg issue, but sure, let's discuss), you don't seem actually curious or interested in the topic at hand, preferring to stick to your original point of view, to ignore the answers from all of us and to ultimately show contempt and spite to people discussing with you here. By the way, frankly, loose the scare quotes, they are ridiculous. Of course Apple Mail's fetch mode will be reliable on iOS but that doesn't make the topic trivially solved and yes, the problem exists despite what you are stating.
You are calling something over engineered without stopping to wonder why the complexity, assuming it's useless.
Many services and apps use E-Mail so there is no need to setup additional infrastructure unlike push notifications. That is the complexity.
Send an email notifying me to test@example.com is more intuitive than setting up a push service.
Push notifications suffer from deliverability more frequently (regularly expiring tokens) than E-mail so much so, that when your device is offline and it gets back online, you only get the last notification. This is by design.
Having apps use fetch doesn't mean it's delayed, you can set it to a shorter duration but 5 minutes is the default.
But, it is over engineered as this is complexity that we do not need and would have a higher maintenance and setup than just using E-mail.
"hey, I just arrived at the restaurant!" You may want to know this now
Gmail does push notifications. Otherwise, have to wait for the client to poll the server.
What if I don't want to use an app?
Email works with every email client that has been ever made.
It is already universal.
The user oriented term refers to on-device notifications that pop up at the top of the screen. Those are technically just normal notifications but people like to call them push notifications for various reasons.
Push notifications in the technical sense however refer to a system for remote services to notify a client that something happened rather than the client manually polling the server for new updates. It's the difference between the client asking "do you have anything new" and the server telling the client "hey pay attention you have something new". This is what we are talking about when we say push notifications.
Email clients that automatically get new emails in the background all either do so using fetch polling or push notifications. The traditional method is fetch polling at a regular interval but nowadays most email clients support push notifications. We are discussing a tool that these clients can use behind the scenes to provide these push notifications for the email client in a battery and data efficient manner.
Isn't the purpose of Google's push service that you can receive notifications for a bunch of services with only a single connection?
I don't quite understand how adding a different service helps, although having an alternative when Google's push is unavailable could be useful, and if there's some way to share that alternate service among many apps, that would reduce overhead of having two notification channels where each app with two channels has their own alternate channel?
The benefit of this service is mainly for people who can't or don't want to use google's push service. Namely for users of de-googled devices and for users of android devices that google blocks from using play services (ex: users of Huawei devices).
But also it's worth noting that unified push provides an optional google FCM fallback so that apps can use google's push services (provided they are available on the device) when a unifiedpush distributor isn't available or configured. So in this sense it means supporting a single push notification standard instead of multiple and the underlying system can switch between which provider to use.
And of course for users of completely self hosted services this means you get all the benefits of a proper push aggregator like google's FCM without having to configure firebase with your services. This is particularly appealing if you self host locally and don't already use cloud services.
I simply don't like having two monopolies (Google and Apple) handling every single push notification for all mobile users everywhere on Earth – except in China, maybe.
There's so much you privacy risks and they can easily infer many things from the meta data alone.
I thought we were in the post-Snowden phase already.
Signal sending me an email to tell me I have a new message would be a bad system.