Push notifications when you have 8M followers [video] | Hacker News Reader

Push notifications when you have 8M followers [video]

techcrunch.com

151 pointspbhowmic44 comments

Reminds me of when Snowden joined Twitter and forgot to turn off email notifications:

https://twitter.com/Snowden/status/649653419620352000

This is unsurprising but also really incredible to actually see. I was first struck by how smooth it runs through all those notifications. I wonder how real time those notifications are. Are the ones coming in now for content created an hour ago?

I'm really curious what a system designed to push that much info out all over the place looks like. Where can I learn more about this?

I developed for an app with 1-2 Million daily active users and we discussed several ways to do this. In the end we used AWS so it came down to having a bunch of workers send SNS messages to amazon. As with almost everything, the internet runs on AWS.

You can also have a bunch of workers with persistent APNS connections open, doing the same thing, but without AWS. Last I checked, SNS had a few second delay for sending push notifications.

Here's a link to the video on Facebook for anyone else that has trackers blocked

https://www.facebook.com/demydezeeuw/videos/957955867617029/

Thanks. Shamefully, the TC article doesn’t include any link whatsoever to the content they are transcluding from an entity which openly tracks everyone across the internet.

A friend-of-a-friend of mine built the Red Alert [0] app that sends a push notifications when a rocket is fired from Gaza. Alerts can be as short as 15 seconds, so if you want your Alert to be effective, you have to figure out how to send out several thousand notifications in seconds.

[0] https://redalert.me/index_en.html

I like how smoothly they flow. Looks like Apple load-tested a continuous stream of notifications.

It's actually just a product of how animations work in OS X / iOS. When you play an animation, it takes the place it is currently on the screen and tweens to the place it needs to be on the screen. This means if you stack two animations, the second one takes priority, but the duration stays the same, so what happens is something like this:

N1, N2, ... NN are notifications.

Time 0 ms: N1 is delivered. It appears at y=0 with no animation.

Time 100 ms: N2 is delivered. It appears at y=0 with no animation. N1 is animated to y=50 over 100 ms.

Time 150 ms: N3 is delivered. It appears at y=0 with no animation. N2 is animated to y=50 over 100 ms. N1 is currently at y=25 (halfway through its animation) and is now animated to y=100 over 100ms.

Now N1 appears to accelerate downward while N2 is moving at half the speed downward, and N3 is behind both of the notifications waiting to be pushed down. The effect is like a waterfall of notifications accelerating downwards, without ever explicitly coding it that way.

Instagram needs to switch that user's account to VIP - in that case (I'm guessing) you stop getting individual push notifications and instead get aggregate events sent to you ("your recent picture has higher than normal engagement").

Let's user story this one up.

[Celebrity wakes up & glances at phone]

>> IG: "You are still very popular and everyone loves you"

[Celebrity breathes a sigh of relief and goes back to bed]

You need to turn off adblock to view the video. Or at least I did, because the video is hosted on Facebook and I have a social media button blocking list. Though the site looks very messy without it.

Second, why does the os allow this many notifications? Obviously this is an edge case, but there are countless apps and any of them could have a bug or unexpected situation like this that causes a lot of notifications.

> Second, why does the os allow this many notifications?

Why wouldn't it? How would you implement a throttle that behaves correctly with all notification types?

Seems to me the better solution here would be Instagram bundling up notifications within a specific amount of time and sending you just one saying "You have 15,000 new likes!" etc.

I'm pretty sure there is a certain interval for the pushing of notifications, and each notification accumulated in that interval gets pushed together as a bundle. They are supposedly still ordered so they can appear in UI like they came in individually, like you see in the video.

Fair enough

> Second, why does the os allow this many notifications? Obviously this is an edge case, but there are countless apps and any of them could have a bug or unexpected situation like this that causes a lot of notifications.

A low and configurable limit exists w.r.t. the number of notifications per app that will appear in the lock screen or notification drawer.

The user also has full control over whether s/he wants any notifications at all from Instagram.

I don’t see the problem.

I wonder if Apple tests for this edge case. If not, good on them for some solid software!

Good question. I'm sure they did, and I'm sure there are cases even worse!

I'm surprised the cell towers were able to keep up without skipping a beat.

Push notifications are pretty tiny, and it's not like one new connection is opened for each notification. Also that's over WiFi.

Looks like he was on wifi.

Can't be that much actual data. Also this user is on wifi.

Indeed, the payload limit for an iOS push notification is 2KB (up from 256 bytes before iOS 8).

Engineer 1: Watch what happens when I make a million notifications!

Seems like they should be able to batch them in some way, and not have to send a notification for every single like.

wonder how android would behave in this situation

You don't have to wonder (or have that many followers) due to broken notification system. Google Hangouts will push all notifications since you were last online. So if you have an old phone that you rarely use... Strava, for example, is stupid like this as well.

They are slowed down somehow. Because even if I disconnect the Internet immediately after I remember what awaits me, it keeps bugging me for quite some time after. Associated ring/buzz is not "grouped" so it is a hellish experience (and perhaps that's the reason why it is so slow).

I dunno about android in general, but my old GS4 would probably crash and turn itself off.

Not sure about Marshmallow, but in Lollipop you get one notification card with a counter that is the actual number of notifications of that particular type/app. So you would not have a nice animation like in iOS.

Deleted comment

Notifications are unique to each platform. Plus things like animation, delay, sound effects, etc all could operate a bit differently.

I wonder how long your battery lasts with that going on.

Wow, even embedded facebook video is served as flash (which I don't have) and only served HTML after changing user agent. Is it ignorance or malice?

That's some terrible UX.

It's an edge case.

It's an edge case of an edge case.

But you'd think they'd do something like '200 users have liked...' A cron like every 5 minutes (or more) after the first 10 likes or something like that..

The problem with these systems is that batching would make the implementation more complex. Sending say 10m notifications per second in a fire-and-forget manner is easily distributable en requires little state. Batching per user would be do-able but requires thinking about distribution (messages to the same user should at some point go over the same machine) and tracking a bit of in-memory state per user.

In short I can imagine the trade-off of less-optimal support for some edge cases like these.

Yeah but then is it really notification worthy? If you're that popular, you just post, and then check back in a while to see how it did. There's nothing really "notification worthy" like when it's more like 1-to-1 communication.

It seems like most instragram users have over 100 followers[1], if they're all active and connected that means a sizable chunk of likes (or whatever else you can do) when you post something.

Surely sending all these notifications isn't entirely free - someone else posted Snowden receiving 47 gigs of mail because of email notifications.

You could do something like send notifications with some sort of back-off. Send them for 10 seconds, then back off to a minute's worth chunk, then 10 minutes, then an hour.

Then again, I don't have any statistics on how much these external services actually cost, nor how often this actually does happen. Simplest thing is for instagram/twitter to not care, and to point the finger at apple/android and tell them they should have UI to squelch high-frequency notifications.

[1] http://www.statista.com/statistics/419326/us-teen-instagram-...

> It seems like most instragram users have over 100 followers[1], if they're all active and connected that means a sizable chunk of likes (or whatever else you can do) when you post something.

Bad assumption, given the 90:9:1 rule (99% lurk while 1% participate)

https://en.wikipedia.org/wiki/1%25_rule_(Internet_culture)

1% does mean a lot when you have millions of followers, though.

I don't understand. Would't each of your 8 million followers get one push notification instead of you getting 8M when you post something?

He gets a push notification every time someone "likes" his post. Since he has 8 million followers, he gets a lot of likes.

Uh. This is when they like (or favourite, or whatever it's called on instagram) the photo on instagram?

Nah, he gets one for every one of his post that gets a new like. He has 8M followers, so that's a lot of likes.

Reply on news.ycombinator.com