Apollo Back end just made public | Hacker News Reader

Apollo Back end just made public

old.reddit.com

496 pointsjviide237 comments

Hi all! I wrote this code, and for educational purposes, I’d be more than happy to answer any questions.

A gentle reminder that the whole history is here for show. This is very vulnerable — you can see the times I was in a rush, or when I took my time. You’ll find my mistakes. Either way, all I ask is please be kind. I know there are 50 things I could have done better, but this is a product of the reality at the time — a balance between something that works and something that was cost effective for Christian.

It’s hard to put oneself out there like that.

Christian is very visible in the community, but I want to say thank you to you too.

Thank you for all the work you did! This made a lot of our experiences great.

This right here is why I did it. To see a friend succeed, and to help make a community happier <3

That's really wholesome which makes this incident much worse IMHO - it's the evil greedy corpo growth bs mindset vs the good will of few people that care about their community.. dystopic world

This sentiment is what made Reddit a great community for smaller and specific subs.

It’s just people talking around a topic and helping each other. Subreddit mods doing everything they can to keep their area clean for people reading. I’ve been on Reddit over for over a decade and this is the power behind it. Just wanting to help and bring good member.

Thank you Andre. For your work and commitment to Christian, Apollo, and the community.

Thanks for your work. Apollo made using Reddit a decent experience, and clearly you had a very large part in that.

User rapnie mentioned the code in the repository doesn't have a license [0]. You might want to review that.

[0] https://news.ycombinator.com/item?id=36256566

I’ll check in with Christian and see what he wants to do. Got any particular recommendations?

You probably want MIT

Got it, thanks!

The advice in https://news.ycombinator.com/item?id=36256665 is more thorough, I wouldn’t just skip to this one.

You may want to also check out Apache.

Here is a good comparison of licenses.

https://gist.github.com/nicolasdao/a7adda51f2f185e8d2700e157...

This gist doesn't even mention AGPL which as other users mentioned is much more appropriate for server-side apps than GPL.

GPL will ensure that the source is not used by any projects not affiliated with you who are trying to profit off of your work.

AGPL will do that, no GPL

GPL on server side software doesn't really matter, that is why the AGPL exists

>GPL on server side software doesn't really matter

That's maybe too general. There are cases where it does matter.

Maybe "GPL on software-as-a-service code has limited usefulness"

Even then, people do sometimes choose it on purpose for SaaS, where they want to ensure any redistributed code, where the entity doing it makes that choice...is bound by the GPL.

Some options spring to mind:

2. AGPL, to ensure that if someone does take this forward, no part of it can be closed source without your agreement.

3. MIT, which is extremely permissive and effectively you don't care what someone does with it, commercial or not.

Depending on your comfort level I would start with the middle option, I don't think people tend to mind you re-licensing more liberally but if a community has formed and you decide to go commercial further down the road it can cause problems.

Gives those who wish to learn and stops from anything taking the future glory.

Apollo was one of the apps that was basically giving up due the API changes, right? If they aren’t doing anything with the code, might as well let people fork it.

Just not MIT or BSD, I like these sorts of licenses normally, but it would be a bummer if Reddit got to steal their code after all this bad behavior.

Agree, would be awful if Reddit themselves just decide to lift and reuse this!

Just add them as a specific exception to whatever license you use, if that's your wish.

Why would Reddit add another backend layer?

If they’re unhappy with their own API, this could provide an abstraction layer for the app. They rewrite their client to use this API and it frees up two teams to work (almost) independently. The UI team can work on their roadmap while the backend team fixes the internal API or retires the logic and implement it into this new layer.

I’m not saying they’ll do this or that it makes sense. But I’ve definitely done it in much smaller projects.

This here is the superstar comment. There are other options but this is an excellent start.

Take these recommendations with a grain of salt and make sure you understand the implications of each license.

I wouldn’t add a license unless you specifically want to allow people to copy and use this code, otherwise just ensure that the appropriate copyright notice is present. No license is better than a badly written or badly chosen license — no license already means that all rights are reserved and no rights are given.

> A gentle reminder that the whole history is here for show. This is very vulnerable — you can see the times I was in a rush, or when I took my time. You’ll find my mistakes. Either way, all I ask is please be kind. I know there are 50 things I could have done better, but this is a product of the reality at the time — a balance between something that works and something that was cost effective for Christian.

The way I took Reddit's comments were there were things that could be optimised to reduce costs. I've not seen a codebase yet where that isn't true.

Sadly, in the hands of Redditors who think you can host an API on AWS for $1 per million requests comments like come across as "Oh they did shoddy work".

What I think is we're seeing production quality code like the majority of us have running.

Christian talked about this.

A payload of 25 comments vs a payload of 100, takes less time to generate and deliver.

So sometimes, to optimize his users experience, he might request 25 to show something quick to the user and then do a request for 100 and append it.

He also mentioned that he’s being compared to other apps, _not_ to the API limit (which he was under anyway).

So, can it be less?, of course. Heck, he could make it 1/minute or even 1/day. But his quota was 60/min. So why not use the 60?.

The Reddit API usage is as efficient as it can be, server wise, although I’d like to be proven wrong by them.

As far as the app goes, an average of ~330 reqs/user/day is pretty good, considering they built the platform budgeting for 86,400.

Yeah, every time I see that number trotted out I’m just like “and the point is…?”

If there’s an issue with the cost to process that many calls I’m much more suspicious of inefficiency on the API side than the client. Those numbers seem very reasonable.

I think the point of inefficiency was that Apollo could say money and make the costs less than they would be by doing less requests. If someone is saying X is too much it's a valid rebuttal it wouldn't be X but Y if you made a few changes.

I don't think Reddit cares about the number of requests. They just want paid. They weren't caring before when it was free.

Maybe a description of what we're looking at in the Readme would be helpful. Why wouldn't the Apollo client just directly call reddit? Or does it partially do that and this server component is additional for certain functionality? What functionality does this offer?

That’s great feedback. Any particular areas I can walk you through?

Since I have no idea what's even here, I can't really answer that. Maybe just a few sentences on what this code does in general?

I’ll be happy to once I sit down in front of my computer.

A naïve question, but why is a backend needed and why couldn’t the app call the APIs directly? Just curious to learn!

Sibling comment above says it was mostly for notification, but flowing calls through their server also meant they didn't need to resubmit to Apple most of the time when something broke on Reddit's side.

Yeah. I’d imagine the less client side code the better when you have somebody getting paid $1/hour to randomly reject apps as the final step in hour launch process.

This was for push notifications to IOS - to tell the user when (for example) someone had replied to one of their comments - even when Apollo wasn't running.

That's just the reality of writing code, we all make mistakes and they haunt us all :)

Wouldn't it be possible to change it to use Lemmy or some other alternative?

Honestly, Lemmy would probably not need half this code. We wrote this to work with an entity that exists in a “come and get it” model. Lemmy being open source means we could build the features right in the server and slash this code to a bare minimum (mostly the watcher stuff at this point.)

Are you proud of your code ? Which design patterns do you use ? Any lessons that you've learned to be shared ? Thanks.

I’m never proud of my code haha. This could always be better. No specific design patterns either.

The one fun thing I learned was about databases. Given enough records, a query planner might decide to start doing things very differently and you will get woken up at 3am with a CPU usage alert. That was a fun one to debug for sure…

So am I right in thinking that the backend was needed to poll the user account in order to generate notification? Was that one of the issues that reddit had with how apollo was using their apis?

That’s how it worked, yes, but we went to great lengths to ensure that call had minimal impact on their end.

The TLDR here is that we only ever requested new notifications (give me anything that came after this thing I know about), which funnily enough then broke for some users, because if a reply you got notified about gets deleted and you use it for reference, Reddit will never return anything new to you, so I had to go and write a 2nd service that runs much more infrequently but does some checks on each account to make sure this isn’t the case.

> Was that one of the issues that reddit had with how apollo was using their apis?

I doubt reddit had any actual issues other than it impacting profits and no optimization to Apollo would fix it.

You made something that people got a lot of value out of. That’s incredible. The actual code and commits are just the journey. They’re not the product. I’m rather delighted every time someone shows a realistic journey full of all the things we all experience, but some of us pretend don’t happen. Thanks for being vulnerable and sharing this.

This is an amazing way to look at it. Thank you for your perspective <3

Any developer should be proud of the value that code/backend provided millions of users.

Amen to that.

When I first started coding (some 15 years ago) I wrote the ugliest, buggiest and "owasp-ridden-est" PHP spaghetti code. Turns out that code went viral and was used by millions of users for a few years.

Yet, I came to the conclusion that that's the best code I've ever written, for it's been the most used code I released so far.

Real software development is far different than the ideal one.

Software managers take note… Your MVP isn’t necessarily your perfect code. It’s the messy, “get it out the door”, value it brings to others. This is your product, refine it from there but don’t hold it back for sake of polishing a turd. Myth busters proved you can indeed, polish a turd. Don’t waste the sprints.

No, that's just wrong. It's totally fine to put scrappy code up to get a product going, but that time has long since passed if you're already at a scale to have dedicated managers.

At that point you're likely going to cause more damage to the brand/corp with the half-assed implementation then the value you're providing with said code.

I tend to agree with this. My spouse is currently entrenched in an over-budget, under-funded, mismanaged project that was recently announced by the CEO.

That’s at a Fortune 500. It all started with a “well we can do an MVP and get that out the door first.”

It turns out that was a massive mistake.

You'll note I said MVP. To get the product going. I agree that once it's going it's best to refine and polish tested features but software managers are still usually hands on at launch phase. Once you have your fit then follow best practice and have a proper process in place (agile, waterfall, whatever). What scale does one have to be to have a manager? Do companies operate without management?

This. A thousand times this. I work on a household-name product with millions of users. It works pretty well most of the time but also has frequent outages, bugs, and usability issues, during which times the 1-star reviews pile up.

By far our biggest problems are:

1. Leadership at the time just wanted an MVP, and they wanted it yesterday.

2. Leadership now believes "You don't need more people, and rewrites are a waste of time. Look what we've built with a skeleton crew and tight timelines!"

Fellow engineers, beware: if you train your leaders to believe your team can deliver with no budget and unrealistic timelines, they will continue to give you no budget and unrealistic timelines.

Those are indeed leadership failings. After-market processes should be put in place to ensure quality. If that means slowing the cadence of releases, fine, so long as the value delivered is there. If you have millions of users, it sounds like it is. In which case my point about just getting it out the door isn’t applicable.

What a completely lovely way to express what I expect many of us are thinking. Thank you.

What an awesome comment.

Don't be ashamed of code you wrote that wasn't intended for an audience.

Most would hate to see what a sausage gets made out of, but nearly everyone will eat that same sausage.

I would demolish a couple hot dogs right about now…

Sorry you feel on the defense. Developers can be very aggressive with feedback and I know the feeling. To echo others, "Build something that works first", and you have something that works :)

IMHO the gesture of releasing the code shows how much integrity there is behind the devs, and I could care less if there's not-best-practice in there.

So, too much work to adapt it to hackernews?

Would love to have a decent client for this site…

HN is a whole other beast. A notification server would have to keep track of any replies to any of your comments / submissions in order to determine what to notify you about, purely because they don’t have a notification model.

I’m a long time paid customer of apollo. I have only used Reddit because apollo exists. What an amazing app. Thank you for all the hard work.

Having to ask people be kind about your code on HN is such a shame of reality to me. Half of brilliance is having your working version, used by many people. You had working code that brought reddit to so many reddit users every day that was better than what the official app of reddit provided.

Some of the 'code' running massively profitable departments in finance are spreadsheets, on shared drives, with thousands of linked cells thought up by a brilliant employee who left the company five or six years in the past.

I wish I was kidding.

Thanks for how great you kept apollo.

I would love if we all agreed to value the joy of building on here. Not saying it needs to be nauseating positive, but just a default expectation of comments like “wow! This is cool, here are some things I might have done differently”

Show HN is always really close to to that ideal. One of the reasons I like this place.

Your code was production ready for millions of users, that's all that matters

Do you think Christian would be willing to open source the app (under a non-commercial license) and allow us to build it with our own Reddit API keys?

That way we could still use Apollo, but pay for it ourselves

Making the code available is a wonderful thing to do. I love being able to see how others managed and structured a Go project that was actually meant to run in production.

Thank you.

Keep in mind that this isn’t rooted in any specific best practices. It just felt right.

Your commit messages [1] are better than of many projects and seniors I have worked with. Nothing to be ashamed. You made something that "just worked" and delivered value. You should be proud!

Sample of latest commit messages:

batch size wants to be smaller for less locking time, not higher

increase scheduler batch size

bump locks again

make pool sizes customized

increase prefetch

[skip render] remove old redis from infra

increase poolsize a bit

bring new redis up to line in blueprints

reduce pool size

bump pg client count

tweak pool sizes and consumer count

double the amount of consumers

small tweaks

[ci skip] update pgbouncer github url

clean allow lists

add new redis instances

make connections to the correct redis url envs

add new env variables

spam prevention

update reddit http client

feat(ops): add distributed tracing (#109)

I'm curious if ya'll have any plans to support a reddit alternative, like Tweetbot -> Ivory and other twitter clients. I love Lemmy. I think it would be an excellent middle finger to reddit, but I understand if you are ready for the next thing

You wrote code that works and powers an app for millions every month and any bugs or 'bad' ways of doing it are clearly sufficient to maintain a large, healthy user base, so you should feel nothing but pride!

Also, thank you for making this code available, but would it be possible for you and Christian to choose a license for it so people can use it? I don't think which license matters all that much, personally, but I can also understand wanting to keep it proprietary, either way, thank you again for all your (and the rest of the Apollo teams') work.

You might want to put this in the readme

You shipped and found market fit on your own. A lot of funded teams haven’t been able to achieve what you’ve done.

As another who will stop using Reddit when Apollo goes dark, I wanted to say thank you for your work with Christian!

Apollo saved my Reddit use after they bought and killed Alien Blue. I have gotten so much out of and out so much into the more obscure technology subreddits. It’ll be missed.

I appreciate all your work. For what it’s worth, I’ve never had server-side issues with Apollo (that I’ve noticed anyway!)

Edit: typo

FWIW, that just sounds like real code to me. You can spend your time polishing code until it shines and jerking off over the perfect crystal of logic you've created, or you can spend your time building something impressive that does cool stuff.

Especially when you're solo or on a small team, technical debt is actually something you can lean into. In this domain, refactoring is cheap and cutting corners can help you get where you need to go faster.

It's a too common mistake to think that because technical debt is a tar pit that kills many large products that this is universally true. The real trick is lowering the tolerance for flaws as the project grows and matures.

I was diving into your code and noticed you were using golang-migrate for running your migration queries. I never heard of it before and the number of databases they support sounds interesting!

How was the experience overall with this tool? If you have experience with Flyway, how does it compare?

Thank you for your contribution. This is a heroic act.

My friend, that’s ALL of our code!

Why not point Apollo at a Reddit alternative, or use the open-source code for Reddit (or code like this https://github.com/libertysoft3/saidit) instead of shutting it down?

If Apollo's users (or a good percentage of them) moved over to an alternative platform, that would be poetic justice, at least.

To go even further, all the app devs should get together and create a new platform or pick one Reddit alternative and point all their apps at it.

That way it would be even more likely to get critical mass quickly. 3 weeks should be enough time to come up with a coordinated plan. Don't let Reddit push you out -- push them out and let your users stick with the app they love.

Reddit is only its users, nothing else. They don't own the users' loyalty -- the apps do. Take advantage of that and teach them a lesson by pointing the apps and users elsewhere!

Why don't you and Christian ask Apple for a small donation, and replace the reddit API with your own? You must have talked about the idea?

I am waiting on delivery of an iPad pro, my first apple device in 10 years. And, even though I hate reddit, I was excited to check out Apollo, if for no other reason than to see what a great ios app looks like.

I know that reimplementing reddit API from scratch will take more than a few weeks. But, I'm sure loyal Apollo users will be happy with a much scaled-down version. Maybe, to begin with you put the app in Read Only mode, and point it to a reddit archive. Then work on making one sub read/write again and expand from there.

I mean, even just a good reddit archive browser has TONS of value. I'd happily pay a small sum for an awesome reddit archive explorer.

"Chatgpt, build the back end for this API: $(cat reddit_api_docs)"

There's a Reddit CEO AMA later today: https://www.reddit.com/r/reddit/comments/144ho2x/join_our_ce...

Can't see it being anything other than a PR disaster for them, unless they change their stance.

It's more than likely that they'll inject their own softball questions and focus on those instead of addressing everyone's concerns in any meaningful way.

good test for the new AI bot accounts

When they removed the upvote/downvote count, it was clear reddit was fine with manipulation.

Anything I read on the website has been met with extreme skepticism since it very well could be an astroturfer.

It's money. Reddit wants astroturfers, it grows their platform and helps sell ads.

> Anything I read on the website has been met with extreme skepticism since it very well could be an astroturfer.

Good advice for any site. It's not like HN is immune to astroturfing.

You have more courage than me to say that here.

> When they removed the upvote/downvote count, it was clear reddit was fine with manipulation

I wasn't a big issue to me tbh. Vote fuzzing had always made those counts meaningless.

May not even need to be an astroturfer, it could be u/spez directly. Don’t forget that he went rogue and modified a user’s comment in the Reddit database!

Or that the Reddit founders used fake users to make the young site look busy:

https://arstechnica.com/information-technology/2012/06/reddi...

> it was clear reddit was fine with manipulation

talking about manipulation, reddit's current ceo edited users comments back when the bad guys were on business side of it.

https://www.theverge.com/2016/11/23/13739026/reddit-ceo-stev...

Let's talk about Rampart!

Love that game!

For some companies, for example Ticketmaster, PR disasters are fine. They don't affect the profitability of the company, nor the user base.

However if a company's CEO holds a reddit AMA that company probably does care what the public thinks.

Or that company cares what the Reddit community thinks and is convinced that they have an ace in the hole. An ace like, say, having full control over the platform and being able to make the AMA look like it goes however they want it to.

Reddit community is far to suspicious of Reddit corporate to fall for that.

Depends on which part of the community you're talking about. Many of the top-voted comments on some of the "sub blacking out" threads were to the effect of "TIL that there are third party apps." [0]

That part of the community is the part that Reddit probably wants to preserve, and that part could very easily see an AMA that appears to clear everything up and then get mad when their sub moderators go ahead with the blackout anyway.

[0] For example: https://www.reddit.com/r/gaming/comments/142p1ax/comment/jn6...

Yeah but reddit is trying to have an IPO this very year

The risk though for PR disasters for companies like Ticketmaster is when they get the attention of regulators. Then it becomes an issue for them.

For this being such an important moment for the site, it’s absurd that the link they announced didn’t even have a set time. And they haven’t added one.

I think I read it’s going to go live at 10-10:30am PT

I don't think there's anything Reddit could do at this point to not let it be a PR disaster. That includes changing their stance. What they're doing is so egregious, naming and shaming and lying about people who brought immense value to the community for years. All out of misguided greed.

I was talking with a friend about this the other day and he brought up a good point. If you were about to IPO as a company, this is not a bad decision from a business standpoint (cut costs, bring as many users as possible under your control). Obviously the way they executed the decision could be harmful to them, but based on the pressures they're under, I can't see how you could blame them too much. Which is why the only enduring platform has to be something federated like mastodon or lemmy, since they won't have the same economic/capitalist pressures to make the product worse.

Rich and influential entities should be subject to more accountability, not leniency. I agree that for-profit entities with motives are good reason to be wary of them, but it's no reason to let them evade responsibility for their actions.

Also, no company's success should ever depend on alienating its users and slandering longtime contributors.

> If you were about to IPO as a company, this is not a bad decision from a business standpoint (cut costs, bring as many users as possible under your control).

the parallel for traditional companies is choosing to bust unions, and a month before you IPO is an absolutely terrible time to choose to bust your union unless you're absolutely 100% sure you're immune to a "strike".

in this case it's not just picking a fight with powerful stakeholders (like mods) but also turning off features that a ton of users interact with the platform via, and trying to force everyone off third-party clients onto the new experience and native app. In this model the users are a stakeholder that you're picking a fight with at the same time.

the best time to do it is 20 years ago, the second best time is today, unless today is a month before you IPO.

The clowns at Reddit couldn't even specify a time.

This will be a spectacular shitshow to watch.

For some reason, they gave Verge a time but not on their own post. It's at 1:30 pm EST

https://www.theverge.com/2023/6/8/23754780/reddit-api-update...

Excellent thank you, I don't use the verge as a source of information, and am surprised how common they have become.

Surprising that Reddits communication on their own platform is so bad, or maybe it's not...

Thank you for the update!

The AMA is confusing - I don't understand what questions they expect to recieve, and how they'll answer them to anyone's satisfaction.

I think the original intention was to get ahead of the story but I think the apollo dev's announcement that he's shutting down the app is going to make that difficult at this point.

> Can't see it being anything other than a PR disaster for them, unless they change their stance.

You were correct

All this backlash is going to kill Reddit. It will take a thousand little cuts for sure, but it started the moment they announced their ludicrous pricing.

I wonder.. The very same thing hardly seems to have hurt Twitter.

Personally, when someone mentions tweets nowadays, my first impulse is to ridicule / discount it all - "what, are you seriously using a service Elon bought to promote far-right topics...?"

Typically it comes out of my mouth in a more moderate stance, like "sorry, I don't use Twitter anymore. Any chance you could link a better forum?"

The most charitable reading of your comment still leads one to question what you would consider the right if Elon is far right. I am pretty sure he is a moderate leaning both ways on differing topics.

That's not what the parent wrote though. They were Elon is promoting the far right.

He is for sure far right. He's against increasing taxes on the wealthy, vowed to lobby against transgender health care, endorsed Republicans in the midterms, launched Desantis's campaign, and constantly rails against woke culture. There is not an ounce of left in this guys body.

I seriously question the motives of people saying this guy is a "moderate." You cannot gaslight America into thinking the far-right is middle of the road.

I don't care what he "is" or might be; his actions objectively helped the far-right. Whether he did it to troll, to "restore" some imaginary "balance", or because he genuinely believes in far-right topics, he still very materially helped far-right speech - and he clearly did it on purpose.

> The very same thing hardly seems to have hurt Twitter.

It has. As somewhat of a Twitter addict, I'm finding it much easier not to log on these days, because the experience and the conversations have really hollowed out. I'm also no longer seriously considering using Twitter for promoting any projects or anything of the like going forward.

If Digg is of any indication, it takes a series of negligent and greed-driven actions to scuttle a social media platform, but it is possible.

Reddit has been more or less re-inventing itself for their asinine IPO aspirations, and in doing so it has moved to closed source code, redesigned their interface, adjusted content policies to court advertisers, and now they're finally going after the crowd of enthusiasts who depend on features that Reddit has failed and/or declined to implement.

This all reeks of venture capitalist sabotage and it's the very thing that ultimately killed off Digg.

To generalize, Twitter relies on celebrities for traffic.

Reddie relies more on genuine content and less personal brand (as you are subbed to subreddits and not public users, mostly)

Reddit also relies on thousands of hours a day worth of free moderation. If the moderation quality drops, then reddit will really go down hill.

Maybe so, but Reddit is not Twitter; API access is far more important for Reddit because its official client is garbage and can't handle the long-form content that Reddit is known for.

This will have outsized implications for the 1% of users who actually post the content the other 99% come to see in a way that Twitter (and its 280-character comment limitation) does not.

> can't handle the long-form content that Reddit is known for.

Isn’t Reddit widely known for memes by the public?

I don't think so? That's more 4chan AFAIK.

Most people have no problem with the official app. They simply don't care enough even if they acknowledge the drawbacks. We are not representative.

https://edition.cnn.com/2023/02/10/tech/twitter-top-advertis... : more than half their advertisers quit.

A quick browse around twitter shows they have plenty of adverts to show.

They may be selling that ad space for pennies on the dollar... But they are managing to fill it.

The owners want money, not the pleasure of seeing ads.

Twitter stats show great growth in musk related topics (space, EV's, etc), but a decline in the amount of content posted in nearly all other areas.

Where are you seeing these stats?

Here, for example: https://twitter.com/stats_feed/status/1666446359619153920

how does this backup what you said in anyway? all I'm seeing is follower growths for popular accounts

Umm,? Their ad revenue is down north of 50% since the 'garch took over. How are you defining 'hurt'?

UPDATE: Oh I see, they did the kill-the-API-to-kill-third-party-clients thing before too, long before the current brouhaha. OK, never mind.

No the kill the API thing came after Elon. It was one of the things he did, but indeed this specific thing does not seem to have hurt user engagement in particular. In fact it was probably one of the things that was good for their ad revenue and probably why they did it.

I don't think it will. It'll IPO just fine, make their shareholders wealthy and become another millquetoast advertiser friendly walled garden for your average internet user while the jilted original users go and find another platform, which will in turn IPO just fine, make their shareholders wealthy and the cycle repeats forever.

The average internet user by the way, is statistically not you reading this, doesn't really care too much about APIs and finds the current Reddit client just fine really and as a recent registration, doesn't understand that the developer community basically built the interest in reddit and without third-party moderation tools, clients and a community that deeply cared about their special website, flawed though it was.

All platforms go through this cycle, but Reddit felt special to me and it hurts to find that wasn't true.

Money always corrupts in the end.

I think this captures how I feel, which is an odd sense of mourning.

It's not that I haven't known for a long time now that reddit isn't the site it used to be, it's that this episode in particular has put that in such focus I can't ignore it.

So now I feel a sense of loss for the reddit of old, even if it hasn't really been such for a long time anyway.

Reddit itself will carry on fine, and perhaps get even stronger with a tighter control over content. Efforts to make a federated alternative are doomed.

I don't think it will because there's no solid alternative. If/when there is, and the most active users - the ones contributing the content and moderating the comments - start leaving, then it will turn into an empty husk of itself and die like Digg did, even if it does have millions of readers.

My argument against parallels drawn Digg is simply current users of the internet. In 2010-2011 people were more likely to experiment with different social forums, websites, and etc. Also the age of the users were always on the younger side. Nowadays it’s fairly different, and seems like users are more sticky and will only flock to an already established websites, so unless Reddit becomes genuinely unusable (broken native client, dead infrastructure and etc.), majority of the users will stay on and contribute. It’s similar to “Twitter exodus”, which didn’t happen. Obviously things might go downhill for the company, but people go in for their quick dopamine hits, not for some dramatic battle between evil (Reddit corp) and good (Apollo).

Not to sound too pessimistic, but most of the mods of the bigger subreddits aren’t going to give up either, since it’s a part of their life, and not really their battle.

I really hope I am wrong though, since it’s about time we have something new and shiny.

This is what will happen, and generally the people who would leave over these changes are the ones who make worthwhile content, and now they will move elsewhere. But where?

People (statistically on average, not you) come to reddit for ragebait and plagiarism and priacy, not worthwhile content.

Exactly my point

I’d be pretty wary to join in on the IPO if they are taking actions like this, which seem likely to scare off all their moderators. If the volunteer moderators leave, they are going to have to change their business model to actually pay people to work, right?

But I’m not an investor, maybe Reddit has some numbers that make this make sense, or something. Or maybe they are just hoping to rip off some investors who don’t know how the site works.

> All platforms go through this cycle, but Reddit felt special to me and it hurts to find that wasn't true.

This is not always true and there’s an important correction to this: all VC-funded platforms go through this, especially monetized by ads.

There are examples of privately owned platforms with subscriptions that are doing great and show no sign of quality decline. Telegram most notably being one of those.

This chase for constant high returns and also the focus on an audience very ortogonal to your actual users (ads buyers) are what seriously misaligns the incentives of users and platform owners.

> It'll IPO just fine

There is nothing inevitable about that.

Same. Last night I felt a sense of grief that all this immense knowledge & the tools to aggregate it have been fully captured by the bland sanitized values of ad networks.

Even today I can’t find answers on Google to my daily inquiries, but will find it on Reddit.

If there is an internet worth building, it’s not on ads.

But we all knew that from the start.

Here’s to those working on the federated web, maybe the outcome can be different.

I'm beginning to explore the fediverse more and have resolved to find an appropriate place to become part of it somehow. I don't think the fediverse as we know it today is the solution but instinctively I feel it's our best way right now to find our way there.

I share your view - good luck on your adventure!

So where are the "OG" users going?

Reddit will go the way of Twitter. By that I mean business as usual.

When old.reddit.com is gone, that will be the true end.

What are you basing this on? Do we have any data regarding the percentages of mobile/old/app users?

I wish I still had the graph, but someone posted something for their (150k subs) subreddit recently, and it was below 10% old. No idea what sub, or what the source was so, so take this with a pitcher of salt.

It's about 5%. Not much variation by sub, even the technical ones are in that range, maybe a few % higher.

It’s not just about users, it’s about principles.

I have no doubt some users will leave, but I’m not sure what principles have to do with whether or not it would be the end of Reddit. Unless you mean in some subjective, personal sense.

Think of old.reddit.com as a canary. If it goes, then you know some shit is going down and Reddit will never truly be the same going forward. Even if you don’t use it, that should be the signal to leave if you haven’t already.

I would be genuinely shocked if old.reddit.com users made up a non-trivial amount of the audience.

I find tech/power-user crowds like that on HN tend to dramatically overestimate the proportion of a userbase they actually make up.

Reddit's doom has been announced over and over again since they started raising the bar for what is an "acceptable" sub-reddit in 2011.

The reality is that Reddit hasn't ceased growing since then. The same thing happened with YouTube. YouTube pushed its content to switch from filthyfrank to Jimmy Kimmel/Fallon. Advertisers are not interested in us, early-adopters, mostly young and middle-aged western males. They want kids and soccer-moms: they're less likely to install an ad-blocker, and have a much wider range of products for which they're the target audience.

I think we need to understand that we were never the target audience.

Reddit, like YouTube, will do more than fine. The early-adopters just won't be part of it.

My hope is that we will go back to independently ran niche-community bulletin-boards, like in the 2000s with phpBB, instead of piggy-backing on some other new for-profit corporate walled-garden.

I have friends who fit the target audience criteria you mention. These people absolutely do not care about the client and never find the need to step outside the default client. My partner has access to Apollo Ultra through iCloud Family, yet she finds the default reddit client good enough.

If these people exist and they’re the majority, why kill 3rd party apps to mess with a minority?

Because if you're a ruthless company going public, you want to avoid ways for your users to block the ads or "cheat the walled garden." Forcing them to use the official app is the way to go. And advertisers feel safe.

I'm not saying I approve the practice. I'm saying that it makes sense from their perspective.

Hah it’s funny you mention Filthy Frank specifically as he himself is now Joji[0] the very serious and not funny musician. He deleted all his old videos and scrubbed the evidence of who he once was, similar (although not as egregious) to the scat video Blippi posted [1] to get internet fame before he became a beloved children’s YouTuber.

[0] https://m.youtube.com/watch?v=K3Qzzggn--s&pp=ygUESm9qaQ%3D%3...

[1] Seriously, I’m not actually going to post the video. Im sure you can find it if you really want to. My kids aren’t allowed to watch Blippi though.

I'd like to move away from Reddit, but I don't see any viable place to go. The best thing about Reddit is how generally casual the user base is. It's not just a self-selected number of tech-affine users. It's people who love books, people who love running, or video games, or space, or literature, or knitting, or anything and everything else.

Reddit has centralized what would otherwise be hundreds (thousands?) of obscure, splintered and isolated communities and forums. What could possibly compete with that without having already reached critical mass and become an obvious competitor to Reddit?

Yep, and after reading this thread (and clicking through to the reddit thread) I couldn't justify not delivering 1 little papercut of my own; I just canceled my paid account.

(Although at little personal cost, since I don't normally go to reddit directly, but I paid just to stop the inline ads that I would see when I would, as often happens, end up on reddit after googling something with kagi.com or bing.com.)

> googling something with kagi.com

This always makes me chuckle. Kaging something may not be mainstream yet but Kagi-fu sounds better then Google-fu!

> All this backlash is going to kill Reddit.

Just like it did Twitter.

GitHub repo link, in case Reddit deletes the thread: https://github.com/christianselig/apollo-backend

Interestingly, this seems to just be a proof of concept, and the repo is archived. While the developer of the Apollo front end is getting screwed in this whole debacle, this seems to me like an opportunity. He could literally build an alternate Reddit back end (ok, pay someone to do it, if he doesn't want to do it himself), switch Apollo to use that API instead, and get an instant network of hundreds of thousands of Apollo users. A mass migration from Reddit to a new Apollo back end seems like an amazing way to start a new social news website.

Realistically, it would cost more to start a new Reddit using Apollo than it would to pay Reddit for API fees.

Sure the server costs for the API costs alone aren't 1.2m a month for Reddit. But when you add in all the things that need to operate. Databases, storage, data transfer, etc. It would realistically be more than 1.2m in server costs. That's without paying people to build and maintain it.

This backend does very very little outside of continually polling the reddit API for new messages and some other sundry tasks for operating the iOS app. There's nothing here that suggests any of this could be built out to a full fledged social media backend that is similar to reddit. Not to detract from the work being done but it's for a very specific use case: supporting adding features to the app frontend

Yep - but the backend could be rewritten to work from a new datastore. The actual data model isn't complex.

The entire public reddit database of posts, comments, and users could be imported as a starting point.

This is bang on.

That’s also one way to kill all credibility from users.

> He could literally build an alternate Reddit back end

He says he has no desire to do so as he wants to develop products, not manage a product.

https://old.reddit.com/r/apolloapp/comments/144f6xm/apollo_w...

>I've received so many messages of kind people offering to work with me to build a competitor to Reddit, and while I'm very flattered, that's not something I'm interested in doing. I'm a product guy, I like building fun apps for people to use, and I'm just not personally interested in something more managerial.

>These last several months have also been incredibly exhausting and mentally draining, I don't have it in me to engage in something so enormous.

Not everyone wants to throw away their soul for money ;)

I promise you this is very much what’s running in production.

Running a social media site and handling all the moderation and other legal issues seems like

* a giant headache

* a totally different skill set from building an app

Anyway the code is o̶p̶e̶n̶ ̶s̶o̶u̶r̶c̶e̶ probably going to be open source soon and his company is probably not going to keep working on it now that Reddit has basically wiped out their business model, so we may as well ask why don’t you or I fork the code and then create the social media site?

It's not open source yet, it has no license. Upthread the author says they're going to go check with the app dev and decide what they want to do license-wise:

https://news.ycombinator.com/item?id=36256596

Good point.

The interesting thing is this exact same thing happened in Hong Kong. There was an app for a popular web forum. It got banned and the developers released the code and subsequently someone turned that into the “Reddit of Hong Kong” and outgrew the original forum.

https://en.m.wikipedia.org/wiki/LIHKG

Dumb question: why does Apollo need their own backend API when isn’t Apollo just a different view/UX for Reddits own API?

https://github.com/christianselig/apollo-backend

I believe it's mostly for push notifications

Mobile notifications is one reason that springs to mind.

I can take a guess - it allows you the flexibility of doing things the way you want it without having to update the app. You can make non-breaking changes to handle things/issues or implement workarounds/optimisations without the user realizing anything.

So I do primarily App/Extension development, and I've been doing it for about 15 years now.

The short answer: You must have an interface you control, because app store review times mean there is no "hot fix".

When something upstream of the client breaks - you are DAYS (assume 24 to 48 hours for review, after you fix the bug) from being able to get a new client in front of users, and several weeks out from having all of your users (all being some accepted percentage - I use ~95% for my releases) actually update to the latest version.

If that "something upstream" is you - you can usually plan for it, release the updated client ahead of time, get it rolled out, then pull support for the old feature.

If that "something upstream" is reddit... They break things, and they might (but probably won't) inform you ahead of time. You will have many, many more outages.

Saved many long outages by being able to push quick dirty fixes to upstream APIs from our middle/server layer that depended on 3rd party APIs.

Also a place to pull in any access keys needed for other services if they connect to so it doesn't sit in the client code.

All the comments in this thread are right. This specific scenario was about providing a consistent experience. All the work that this backend does could be done using iOS’ background updates, but:

1. Those aren’t consistently scheduled 2. They have an extremely tight deadline (and sometimes Reddit can be a bit slow)

Making it server side means we controlled that experience better.

This is only used for the push notifications in the subscription version of Apollo. I haven't checked the code but it probably polls the Reddit API with the users OAuth token and sends a push to the Apollo app if new activity is found.

I really love the practice of releasing the source code once a project comes to a commercial end, and I respect the devs greatly for doing so. Good job!

What if all the blackout communities move to an open source, community funded, community owned and run version of Apollo?

I’d be happy to try and convince my 100k community to Apollo and if the UX is similar for web and mobile then I’m not sure switching costs would be that high during the blackout.

I’d help fund it but all my money is wrapped up in my trash cooperative.

I haven’t seen a compelling OS alternative to Reddit yet. The ones I’ve seen have all either been big on decentralization at the expense of long load times and a lot of UI complexity, or big on 4chan-style dialogue with little moderation. Neither of those sounds like a great migration path. IMO this is the same problem “migrate off Twitter” efforts had.

Have you seen Tildes?

- incredible moderation and respectful community

- you can't sort by controversial

- non-profit, no investors can corrupt it

- beautifully simple UI, better than old.reddit on mobile

- fast and not decentralized

- no images just text

- open source

- The site is the main mobile interface, not an app

- No limits to logged-out browsing

- Completely functional for browsing without javascript

- Zero third-party scripts/assets during normal use

- Uses modern versions of simple, reliable, "boring" technology

https://docs.tildes.net/philosophy/site-implementation

This looks interesting for sure thanks for the link

I want to like Tildes, but to be completely honest, it annoys me that the '~' character is used heavily. Both from a look perspective and a typing perspective.

Maybe I could get used to it overtime, but we'll see if it ever becomes non-invite-only.

I’m interested in trying it. Do you have an invite link by any chance?

I can send you an invite if you put a means of communication on your profile for me to reach out to.

Please see updated profile and many thanks

Sent!

Interested to get on tildes too.

Wondering if you still have any invites to send my way?

Email in profile. Thanks!

That's by far the best Reddit alternative I've seen. Still, I feel that "no images just text" and "invite only" (although I 100% respect the rationale for both) make this a non-starter for any sort of mass-migration.

Why inability to sort by controversial is seen as a positive?

Prevents a community of people awarded for throwing rotten vegetables at people like Twitter and Reddit.

> I also notice that there is no such thing as sort by controversy

> This is one of the things I like about Tildes so far. An option like that is specifically designed for hate. I want to spend my time on these sites talking with other nerds about homelab stuff and niche 30 year old RPGs no one cares about. Not feel the urge to constantly validate myself by looking at comments people are metaphorically laughing at for being ridiculous

https://tildes.net/~tech/15v5/stop_trying_to_make_a_good_soc...

Some threads on Reddit represent an echo chamber and I like then to sort by controversial to check if there are also people with a bit more divergent opinion. I was not aware of this being a wider problem for people.

> An option like that is specifically designed for hate

This is very short-sighted, I feel. The controversial comments often have the most interesting discussions, and helps to avoid the issue of the top-voted comment being a popular, but incorrect, view/discussion where all the activity happens.

The effort I've seen pointed to a "recommended lemmy instances" page which had a wide range of things from the anodyne to the horrific.

I can't think of much else that could have done a better job at putting me off from ever touching Lemmy than some of the descriptions of federations it "recommended".

Also, by being open about how many people were federating in each community, it also made it clear that even the biggest server was smaller than most discords.

I don't think it's too bad, I see like one loli server but everything else looks reasonable.

I don't disagree, but Twitter never had a site wide Billion user blackout

If we blackout long enough and then figure out how to cooperate to create an actual community alternative in the blackout period, it could work

Not easy and will take real leader with 80 hour a week work ethic for a year, and probably a few million dollars to start.

However it would be an amazing groundbreaking feat

I was very confused about why you would tie your money up in a cooperative you think is run very poorly, then discovered it’s literally about waste management!

Ha yes this is a common response when I say "My trash cooperative" sign of the times I suppose

I should probably just say "recycling cooperative"

Apollo isn’t a Reddit competitor, you’d have to build out the whole Reddit backend for it to talk to. Christian understandably said he’s not interested in working on that himself. If he open sourced the client and it could be forked to work with a Reddit competitor that’d be amazing, but I’m not sure if he plans to open source that.

Fair, however I think if we just assume that the community is what is important then we can bootstrap into a new "clean" architecture - but that's going to take leadership and someone with cash to spin everything up over the weekend basically.

I think it's doable, but we need a leader to step up.

The founder of Apollo literally said in his shutdown announcement that he has 0 interest in what you’re describing.

Politically reddit have played a blinder.

They've succeeded in killing their main competitor (in terms of ad share) and could even now walk back the api charges for any smaller operators and leave Apollo dead and avoid the biggest backlash.

It's scummy as hell and I won't want to continue to contributing to reddit, but I can't help but feel they've been shrewd in getting what they wanted.

Does Apollo have any ad share?

It has a reasonable chunk of the most active users.

Sure, it doesn't have ads in today, but it could, so there is opportunity cost.

What's a reasonable chunk? I had never heard of Apollo before this and I visit reddit every day.

It doesn't show adverts that otherwise would be shown on the official reddit app.

Sync is similar. I have a one-time purchase for "ad free" Sync. That stops me seeing adverts I would see in the official app for a fraction of the cost of reddit premium.

Why would that leave Apollo dead? If they set the API charges back to zero all third party clients, including Apollo, would be happy to continue.

I said they could walk back the charges for smaller operators, e.g. by having generous API limits that most third parties wouldn't hit.

How do you know? I would definitely not come back after getting screwed like this.

He makes money from Apollo. Unless he has another job lined up, or a sizeable savings, he would probably enjoy his income staying as it is.

he does well https://www.macrumors.com/2022/09/16/apollo-pixel-pal-dynami...

> The goal of making the code for this repo available is to show that despite statements otherwise by Reddit administrators, Apollo does not scrape anything and users purely authenticated Reddit API requests, and does a great deal of work to ensure the Reddit API rate limits are respected.

If this is true why does Apollo need its own server at all? Couldn't the client make requests directly to the reddit api servers instead? Then each user could use their own api key, rather than Apollo having to pay millions for a single key that accommodates everyone.

Now it's open source, you can look at the code and find out why.

Theoretically it could do it all from the app, but it would have to resort to background updates, which are scheduled at iOS’ discretion. Christian was aiming for consistency, which is why this exists.

The main reason is push notifications. A lot of the communication can be direct, but not the push notifications.

Reddit's current situation reminds a bit of Yahoo Answers in the late 2000s. Considering they were launched 5 days apart, Reddit has been incredibly resilient.

Please change title to Apollo Backend. Gives the impression Apollo has returned “and” made public.

Also: https://news.ycombinator.com/item?id=36256215

Thanks. Relates to the Github repository, which is source available. Looks like there is no license.

> Looks like there is no license.

No one said they were open-sourcing it. It's just published for information/transparency.

I've seen an increasing amount of support for 'nuking' your personal reddit history—posts and comments—in protest of the upcoming changes, taking back what Reddit are profiting from. I'm hugely in favour of it, anyone else considering it?

No. I could lie and say my time is more valuable than editing my reddit post/comment history, but it’s not. I just don’t care enough to try to “stick it” to Reddit.

I did. Edited post and comments and deleted everything. Takes like 5 minutes.

That’s my plan if they don’t change course by 30th

Direct link to submitted content: https://github.com/christianselig/apollo-backend

Is anyone who has experience with Go able to decipher the instructions in the README?

Reply on news.ycombinator.com