Reddit has been around nearly 20 years to harden their infrastructure. Relax and give them a minute. Their page loads for me now that I've typed this out, anyway.
Comparing it to Reddit is silly. And yes I’m sure there are more than a few here that can honestly say their servers could handle this load - it isn’t much.
As far as I'm concerned there aren't any core issues. There are a lot of queries to a single DB which is one the same server, but there's nothing preventing you from running the DB on another server and sharding it if need be. I wouldn't call that a core architectural issue because the way you set up your DB is not core to the architecture.
Yes of course in theory you can beef up any instance, and shard, and add a caching and queueing layer while you’re at it. And tada you’ve just redid Reddit.
As the ecosystem grows, a shortlist of popular, robust, federated instances will crop up for people who just don't want to bother.
Again, if you have any core flaws, I'm all ears.
I didn’t say it did, but it doesn’t enhance it either.
> As the ecosystem grows, a shortlist of popular, robust, federated instances will crop up
How short is a shortlist? When does that mean just new Reddit?
I suppose I would be more enthused if at the least the basic design eased standing up high traffic (or let’s be honest even mild traffic) instances. The performance story right now is: it’s written in rust - which is not nothing but it would be more interesting if supporting even a moderate amount of traffic on minimal hardware was an architectural priority. The flaw as I see it is that this is just not a design goal.
https://github.com/LemmyNet/lemmy/issues/2877
https://github.com/LemmyNet/lemmy/issues/2910
You can argue that things can be improved but initially well engineered systems help to enhance initial mindshare.
The performance story isn't just that the backend is written in Rust. It's also that the front is very lightweight (80kB), that the architecture is horizontally scalable, etc...
I don't see in any of those links a core flaw, a problem in the initial engineering that makes it impossible for the architecture to scale. All I see are some poorly written queries, for which the main devs made a root cause analysis and described how it can be fixed. Is that evidence that the inherent design isn't capable of adequate performance? No, it isn't. It's evidence that there are some small performance gotchas that can be fixed easily, and this is normal for an ambitious project.
But even as it is, yes, it can handle mild traffic. lemmy.ml runs on potato hardware, hexbear.net is an example of an instance which has ~3'000 comments a day, which is roughly the scale of this website, and it runs fine on a single dedicated server.
I’ll quote the top-voted thread here
>> Nobody wants a federated, slow, difficult to use version of reddit. Nobody wants to choose a server.
> I want this. I want this because it's a sustainable way to have Reddit without the ads. The bad UX is an acceptable tradeoff for a platform that doesn't go to shit.
I’m just not buying how a federated system of isolated instances solves this. What fundamentally prevents the dominant oligopoly or monopoly server(s) from just being Reddit running on Lemmy? Lemmy doesn’t dictate how things are run - so why won’t a major funded instance just evolve to a new Reddit? How does Lemmy decisively get you to a Reddit without the ads? What stops a major Lemmy instance going to shit?
Just having federation as an opt in feature doesn’t force the system to evolve in a particular way.
If a “Voat” equivalent pops up it’s not like the dominant instances are going to federate with it.
A federated system of isolated instances doesn't solve it, because that's not a federated system. The point of federation is that the instances aren't isolated, and they're highly interchangeable. No major instance has the incentive to become a new Reddit, because it's so easy to switch instances at every level that they just don't have the moat to make that happen. It's exactly the same thing as an email provider going to shit, but even less problematic.
If a Voat instance pops up, sure, dominant instances won't federate with it. That's perfectly fine. It doesn't mean that the inter-federated dominant instances can get away with pulling a Reddit.
Federation is an opt-in feature in theory, but in practice, what's the value proposition for an instance to turn off federation today?
It's really quite simple. Without federation, there is an incentive to pull a Reddit. With federation, there is no longer an incentive to do so, because you don't have a moat. What do you think would happen if Reddit only had 1/2 of the subreddits anyone used and if you could keep access to all the same communities on a competitor?
Domination is orthogonal to a technical federation feature. Once there is enough imbalance you defederate and that’s that. There’s nothing that inherently prevents gross imbalance from forming and the natural forces favoring centralization - such as funding one beefy instance - still apply.
> value proposition for an instance to turn off federation today?
Maybe not today, but it would be the same as any historical netsplit.
> What do you think would happen if Reddit only had 1/2 of the subreddits anyone used and if you could keep access to all the same communities on a competitor?
I think that puts them still in a fucking dominant position. And why automatically assume competitor vs cabal?
Anyway good luck with your project.
I’ve been using the internet since IRC and Usenet - both federated in their own way - both completely marginal.
IRC was never federated. Usenet was, and it did die out - but so is email and it's working far longer than it had any right to.
What? What do you think the term “relay” in IRC means? The jargon term netsplit used even for newer federated networks (even used in these comments elsewhere) comes from IRC. There is literally an entire network named after a defederation event.
Now we sort of take for granted that IRC is basically a closed federated system - but the original design of the network was one dominant set of relays - EFnet is a direct descendant of this network after all - if anything it’s just a specific example of politics and network evolution. There are technical reasons as well - but at the time of the early splits of the 90s (EFnet, Undernet) it was not primarily technical problems.
Anyone involved in a fediverse I think would do well to learn some lessons from IRC even if their system is technically superior.
> but so is email
As a federated system barely - go try to stand up an email server on your home network or VPS and see how well that works. It’s run by a cabal of large providers.