I wish the artists well in their AI copyright legal pursuits.
In contrast, DeviantArt saw dollar signs from AI art and rugpulled themselves. Their business model relies on art remaining scarce enough to not exhaust the demand for art. A machine that lets you create unlimited art for the cost of some GPU time completely destroys the economic underpinnings of most artistic endeavor. While not all artists are solely economically motivated, the ones that are economically successful are the ones paying for dA subscriptions - the things that keep the site alive.
Before, under Yahoo, they'd just put some new hoop in the app to prevent users from accessing adult content, but by the time this particular Apple review rolled around, the sale of Tumblr to Verizon had already been finalized. Which created a situation where an outgoing management pretty much ordered to not bother fixing it and just banning it all, hoping that Verizon's "family friendly" policies meant that it wouldn't jeopardize the sale (which it turns out, the sale wasn't jeopardized).
You can still kinda see it in how broken the actual removal was; they just excised the NSFW from the frontend by marking the posts as sensitive on the API and then preventing the frontend from viewing anything sensitive. For years (and maybe even today) you could just scrape the API to find NSFW posts, although that's on the decline since most NSFW Tumblr accounts have been deleted entirely by the actual people behind them.
That's not true - Tumblr already had their "no adult content" plans in place well before the CSAM problem caused Apple to temporarily suspend them from the app store. That suspension just brought the plans forward by 6 months in a panic rush.
And it's not just Apple, payment processors also have strong opinions.
In general, sexual/erotic stuff has become a really hard thing to keep allowing in mainstream platforms.
Using that situation as a reference point, I'm not totally convinced that AI art will supplant digital art/drawing. AI art can be used to produce some types of drawings, sure, but not all types of drawings can be convincingly produced using AI. AI's stylistic and compositional abilities are constrained to its training set, for example.
Another thought: when the DSLR became affordable in the early 2000s, it wiped out an entire segment of low-level professional photographers. Suddenly, any schmuck with a DSLR could take "good enough" photos with ease, and without the friction of film. I'd expect AI art to have a similar effect in the realm of digital illustrators. But, professional photographers still exist, and so will professional illustrators.
Social media sites that are still founder-owned or have strong individual leaders can continue fine (consider e.g. Dreamwidth). Though I guess whether you can sustain that past one person's lifetime is another question.
ActivityPub, used by Mastodon, Misskey, Lemmy, and Pixelfed among others is an example of such a protocol. BlueSky's ATProto is another, though it's in an earlier stage without mature third-party implementations and service providers. Email, too is decentralized, though it may serve as a cautionary tale; spam, attempts to block spam, and feature stagnation have all degraded the user experience considerably.
it's no wonder that the fediverse is most active on the fringes, especially outside of tech bubbles who use it because they like the idea
That's not to say a million fiefdoms is necessarily a bad approach. A small server where all the members know each other is much more likely to be run in a way that's satisfactory to all its members than a big one. Furthermore, users have the option to maintain multiple accounts.
And? I guess that somewhat hinders enshittification just by making it hard for the platform to ever evolve at all. But the cure is worse than the disease, you can't ever build something new that way nor can you really improve something that has any level of traction. Look at how IRC users revolt when you try to fix even the most glaring problems.
ActivityPub dictates how to meditate relationships between activities on collections of objects, and a few default objects types. It's specifically designed to let you "subclass" (not really inheritance, and more composition as you give a list of types with no enforced hierarchy) objects so you can create new object types nobody else understands but still give them another type that allows them to carry out basic operations on it.
It's not perfect but it's far from as dire as you make out.
Enshittification is hindered not because nobody could create a Mastodon fork (or green-field project speaking the same protocol) that's riddled with ads, but because people can select a different service provider and still access the same network.
In theory, certainly.
In practice, one instance dominates and all the other instances have to censor accordingly or die.
Why not start a centralized social-network service as a non-profit / benefit corporation, paid for by donation?
A social network has to make money somehow because it has bills to pay. Hosts aren't free. Servers aren't free. The cloud isn't free. Staff isn't free. Moderators are usually free but shouldn't be.
Sure, but probably a FOSS social network would need far fewer of these than a paid one, because 99% of the server costs of something like Facebook or Twitter, go toward the backends, analytical DBs, and graphical / ML models used to power "features" that no user wants, but which make Facebook themselves money.
And a FOSS social network would just... not build those kinds of features.
Instead of, say, a feed constantly rebuilt to drive engagement and rage-bait, you'd just get a simple chronological feed of what everyone you're following is posting; with maybe the ability to dial down the number of posts you see from any given person you're following ("show me the only the most-liked Nth percentile of posts from this user") without unfollowing. But even that kind of filtering — in fact, even the merging of followed users' feeds! — could all be done client-side. The whole "feeds" part could be as simple as a post-triggered static-site-generator pushing JSON into an object-storage bucket hiding behind an edge cache.
And I may be wrong but I don't think the recommendation algorithms and other such features take up as much of the cost as you're claiming. I think a lot of the cost of something like Facebook is probably taken up by infrastructure and storage. Recommendation algorithms probably aren't that expensive.
Having extra entire complete copies of the relationship-plus-posts graph, denormalized in various ways (incl. in ways that inherently prevent use of easily map-reducible algorithms, and so require heavy vertical scaling) such that you can run the algorithms, is what’s expensive.
And constantly feeding the data into those denormalized models, using specially-tuned realtime ETL technologies that themselves do distributed scaling to ensure no infinite queue backlogging from activity bursts, is also expensive.
I don’t think it’s true that it’s intrinsically impossible for a public service to be self funding, and I think that not everything has to grow / change forever to remain relevant.
We need to figure this kind of stuff out, I mean Wikipedia is nice and all but it’s really bad that humanity in general has to rely on megacorp for things as basic as maps while we say we’re living in the Information Age
There's other AP implementations that aren't a constant server hog like Mastodon is and can run on much weaker hardware (some of it can run on a raspberry pi). You don't need a full rails stack if your user count never exceeds 100 (which y'know, is the ideal state of AP - small communities who can remotely interact with each other).
Modern Usenet :-)
This is incorrect. From direct personal experience, I strongly believe the backend infrastructure and staff required just to operate the core product functionality of a successful large-scale social network massively exceeds what could be provided by donations.
Just in terms of core product OLTP data, Tumblr hit 100 billion distinct rows on MySQL masters back in Oct 2012. At the time, after accounting for HA/replication, that required over 200 expensive beefy database servers. This db server number grew by ~10 servers per month, because Tumblr was getting 60-75 million posts/day at this time.
Then add in a couple hundred cache and async queue servers, and over a thousand web servers. And employees to operate all this, although we kept it quite lean compared to other social networks.
Again, this was all just the core product, not analytics or ML or anything like that. These numbers also don't include image/media storage or serving, which was a substantial additional cost.
Although Tumblr had some mainstream success at that time, it was still more niche than some of the larger social networks. At that time, Facebook was more than 2 orders of magnitude larger than Tumblr.
Because these social networks are designed for analytics. It's in their blood. It permeates everything they do, and causes immense overhead.
Check out mailing lists or usenet.
Social networks store a lot of OLTP data just to function. Every user, post, comment, follow/friendship relation, like/favorite/interaction, media metadata -- that all gets stored in sharded relational databases and retrieved in order for the product to operate at all. For successful social networks, it adds up to trillions of rows of data (on the smaller end, for something like Tumblr) and that requires a lot of expensive infrastructure to operate. Again, none of this has any relation to analytics.
As for usenet, what? It's basically dead, after becoming an unmanageable cesspool of spam (or worse) more than two decades ago. It was great in the 90s, but the internet population was substantially smaller then.
Yeah, because no one is interested in promoting it because it doesn't have analytics baked in so you can't make money from doing so. Of course it deteriorated over the years. It's also cheap to run and can handle a massive amount of users.
> store a lot of OLTP data just to function
Right, so they can run analytics. You could reduce your tracking data to aggregates, but then you can't go back and run analytics on your users. You don't need to keep that data forever.
Especially with modern social media where content older than a day is effectively dead and ignored.
> it adds up to trillions of rows of data
This was a lot of data a decade ago. Nowadays a single postgres instance will handle billions of rows without breaking a sweat, and social media content is exceptionally shardable.
Stop gaslighting me, it's not OK! I'm describing first-hand experience of things that were not related to analytics IN ANY WAY, SHAPE, OR FORM.
Try running OLAP queries on a massively sharded MySQL 5.1 deployment, or any aggregation at all on a Memcached cluster. These technologies were designed for OLTP data, and were woefully incapable of useful analytics over massive data sets.
I was Tumblr's fourth full-time software engineering hire. When I joined (nearly 4 years after the company was founded) the only thing remotely related to analytics was a tiny Hadoop cluster, where logs were dumped and largely ignored. Nothing about analytics is "in their blood". All you needed to sign up for Tumblr was an email address. WTF do you even think they are "analyzing"? Your comments are completely fabricated BS.
> You could reduce your tracking data to aggregates
Once again, I'm not describing "tracking data"! I'm talking about things like content that users have posted, comments they have written, content they have favorited, users they are following. These are core data models of a social network. It has nothing to do with tracking or analytics.
> You don't need to keep that data forever.
The OLTP product data I'm describing does need to be kept forever. Users don't like it when content they have written on their blog suddenly disappears.
> Nowadays a single postgres instance will handle billions of rows without breaking a sweat, and social media content is exceptionally shardable.
Yes, but running a massive cluster of hundreds or thousands of sharded database servers is still very expensive.
There is no central concept of "content that have favourited" or "users they are following", that's all handled locally in that model.
Not by volume. Posts and comments make up the vast majority of the storage requirements, and none of that can be purely client-side.
> There is no central concept of "content that have favourited" or "users they are following"
I'm aware, I used Usenet quite a bit in the 90s, as well as dial-up BBSs.
Usenet is a distributed forum / discussion board, which is related but not equivalent to the core functionality of social media applications being discussed here.
With Usenet's model, there's no concept of a profile aggregating content from a single user. This means you simply cannot replicate the primary experience of Facebook, Twitter, Instagram, Tumblr, Pinterest, DeviantArt, MySpace, Friendster, or any other social media site/app with Usenet's approach. Nor can it reproduce the experience of even modern forums like HN or Reddit.
Usenet also didn't actually scale massively. Every estimate I've seen of the peak Usenet userbase puts it at a tiny fraction of modern social media.
In any case, Usenet essentially failed. We already have empirical evidence about how these ideas play out! Why are we even seriously discussing this?
These aren't really good "social media" examples. Both mailing lists and Usenet have limited retention, with mailing lists there may be almost no retention beyond the amount required to deliver a message.
While low retention might be a desirable feature and something you might actually want in a FOSS social network, it means old content will disappear from the central server. If it's not archived by clients it can easily disappear or end up locked away only in private backups. Google's buyout of Deja News should be a cautionary tale of retention and the locking up of public data behind a private gate.
Usenet history today is largely only available because someone at Google hasn't noticed Google Groups still exists and terminated it yet. If that happens tomorrow there's not any good complete archive of historical Usenet content. There's no guarantee Google won't kill those Usenet archives in the next year let alone the next five years.
We have 1.5 trillion distinct rows living on a PG cluster right now. Growing at a higher rate. And that requires... four DB servers (four shards). And no replicas — all our prod query traffic hits these servers directly, as a mixed workload with the writes from our queue-consumer ETL agents.
(The servers are "beefy", sure — but they're not that beefy. They've got 128 cores and 1TB of RAM each. That's not even top-of-the-line these days.)
Despite serving a whole ton of QPS, these DB servers are not CPU-bound or memory-bound or even IO-bound. Our primary horizontal-sharding factor, in fact, is PCIe lanes — we're constrained mainly by the number of NVMe drives we can keep connected performantly per CPU [1] — and thereby, generationally, by "fast storage capacity" per server platform.
One thing that's perhaps "special" about our use-case, though, is that our data is inherently append-only. Which is pretty great for OLTP performance: there's very little write contention, as we just partition the data by insert time (which is also a natural partitioning key[2] for most queries our business layer does) and then only write to the newest partition, with all previous partitions becoming effectively immutable.
Most workloads aren't like this... but they could be, if you model your data carefully, with temporal tables holding versioned records and so forth. You trade off more storage growth, for far less worry about write contention. ("Event streaming" is not the silver-bullet solution it would seem here — as you would still need to create an aggregate to query from; and writing into that aggregate would still be an OLTP write bottleneck. No, the true solution here is data-driven [schema] design — i.e. forcing your API engineers to invite the DBA to feature design meetings, and ensuring the DBA has the perogative and incentives to say "no" to a bad design. Or you could just hire database systems engineers to code your business layer, like we did.)
My point here, is that if you're very careful in designing for not just "scalability" but economies of scale — and if your org is engineering-driven rather than product-driven (as a non-profit social network would likely be!), and so has "mechanical sympathy" — then you can achieve things with just the budget of an average B2B SaaS, that would rival a (smaller) social network.
But with the budget of a benefit corp that nevertheless charges even $1 for each install of its social-network mobile app? Who knows what you could achieve!
(See also: WhatsApp, pre-acquisition. They were just charging $1 for each install. With just 50 employees, that got them a lot of operating budget.)
---
[1] And actually, that's part of why the DBs are not IO-bound. Put enough NVMe disks in RAID0 together, and you get something nearly functionally indistinguishable from memory. (Yes, RAID0 — because our DBs aren't the store-of-record; our data lake is.)
And even where NVMe reads are not indistinguishable from memory reads, they're still complementary. Postgres, in relying on mmap(2)ing heap files and thereby on the the OS page-cache as a buffer pool, makes per-query serial page faults expensive, and causes multiple threads wanting the same cold data to bottleneck by repeatedly becoming coalesced awaiters on the same sequence of pages faulting in. But when you've got highly-concurrent queries [= many different OS threads] that are all page-faulting for different cold data, then you get deep IO queuing, and everything works out optimally. So the expensive cases don't really come up — the page-cache serves the head of the request distribution, while the deep IO queues of the RAID0-NVMe-pool are a perfect match for the long tail.
This makes it almost irrelevant, at runtime, whether data is hot or cold. As long as you have sufficient memory to host the very hottest data (e.g. relatively-low-cardinality fact tables like users/blogs that get joined to everything, and esp. their indices), the rest can be read straight from disk with nearly no penalty.
---
[2] Funny enough, given that we're doing heavy joins for some queries, the one thing we do sometimes run out of at runtime, is timeslices of the Postgres postmaster to coordinate lightweight locking of the global locks table.
We not only have time-partitioned tables, but also something akin to tenant-partitioned schemas. This results in a lot of relations. (It takes hours for us to run vacuumdb --analyze-only.)
And some OLAP-ier queries (that we nevertheless have to run, synchronously, in response to client requests!) need to touch many of the partitions and many of the schemas. That's O(MN) locks they need to take — which take time to write into the global locks table, even though those are only memory writes.
Every once in a while, we have to "consolidate" our table partitions — not by rolling them up with aggregations to reduce row-count, but rather by just copying all the data into fewer, larger partitions — so that we can take fewer access-shared locks per query, so that transactions can spend less time waiting on a handle into the global locks table at startup just to write these O(MN) read locks into it. (In other words, we consolidate data to reduce O(MN) read locks per transaction, to O(N log M) read locks per transaction.)
But that's a PG problem, not a resource problem per se. We've been considering writing a patch for PG to allow relations to be marked as "immutable", where an "immutable" relation doesn't have locks tracked for it (or its indices) at all — but also can't be written to, re-indexed, or even dropped. We'd then just apply this setting to all of our historical partitions. (If you're curious: given its semantics, this property could be enabled for a relation at runtime; but would need an instance restart for disabling it to take effect, as the instance would have no idea what read-locks "would be being held if the table would have been tracking locks", and so couldn't safely do any writes to the table without a barrier that drops all existing MVCC read-states — i.e. an instance restart.) As a bonus, such relations could also have "perfect statistics" calculated for them by ANALYZE; and those stats could then be marked as never needing to be re-calculated — allowing both VACUUM and ANALYZE to be skipped for that relation forevermore.
"Four shards" and "no replicas" is a complete non-starter for a popular social network. That low shard count means a hardware failure on a shard master results in downtime for 25% of your users. Putting ~400 billion rows on a single shard also means backups, restores, schema changes, db cloning, etc all can take a tremendous amount of time. No read replicas means you can't geographically distribute your reads to be closer to users.
I'm not sure why you're assuming that social networks aren't 'very careful in designing for not just "scalability" but economies of scale'. I can assure you, from first-hand experience as both a former member of Facebook's database team and the founding member of Tumblr's database team, that you have a lot of incorrect assumptions here about how social networks design and plan their databases.
I can't comment on any of the PG-specific stuff as I don't work with PG.
Append-only design is not usable today for social networks / user-generated content. That approach is literally illegal in the EU. In years past, afaik Twitter operated this way but I was always skeptical of the cost/benefit math for their approach.
The "200 expensive beefy database servers" I was describing were relative to 2012. By modern standards, they were each a tiny fraction of the size of the hardware you just described. That server count also included replicas, both read-replicas and HA/standby ones (the latter because we did not use online cloning at the time).
I haven't said anything about event sourcing as it generally is not used by social networks, except maybe LinkedIn.
Didn't we have some more forms of government available for discussion?
E.g., after the xz backdoor, I read a call of OSS maintainers that critical OSS projects should be state-funded as they literally comprise critical infrastructure. Why couldn't we do the same with social networks?
If the goal is to avoid "enshittification" I don't think the solution is to have governments control social media and then have those platforms be subject to bureaucracy, decency and profanity laws, surveillance and propaganda (more so than currently) and the fickle wrath of the taxpaying voter. Realize how little critical infrastructure actually gets funded, and then add the psychotic dog-water paranoia of half the US thinking that infrastructure is a psyop by communists and "groomers" to turn their kids gay, and making that an issue in the polls.
PBS is probably the closest analogue to government run social media I can think of. Half the government and their constituents consider it "liberal propaganda" and want to defund it entirely, and it has constantly has to go begging hat in hand just to stay afloat, and then pursue commercialism to make up for the deficit.
You think social media is bad now? Imagine if you're required by law to sign up with proof of citizenship and your SSN is your password. And every platform is constantly putting Wikipedia style donation popups. And it's a misdemeanor to post swear words or material deemed "inappropriate."
Functionally, the “public good” part of social media networks is almost certainly better served by a single organization.
However, the “freedom of speech and ideas” part, runs in horror at this idea. (rightfully so).
The best middle ground concept I’ve heard is to contrast the current state of the web with libraries.
NB: Enshittification is going to become a term like “Fake news”, completely divorced from its original roots.
I would expect government run social media to be especially enshittified, honestly
Just for different reasons
There are countries which CAN make it work, but man, a central nervous system co-opted by oligarchs, tyrants or other worst case scenarios, would be the outcome for most of the world.
Whenever a platform is owned by shareholders who then need to extract rents from the ecosystem, this will happen. Whether it’s couchsurfing or twitter.
Expect it to happen to Reddit etc.
There is a direct line from the profit motive to platforms becoming enshittified, promoting the most outrageous content and making people emotional and angry.
The AI is just another level of appropriating human work. Whether it’s google’s disruption of publishers through AI-generated answers, or OpenAI training on artists’ work.
These platforms need money to survive. Automattic, the latest owner of Tumblr wrote a great post on all the things they've tried and how Tumblr is still losing $20MM a year IIRC.
Wordpress by that same Automattic doesn’t need money to survive, in the sense of money going to one large corporation. Anyone can self-host their own copy of wordpress, buy plugins etc.
If you want to know more about how to monetize digital content without a trusted central actor, we are working on a Web2 version of that ecosystem btw: https://qbix.com/ecosystem
Also, science and wikipedia and openstreetmap are examples of open gift economies.
Sorry the experimental stuff is not slick enough for you yet, we don’t have the resources of Facebook or even Automattic. We worked very hard for 12 years on the foundations at https://github.com/Qbix/Platform but I am sure you can find many faults there. (I’d like to hear about them btw.)
On the other hand, many other projects like the E programming language, Capn’proto, Linux etc. are also very complex and did not have fantastic and slick documentation, first adopters also had to read some words in order to get it.
This is an open source project. You are welcome to reduce the words and make a summary. Perhaps when we start marketing to a broad audience, we’ll reduce it to 10 word slides and sound bites, or jingles.
Until then you can try it yourself, the documentation is at https://community.qbix.com and technical documentation is at https://qbix.com/platform/guide
Wordpress' Tumblr problem is that costs exceed revenue, not distribution.
Note that "decentralized" doesn't reduce total costs - it just spreads them out. Server costs do not go down with the number of owners.
At a certain point, people seem to start looking at self destructive options to make that happen.