Hacker News Reader

Comment by Ciantic | Hacker News Reader

I presume this is here partly because Threads is in the news. To me interesting part will be what subset Threads will support?

For instance, ActivityPub specs start like this:

"It provides a client-to-server API for creating, updating and deleting content, as well as a federated server-to-server API for delivering notifications and content."

However, that client-to-server API is not even implemented by Mastodon. It's unlikely Threads will support it either, leaving those who want to make own clients in the cold.

Then there is stuff like outbox, which contains past messages, but for instance, Mastodon does not fetch old messages at all, causing a subpar experience: "Older posts from other servers are not displayed" when browsing profiles.

If Threads don't see the need to support outbox either, it makes browsing Threads content from your own small server an infuriating experience.

View on HN

I posted this not because of 'Thereads' but because I have been trying to implement ActivityPub and somehow I missed this document in my previous searches. I want to implement ActivityPub support for my blog in some form so like RSS, anyone subscribes can listen to it.

This will be helpful for you https://github.com/dariusk/rss-to-activitypub

>Then there is stuff like outbox, which contains past messages, but for instance, Mastodon does not fetch old messages at all

It's infuriating how little support activitypub outboxes have. It seems like the perfect way to make small instances work well.

I think the justification Eugen (Mastodon's main developer) gave for not loading a user's past activities is that it would actually harm smaller instances when a user goes viral and the whole network hammers their servers for their content.

I am not convinced of the reasoning, and, in my opinion it's actually working against the idea of "federation", and even against the whole paradigm the internet is built upon: as a UserAgent you should retrieve content when the user requests it.

Some day computers will be fast enough to serve hundreds of bytes of json many times per second.

They are already if you architect your service for the real purpose, which is to store and process ActivityPub data. Mastodon however is a CRUD application that has its ActivityPub bolted on a totally different data paradigm. It takes a lot of computing to make it run fast.

I created an ActivityPub service which (coincidentally) uses a similar storage idea as HN - raw files on disk - and it runs orders of magnitude faster than a similar sized Mastodon server would. Sadly it's not as fast as HN, but I'm getting it there. :D

I think the person you are replying to was being sarcastic.

they were, but the point was that mastadon is incredibly inefficient and this guy is making a more efficient version

I see your profile is full of a lot of the info about this, do you have a more detailed write-up expanding on how this all works together?

There is some documentation about various things[1], but I'm not entirely sure what you mean about "how it all works together".

[1] https://man.sr.ht/~mariusor/go-activitypub/

> I think the justification Eugen (Mastodon's main developer) gave for not loading a user's past activities is that it would actually harm smaller instances when a user goes viral and the whole network hammers their servers for their content.

Maybe we need some kind of protocol that can spread the load where popular messages are duplicated across the network instead of being stuck on one node that can get overwhelmed.

Since ActivityPub is mostly composed (75%) of immutable json payloads, they are very suitable for caching. The existing infrastructure should be more than enough.

> Maybe we need some kind of protocol that can spread the load where popular messages are duplicated across the network instead of being stuck on one node that can get overwhelmed.

You mean like any proper P2P protocol?

Add to this multi-cast on internet level, and everybody could actually operate something like YouTube on their internet router…

But of course economic interests of some (already ultra rich) minority will prevent that forever, even we could have it just right now form the technical standpoint.

BitTorrent?

Not the hero we deserved but the hero we needed.

The irony and tragedy of the Internet is BitTorrent solved Internet Scale at the protocol level, but is at odds with how content companies, ISPs, and Silicon Valley want to organize themselves.

It's also at odds with how people consume content.

If you cram every social interaction a network has in a massive common distributed store (which is what using a bittorrent like protocol would entail) you have every node requiring to consume gigabytes over gigabytes of content, despite being interested in a fraction of a fraction of it.

I don't know if nostr or scuttlebutt resolve this issue, but if they don't ... how do you imagine a mobile app for keeping in touch with your grandma, requiring gigabytes of data transfer and storage would work?

At this point people saying bittorrent is a solution for social networks are basically just the guys who know about hammers and seeing nothing but nails.

I'm advocating it as the content distribution layer, not the social interaction layer. You don't even need all of the metadata, as there are BEP extensions to distribute that as an index [0] or an RSS-style feed [1]. Images are particularly painful, like the previews in Mastodon, because they hammer the origin server rather than looking around at what other peers have. This problem will only get worse as the federation grows in size, both in users and in servers.

I say irony because we had all these scaling problems two decades ago during the peak P2P years, including federation and implicit DDoSing, and BitTorrent emerged as the lasting protocol because it could handle the scaling issues.

I say tragedy because for this to work you do need some openness, which puts people at risk of content companies trolling for you, ISPs dropping you, and is completely antithetical to the control Silicon Valley demands over its users and data.

[0]: http://bittorrent.org/beps/bep_0049.html

[1]: http://bittorrent.org/beps/bep_0039.html

Then the fediverse is already there. The video hosting service PeerTube uses bittorrent to distribute the video among viewers.

There were some plans at one point for another project which would deal with music and would host the media on IPFS. Sadly I've never seen any progress on it, despite the idea sounding very good.

It’s not so much Silicon Valley demanding that control as Hollywood

The mobile app isn't a peer in this hypothetical torrent network. Your mastodon instance is and it serves you using the same client protocol we have right now.

That way, you solve for the potential hosting issues of smaller hosts not being able to handle the load in case of viral posts etc, all without the clients having to know anything about the implementation details.

If you still require servers that you need to connect to then you kind of lose the benefit of having a social network over a p2p protocol. Or you're also thinking that the social serving aspect is parallel to the media serving aspect that another sibling mentioned.

Modern smartphones are like the computers end of the 90's.

We've run back than Napster and later eDonkey on those boxes just fine…

Yes, but you were downloading a movie and then you'd delete it to make room for something else on your 40GB HDD. In this case it would be like you'd try to download the whole internet because you're interested in one movie.

It's possible to download parts of torrents though.

NNTP

Maybe we don't need to value virality so much.

I doubt "valuing virality" is the real problem. But when you're exposed to a universal network whose numbers increase every day, there are moments when it happens no matter if you want it or not. The "slashdot effect" has been coined maybe 25 years ago, so it's obviously not a new thing. :P

Virality will happen regardless of your intentions. Your systems need to be designed to handle it.

I don’t understand why other instances wouldn’t cache the older content and periodically look for updates

https://www.w3.org/TR/activitystreams-vocabulary/#extendedty...

> Support for specific extended vocabulary types is expected to vary, with implementations only selecting the extended types and properties that make sense within the specific context and requirements of those applications. However, to avoid possible interoperability issues, implementations must avoid using extension types or properties that unduly overlap with or duplicate the extended vocabulary defined here.

The client server protocol lacks support for many basic operations, like search. It's a late addition to the spec, because somebody felt a server protocol shouldn't be released without a client protocol. Swap the polarity of the inbox and outbox, and presto you've got a client protocol. Except clients don't want to talk to servers the same way servers talk to other servers.

It seems logical if Threads focused on client-to-server interoperation first, since in theory that would allow more people to use third-party clients but not third-party servers so Meta gets a hold of all the unencrypted messages.

I’m pretty sure they are going to do the opposite, support server-to-server, and never support client-to-server. They have been meeting with the maintainers of Mastodon and admins of large instances. Their stated goal is to allow migration off without losing your social graph (this is obviously a lie about motivation, but that’s not what I’m taking about here). This is all about server-to-server implementation.

So, they are going to be part of the server-to-server Fediverse, and keep all of the juicy app data from Threads users to themselves.

Yes, plus if they support client-server then kill it later, they risk the same kind of backlash that happened to Twitter and Reddit when they killed third party clients. But if they only support server-server when they kill it, they can just send an email to all the geeks telling them "import your account into threads by X date!" and most "normal" users will never notice

Supporting server-to-server ActivityPub is slightly risky for them as it gives people an alternative, and there will always be outward bleed. On the other hand I think it takes a lot of pressure off potential legal troubles around being a monopoly (which they must be worried about if they actually manage to kill Twitter).

I predict if they manage to be the overwhelmingly dominant platform in the Fediverse, they’ll keep it working. If they see another big competitor jumping in and taking a lot of users (or hell, if Mastodon starts to eat their lunch), I’m not so sure.

Is that client-server API robust enough to .. care? Why don't other (any other?) FOSS implementations support it?

I'm toying around with my own and that's interesting. I'd love to support it but if no one else supports it no clients will support it either. Granted eventually i expect i'd want some client-server API so i may as well use ActivityPubs. Nonetheless the lack of adoption feels odd.

There are other projects that support the client to server part of the specification (see my profile for an example in the FedBOX project, and from the more mainstream ones there's Pleroma AFAIR). Generally developers justify not adding it to their implementations with the fact that is not specified in sufficient detail. Personally I think it's mostly laziness and a lack of imagination - but I'm biased.

Reply on news.ycombinator.com