Spritely – leveling up the federated social web
spritelyproject.org
spritelyproject.org
It appears to be both a game and a messenger, sort of like a distributed MMO.
Edit: Someone please correct me if I've misunderstood.
> Spritely consists of a number of modular components bringing new and rich features, from distributed programming, to decentralized storage, to virtual worlds.
"Fantasary" in particular seems to match with your description. I'd guess it would be built on top of the other mentioned things.
I had read it as all of the libraries/characters(?) being different parts of the same whole project, with the whole project being to achieve that goal.
Thanks!
ActivityPub is a general protocol for decentralised social networking. Instead of Facebook or Twitter servers generating a feed for each user, user's servers send updates to each-other.
Hi! Spritely's main author here. Spritely is taking on quite a few things, so it's understandable that it might not be completely clear what from the tagline.
Here's the short version: we'd like to bring better security, privacy, and rich interactions to the federated social web. I was one of the co-authors of the ActivityPub standard so I have some strong opinions about what it's capable of that isn't happening yet.
If you want a friendly, semi-visual overview, here's a video I just recorded for the ActivityPub conference: https://conf.tube/videos/watch/18aa2f92-36cc-4424-9a4f-6f2de...
It goes over (most of) the layers in a fairly friendly way.
Within all the subprojects listed, the most ambitious is the Fantasary one, which is distributed virtual worlds. Not as out there as it sounds, such a thing was built in the mid/late 90s: https://www.youtube.com/watch?v=KNiePoNiyvE
But in the dot-com crash, went up in flames. However many of the ideas were carried forward by the object capability community, particularly the E language, on which Spritely Goblins is heavily influenced: http://www.erights.org/
The distributed games goal then becomes a driver for the other components, so even if that fails we should hopefully still get cool stuff. Spritely Goblins, the foundational layer, already exists and has support for such features as time travel debugging: https://dustycloud.org/blog/goblins-time-travel-micropreview...
and distributed networked programming, safe to run in a hostile network... here's a chat program where the "protocol" authenticates the users joining the channel and also that the messages came from them, with the chatroom and user "protocol" both written in 250 lines of code combined... and without being planned at all for networked interaction, it just worked over the network... here's a gif of it working over Tor Onion Services: https://dustycloud.org/misc/goblins-chat-captp-onion-service...
No code was added to the "protocol" to expect network interactions; Goblins' CapTP (Capability Transport Protocol) took care of that for us.
Already that should sound pretty cool, but that's just the foundation for the system. Look at the other subprojects to see where we're going from there.
Happy to answer questions if anyone has them!
Good luck with the project, I hope it will work.
So, Spritely's approach to vaporware avoidance is to break things into subprojects and show how we're making progress with demos. If some of the subprojects fail, but the others still exist and are useful independently, that's a big win.
Spritely Goblins is already quite usable, and does quite some interesting things: https://docs.racket-lang.org/goblins/index.html
Even if that's all that succeeds from Spritely (and I don't think it will), I think that's a useful contribution.
But seeing is believing, and skepticism is well warranted. All I can say is, I hope that in two years we can come back to this post and re-evaluate it and see how far Spritely is along. I hope we can prove ourselves. I'm working very hard to do so.
I don't actively use anything on the fediverse and still avoid Twitter, but I would still like to follow the project.
(Also, I hope we can get to the point where we dogfood Spritely for Spritely communications, but I think that's ~1-1.5 years out from now.)
I wouldn't necessarily be worried about having the mailing list be open for discussion, irc/gitlab/fediverse covers that pretty well.
That being the case, all the mailing list would need to do would be send the content/links of/to the fediverse posts whenever there's a large enough update to be newsworthy. I just brought it up because that's a way to make sure people can get notifications without having to sign up to new services.
Some phrases that come to mind as alternate ways of defining the issues being addressed:
"Quality of service"
"Personalized service"
"Features as services"
"Self-healing"
Or maybe go for the classic Unix reference and call them "daemons". Except that would have overlapping technical connotations.
I don't actually have any idea what it's for. Is it a twitter replacement? facebook competitor? Message board for fantasy nerds?
There's no way mainstream users would go for this. It looks like an 90s fan site for some YA fantasy series. They're going to scoff at this if they stumble on this.
As a technical user looking for something I could setup for small groups to use as an alternative to facebook, I can't even tell if this would work or not. I'm still not 100% this isn't some kind of cute indie mmo...
Maybe theya are just trying to recruit early techy users for now?
From what I can tell it seems to be a project with the long-term goal of building a federated VR world atop the Fediverse, with a bunch of building blocks for that split out as separate projects: running untrusted programs from federated sources, storing files on federated sources, transferring money via federated sources, etc.
From a higher level I feel like, you know how there's a whole bunch of projects/companies/VC sinks whose description is "here is a familiar function but ON THE BLOCKCHAIN"? This is a bunch of those but ON THE FEDIVERSE instead.
And with cute painterly mascots instead of weird little single-color abstract logos.
Now ActivityPub is the most widely adopted federated social network standard on the web ever, with dozens of projects using it, thousands of servers, and millions of users.
I did sometimes feel like giving up when I heard that kind of feedback then, but I'm glad I didn't. I'm not going to give up here either.
The project is ambitious. It's broken into many subprojects so that even if the larger goal doesn't succeed, we might still get a number of useful pieces out of it.
But I've learned not to let comments like this dissuade me. If I had, ActivityPub wouldn't have happened. We might fail, but more likely, we'll have a partial success, which is still useful. And it's worth trying.
But my point was that when beginning work on ActivityPub as a standard, the kinds of comments on here were similarly negative. Of course you're not going to get a universal social network standard! Nobody's been able to do it, it's just out of reach. Might as well give up now.
Of course, now that ActivityPub has succeeded, people take it as a given that it was going to succeed. Criticism instead moves to certain design decisions, and that's fair and warranted. But hindsight makes those things that have been done seem as if they were always going to be done.
But at a time, it looked like ActivityPub itself, the thing we do have, could not possibly succeed, and people told me as such in many places, especially on here. Nobody questions that it happened now of course. So now I am proposing a multitude of layers in which we can make things better.
But it hasn't happened yet, and so of course the response is going to be, once again, that of course it's not possible...
So, Spritely isn't just an end-user thing because it's multiple layers, though a few of them are end-user layers (it's more of a laboratory for advancing the federated social web). It's true that it isn't a web standard itself, even though it aims to enhance the ecosystem that uses that web standard. But the point is: it's really demoralizing to get these kinds of comments, but they're the most common kinds of comments one gets on HN. Sometimes I did think about giving up on ActivityPub standardization. Ultimately I'm glad now that I didn't. So I think we can do cool and interesting things here with Spritely, and my lesson from the past is to not give up now either.
They're not coming at it from a place where they grok what you're trying to do and saying that you're doomed. The issue is that they've failed to satisfy the precondition of understanding things well enough to have an opinion about whether you're doomed. They think that whether "mainstream users would go for this" is somehow in scope.
But I think part of the problem is that it's very hard to explain to people in words how something is going to work. In general people better understand from experiences. That's why Spritely is actually taking a demo-centric approach... people understand better from experiences than from being told. I wrote about this more here recently: https://dustycloud.org/blog/if-you-cant-tell-people-anything...
Which is a response to Chip Morningstar's article, "You Can't Tell People Anything": http://habitatchronicles.com/2004/04/you-cant-tell-people-an...
... which has a really interesting quote in it about how baffled people were about the idea of hyperlinks:
> Years ago, before Lucasfilm, I worked for Project Xanadu (the original hypertext project, way before this newfangled World Wide Web thing). One of the things I did was travel around the country trying to evangelize the idea of hypertext. People loved it, but nobody got it. Nobody. We provided lots of explanation. We had pictures. We had scenarios, little stories that told what it would be like. People would ask astonishing questions, like “who’s going to pay to make all those links?” or “why would anyone want to put documents online?” Alas, many things really must be experienced to be understood. We didn’t have much of an experience to deliver to them though — after all, the whole point of all this evangelizing was to get people to give us money to pay for developing the software in the first place! But someone who’s spent even 10 minutes using the Web would never think to ask some of the questions we got asked.
I think that the level of incredulity that we're seeing here is understandable then, in that sense... human beings are better equipped to understand something in retrospect than to think ahead and try to join you in envisioning it. Oh well... as more demos come out (we've already done a few, a couple are shown in the video but not all), maybe people will believe more and more. When it gets in peoples' hands, even more so.
> I think that the level of incredulity that we're seeing here is understandable
Another issue, I think, is that in your reply the the previous commenter, you're not really replying to their comments so much as you are responding to the gestalt of the overall discussion here and about HN in general.
As Jobs said, it's not about personal computing, it's about interpersonal computing. You seem, to me, very focused on applications & products, but to me, there is a missing layer of tools & technologies that are available to begin to make applications & networks. While you "don't actually have any idea what it's for", I can picture dozens, hundreds of applications that could be built on the basis of Spritely. I'm not sure why you are so imagination starved? In contrast, you could put me on stage & I could enumerate different applications & systems that could use this tech for 15 minutes.
The realm of what is possible with better connected computing primitives, that are available off the shelf, seems near endless. And my hope is, it's friendly to users. That people appreciate the spirited presentation. I don't see that "[mainstream users are] going to scoff at this if they stumble on this," I see a means to give people far more powers & to upt them in control.
The more networked you want to get, the clearer it seems Spritely is a win. If there's only some networking- a todo application shared between devices- your advantage is significant. If you want to share that todo list between a bunch of family members, & delegate tasks, you go from using the data-storage system & some modest networking to relying much more heavily on other aspects like Goblins distributed programming/ocap, Brux contact management, Mandy activitypub for sharing.
Basically I'd just be standing on stage listing different apps. All apps need many of these components, for almost all apps are connected apps these days: the assumption is that the data will be available across systems, across devices. We have surprisingly little to bring to bear here; most apps end up doing an enormous amount of artisinal hand-craft work to build their own back-end infrastructure layers. The few couple platforms targeting general app-dev like AWS AppSync & Google Firebase have very limited scope & goals, less concern with our public, standardized multi-user network tech like ActivityPub & ocap, & only provide what they provide, are not open enough to themselves be enhanced & extended & experimented with, only atop.
But the work has actually been happening for a couple of years, and the foundational layer, Spritely Goblins, is already something you can use: https://docs.racket-lang.org/goblins/index.html
I think people don't tend to get very excited until they see the time travel demo, and then they say "whoa, something cool is going on here" https://dustycloud.org/blog/goblins-time-travel-micropreview...
But that's not really the most interesting part of Spritely Goblins... the safe to run in a mutually suspicious network distributed programming environment is.
Some background: I was one of the co-authors/co-editors of the ActivityPub social network standard... you might know it because it's the protocol used by Mastodon, but it's also used by Peertube, NextCloud, Write Freely, Funkwhale, Pixelfed, etc. I actually got involved in standardization because we needed it for the previous project I worked on, MediaGoblin. After standardization of ActivityPub ended, it turned out that a number of ActivityPub-using applications such as Peertube and Pixelfed and Funkwhale were using the protocol we worked on standardizing to do the kinds of things we meant to work on in MediaGoblin. So I asked myself, given the finite amount of time I have on this planet, what's the best use of my time? At this point I thought (and with a lot of feedback and phone calls with close friends): I should work on advancing the things that the federated social network can't do. And there are quite a few of those because ActivityPub left some gaps in the spec, especially around authentication and authorization, but also because people aren't building as rich of experiences as I think are interesting and possible yet... they're mostly mimicing contemporary social networks.
So you can see in that long list of things on the Spritely website that one of them is Fantasary, which is the goal of bringing distributed virtual worlds to the fediverse. That may sound like a hyper-ambitious and also kind of strange goal, but consider that modern social networks are really degenerate versions of massively multiplayer games; you can chat in both, but most social networks don't have the sense of space and rich interaction that games do. But I also want to make more obscure peer-to-peer technologies available and in the hands of users without the usual "Oh no those are only for the bad people on the internet"... normalizing things can help. Those of us old enough remember when demanding https was similarly brought up with the "well, that's only for people who have something to hide". A use case of e-commerce changed that. Well, I want another use case... but one that involves fun. Hence the distributed game stuff. We can see how motivating building worlds together is by the success of minecraft.
But what if the Fantasary component fails? It's the most ambitious piece! That's actually okay, because it's really a driver for all the other layers. Distributed programming that's safe to run in a hostile network, portable encrypted storage, the ability to safely run untrusted code... all of these are useful things, and useful things to bring to users generally, even if the game approach fails.
But actually all of these layers are quite well researched and possible. Part of the reasons that Spritely development happened before two years before the launch of the Spritely website is because when we ran MediaGoblin, we opened up the project and within a couple of weeks had dozens of developers... but that was quite feasible to accomodate because MediaGoblin was a fairly "traditional" web application, easy enough to point to how it works by other similar web applications. Spritely is treading a lot of what appears to be "new ground" to most people, so I had to figure out how it works enough, and lay enough of the foundation, before I was comfortable making a public image for it and asking people to start looking at it. But with the last release of Goblins with the beginnings of distributed network programming and with a fairly reasonable tutorial, that's starting to change.
But I said "new ground" to most people, but few of these are actually new ideas. Most of them are old, well researched ideas, from a somewhat obscure (but becoming quickly less so) branch of computer science called "object capability security". I've been very fortunate that the object capability security community has been willing to take the time to walk me through how many of these concepts work... Spritely wouldn't be possible without their help.
If you want to hear more, the video embedded on the site is a good overview, but here's a direct link: https://conf.tube/videos/watch/18aa2f92-36cc-4424-9a4f-6f2de...
Anyway, happy to answer questions if anyone has any. And feel free to join our irc channel... we're #spritely on irc.freenode.net!
I'm curious what the drive was to use scheme and E for development?
In fact, we're in talks with the wonderful folks at Agoric, who are building very similar things in Javascript-land: https://agoric.com/
And we want our two CapTP implementations to be compatible. If we do that, you can have programs written in Scheme/Racket talking to programs written in Javascript just fine.
Hope that makes sense!
I'd be keen to see what your thoughts are on transactional logs here- you're using snapshots so technically you could do a transaction log, which in turn can be used to help resolve distributed remote call issues eg. if you have 2 nodes and a split-brain scenario you can use the transaction log with a (vector) clock and defined types to resolve the split when connectivity is restored.
Additionally, do you have any thoughts on GC of objects? I'd have thought that Object Capability Security would factor into this.
Goblins already supports distributed (acyclic) GC! And yes ocap security does tie in nicely. In a sense ocaps make this really easy... if you think about an ACL approach the user has to point at the object they have access to and the object has to point back, but since ocaps are based on "having references to objects", in general GC locally can just be the very GC that the native runtime provides. Across runtimes (in the distributed code), distributed GC likewise falls out easier, though the algorithm doesn't know how to handle cycles when Alice on machine A points at Bob on machine B which points at Alice on machine A again. So you have to manually cycle break in those cases, but they're relatively rare.
Amazingly, Electric Communities Habitat had distributed cyclic garbage collection, which just blows my mind: http://erights.org/history/original-e/dgc/
But that requires special cooperation to traverse the object graph held by the runtime's garbage collector, which we don't have access to in Racket (or most languages) unfortunately. It would really be cool to see that tech revived somewhere though.
Hope that answered your question somewhat!
With the DGC, yes it makes sense that, given a local GC, you'd be better off extending it.
There's a paper https://concurrency.ch/Content/publications/Blaeser_Componen...
which details an RAII + message queue solution for managed memory without GC. The idea there is that provided your message API is well defined, you can determine the queue size in advance, and thus deterministically manage the memory (ie. object lifetime).
I'm mentioning it here, as if capabilities and RPC are added to the mix, you can treat remote objects as part of the memory DAG - RPC calls are just another message on the queue (capabilities here act as a root trace as if you have a message on the queue, you can trace back to the root) and the object can be removed when the queue is zero.
In addition, as per Rust, we should be able to lean on the compiler somewhat, to do some static cycle analysis. I don't think you can get away from having a cycle checker at high level, but with a transaction log + managed memory + functional code it should make a cycle checker easier to write.
Lastly, I was thinking about the DAAS space - all of this distributed stuff will require authentication (authN) and authorization (authZ).
I think primarily this comes down to IBS (Id Based Signatures) for authN, and then storing encryption secrets to access resources in a distributed hash tree (https://wmcode.nl/dh3/dh3_paper_v0-5.pdf), but any thoughts on how to secure items without POW (proof of work)?
Effectively the DHT is a POS (proof of stake) as you define who owns the resources by attaching an access hash to the secret.
It looks like this could be it.
And it's true that it doesn't give a definition; presumably at this earlier stage, the people most interested in the website are people who are already interested in some way in the federated social web. The ActivityPub spec is linked to though, so it's not hard to find out more info about it.