Software design mistakes that Diaspora needs to fix.
buddycloud.com
buddycloud.com
I agree with this point, I've made it here and I don't think I'm the only one.
But everyone knows the Diaspora folks went the "product first, protocol later" route and a fair number of people agree this approach. So it's a disingenuous little to act "oh, they made a basic mistake so it just won't work." They went a route, at least, that lots of people think will work. It's not like they used hard coded eight-bit node identifiers, had each node multi-cast its information or some other design decision would guarantee things couldn't work.
Naively, on first glance, to me Facebook looked like just a nice interface on top of standard Internet functionalities. And there are standard protocols for all of these.
* Messengering we have * Micro-blogging we have. * Selectively sharing information is pretty simple. Give your "friends" a password to your website and let them connect via SSL or Oauth or something. ...
Nothing seemed that new. My first thought was, "why couldn't you just slam a UI/Application on top of this existing stuff and call it an open Facebook implementation"? Ah, but there is a funny thing I realized looking at the situation (and discussing it here). The unique selling point of Facebook and its open competitors boils down a "share and revoke" function. The umpteen Facebook users want to imagine they are "safe" when "privately" sharing their photos. They don't look too carefully into what "safety" or "privacy" involves and they probably don't want to look that carefully but they really want to have it. Facebook has flattered them into thinking they can just carry their real world intuitions into virtual space. Of course, this isn't quite "real world privacy" as a number of fired employees can atest.
Thus the "competing Facebook protocols" aim to provide something like "personal DRM". Without going into the details of the inherent problems, I'll just ask, "why would you want to?".
I would argue that rather than trying to simulate something that's impossible, that Facebook doesn't and can't give you, why not educate people. Allow secure sharing of information but start people learning about what makes sense to share or not. And sure, that's a big job but the Internet itself is kind of doing that.
Looking at the situation as a whole, Facebook is leveraging people's lazy, foggy and "intuitive" notions of privacy taken from the real world. Anything Facebook-like is going to "leak-like-a-sieve" and the more people learn about the Internet, the more they will realize this.
The questions raised in that post are good, and we intend to address them as they become relevant to what we have build. Discovery, namespacing and routing are areas where we are starting to feel tension on in the product, and will probably be tackled quite soon. The point about affecting interface design is good, but we look at it the other way. We want our interface design to drive the protocol design, so we want to make protocol choices as late as is reasonable.
Finally, it's fantastic that people are reading and forking the code. The more people out there trying things, the better.
What I was trying to get at in the article was that app designers need to think about the protocol at some point since questions such as "are we relying on long polling or will the user refresh pages" start to play a question on the UI side too.
Also, if a product is marketed as being distributed, some thought as to how this will work and scale become important.
It's really good that Diaspora are thinking about what the user sees and programming from the user's POV. I spend a lot of time in the XMPP community and there is not enough of this focus. But a good programmer has to balance design with functionality and focus on many areas and some more thought on the protocol side would not go amiss.
I don't think that the UI polling/refresh question affects the inter-server protocol. Your seed is not your browser, and it's seed-seed communication that's the part of the protocol that matters in the long run.
Scaling is the pressure we're feeling most right now, and we're talking about how to deal with the issue of a distributed cluster with a coherent namespace. There are some interesting things about the Dynamo pattern that might be really useful to make this work. But without a running system with load and users and remote pods to connect to, it's very hard to figure out what you want.
I think that were at the point where the protocol needs something of a refactor, and I will probably make a post about it on the Diaspora-dev google group soon. Maybe we'll get some good feedback and design help!
There's an old saying that you need to build three apps on your framework. That first app shows that your framework works for something. The second app shows that its more general than just the first app. The third app shows that you weren't lucky.
Curious, how was Facebook designed? Are there any post-mortem docs on it? I know that MySpace seemed really ad hoc. I wouldn't be surprised if Facebook was more ad hoc than we imagine it to have been
If I had to guess I'd say that both MySpace and Facebook were completely ad hoc, but the difference was probably that Facebook kept their technical debt down so that they never accumulated an area of code that was too hairy to refactor or replace. MySpace on the other hand probably got complacent in their success and didn't worry too much as their architecture calcified into an unmaintainable mess. By the time they saw Facebook nipping at their heels their code base was past the point of no return, and the company culture made it impossible to hire the level of technical talent they would have needed to right the ship.
Of course I'm completely talking out my ass here, but that's the impression given by the respective product histories.
Maybe just me. I don't know much about the history of the project, except Zuck started it in college.
As the OP points out, protocol cannot be truly decoupled from architecture, especially when it comes to things like routing. There's a huge difference between a Twitter-like @user, an SMTP-like user@domain, a URL-based system like OpenID, etc.
For instance, building around events ("posting a status update", "sending a friend request", "verifying an identity") and abstracting out those events into hooks is how Appleseed does it.
I have no misconceptions that this will hold up to every possible social protocol, past, present and future, but thinking about the protocol as an abstraction, and building the platform out from there, has already shown itself to be very useful.
I think once that is done, people won't be so fast to suggest to them to hire a $100k/year engineer from Google or other things. Just my $0.02...
[edit: spelling]
Yes, it would be great to have a well-thought-out protocol and secure software. But then again there's also huge value in having an implementation that people can play with. We hear all the time about how important it is to get your best guess at an minimum viable product into users' hands, and that's pretty much what Diaspora has done.
The article made the point about federation algo, and I think that's the most important: proper federation/decentralization IS the product. As of yet it is an unsolved problem in their camp (and many others). This isn't to take anything away from them, because its a Big Problem, but if they can't solve that, a slightly polished CRUD social networking app is just fluff.
The product is the protocol, not the app. That's the only potentially innovative thing they are doing. Sorry guys, aspects are a product made for privacy concerned software devs - normies don't care.
> Sorry guys, aspects are a product made for privacy concerned software devs - normies don't care.
I'd argue that normals care about this too. Teens might want to keep their friend/school/personal stuff private from their family/friends-of-family/adults-they-know stuff. And adults often want to keep their "adult adult" stuff hidden or distinct from their family and/or coworkers. The whole "my mom wants to friend me on Facebook, ew!" or "my boss wants to friend me on Facebook, noooo!" factor.
Agreed 100%, but this feature alone isn't worth leaving facebook due to network effect. That may change, but if you have to boil it down to a black and white, it still leans far to the side of "not worth it to switch." Also, should this change, I find it highly unlikely facebook would find itself unable to pivot. Regardless of what you think of fb (I personally hate it) you really can't deny they have a world class engineering team that has a finger on the social pulse more than any other company right now.
I have tried building apps around these kinds of features, they don't really work (you end up building a new interface to email awfully quickly).
The killer feature that moves people over to distributed social networking is not distributed social networking itself. But third party developers building onto a distributed social networking framework is what will allow the next killer feature to be built.
For example, not many people used Tim Berner-Lee's original web browser but it did help define the standards that then went on to be picked up by Mosaic which was the first really popular web browser. However, by the point that Mosaic came out most of the features of the architecture of the web were in place and remain recognizable to this day.
Surely the goal is to provide common lower levels of a "social" protocol stack and allow as many and different user interface applications to layer on that common core?
That's one of the goals, but it's not the only goal. I think what you're bringing up is what makes social networking somewhat new terrain for open source. So far, open source has been able to dominate the server space because of superior tech implementations for an insanely low price point.
For social applications, that doesn't matter as much when you're trying to draw users. You need a usable UI, you need to learn all the lessons that the proprietary systems have learned over the years. It's not good enough to just have a better technology, you have to present it as well or better than the competition.
I think it's a new trajectory, because there's less incentive for walled gardens to open up than there was for Netscape to use http. So I think this is a necessary trail for open source to blaze through.
So in a lot of ways, there's very much a chicken-or-the-egg dilemma: Usable protocol stack that others can build on, or sizable enough user base that developers want to build on it. And the closest I've seen to open source dealing with these questions was actually, the gnu/linux desktop projects.
It would be interesting if there was an equivalent for social applications... a common server component, an application library and maybe a reference application but with plenty of scope for people to innovate in the UI without the need to continually reinvent the core communication components if they don't want to.
I don't think anyone should start by building a framework without any conception of what that framework is meant to do. But Diaspora already has an app of sorts, so they wouldn't exactly be doing that.
:)
I'm just teasing, no offense intended. I actually do agree with you and upvoted you - your comment just really reminded me of this quote from Steve Yegge (supposedly found on Reddit): "I upvoted you for the appropriate uses of 'its' and 'their' in the link title, but downvoted you because your link actually appeared on a little-known German social networking site several hours ago. I feel it is important that you understand that this is not a zero-vote of abstention, but rather a single upvote and a single downvote cancelling one another out."
Source: http://steve-yegge.blogspot.com/2009_03_01_archive.html
* Protocol
* Interface
* Framework
* Application
(with lots of intermediate layers)
A concretely realized framework isn't all that much more abstract than an application. Creating a framework doesn't mean you've created a well-defined interface, much less a well-defined protocol.
Moreover, a protocol is pretty much the only thing that has meaning over-a-wire. After all, you don't know what object you're talking to on the other side of a network connection so either you each have a well-defined protocol you're using or you don't.
I am not saying they are doing a bad job. Just that it's a very hard job to do. Most rock stars of today that built the next big thing had the luxury of not having to show anyone the code. And I'd say 90% of the time, that code is fucked, at least in parts. That's pretty recoverable - your product isn't the code it's the service or executable. Many opensource projects are similar - they can be hacky at first because no one even knows they exist - as they get traction they can fix things as they become issues.
But now with the whole internet breathing down their necks, celebrity status and a bunch of money burning a hole in their pockets they have no such luxuries at all.
I am inclined to agree though that they should really consider closing back up and rebooting both the design philosophy and the code base after folding a few key contributors into the core team.
Not that I'm right or that I know what I'm talking about, but just my opinion.
The other is that I'm not sold on XMPP as the best protocol for distributed social networking. I think it can be overly verbose, and a simple protocol over HTTP/HTTPS makes more sense to me, and I've had good luck with that so far. That said, I'm not completely sold against XMPP, either.
Protocol and the architecture. Protocol-wise I have no problem with http - it's well understood and any RoR dev can pick it up an use it. And indeed you could even, with a lot of work, build a federated network off http(s).
But http is not a a federation architecture. XMPP provides this out the box and makes bootstrapping a secure federated social network much easier.
https://github.com/appleseedproj/appleseed/blob/master/_docu...
Although the software is protocol-agnostic, and hooks could be written to support XMPP, as well.
I think my biggest issue with XMPP (beyond it's verboseness) is that it requires a whole separate XMPP server. This is less of an issue for Diaspora, which already requires a series of services running, but for my project, Appleseed, the goal is to get everything running under a LAMP stack which can be run on any shared host.
If I setup a new appleseed node and how do you really know it is me/example.com? I've yet to see that done well in http and would love to be proven wrong.
Which, if I'm not mistaken, is very similar to how XMPP does it.
The protocol also tries and minimizes the amount of actions that a third party node can take. You only trust your own node to take actions on your behalf.
This is critical. Diaspora could spend a lot of time in the architecture tower with nothing to show, and the early hype will bite them if they have nothing to show. I think they've done a good job making sure that doesn't happen.
Now that they've shown they can produce something, they need to slow down and make sure they are producing the right thing. When you are building a simple application, you can get away with design-by-coding; if you are trying to build a federated, global, social wonder, you are going to need to spend some time thinking about it.
http://www.reddit.com/r/programming/comments/ehhbz/i_made_a_...
The main issues with the diaspora project, besides buggy code, was that there was never a good reason to build it. People were not angry enough at Facebook to start learning how to run their own servers (which costs money) or pay diaspora to do it (which costs money)
(Before you retort, yes, I am capable of spending the hour or so build the stack, but it's nice to have plug and play sometimes).
FTFY
Since it is open source, Diaspora could probably use the writers of these critique pieces (bad design, security holes, etc.) to actually make a better product.
(Tangentially, why are there so many blog posts about Diaspora's code? Is the code actually unforkable? Or is it just because Diaspora painted itself into a corner promising a secure social network on a relatively quick timeframe and now everyone wants to prove them wrong?)
The writer of this critique, as well as myself, are both working on what we feel are better products, respectively.
Take that as you will.