Hacker News Reader

Comment by smoody | Hacker News Reader

"Changes are made in absolute real time, something even Google hasn’t been able to do (Google docs update every fifteen seconds)."

To imply that Google 'hasn't been able to do it' is kind of silly. It's more likely that they chose not to do it and/or their scaling requirements made it impractical.

View on HN

It's not Google's scaling requirements that limit their simultaneity (or, obviously, a deliberate choice). Google Docs uses a different architecture, where a master copy of the document is kept on the server and broadcast every n seconds to all the clients. This is easier to implement, but can't be made fast enough to feel realtime.

Etherpad uses a different approach: they ship all the edits to all the users as soon as they happen, and each client processes them to produce an updated document. The hard part is making sure each user is seeing the same thing. This is easy with the Google Docs model where a single master copy is regularly rebroadcast. To do it the Etherpad way, without a master copy, is extremely hard. From what I've heard about what they had to do, it sounds like one of the more impressive hacking achievements by a YC-funded co to date.

Hopefully the mechanism they chose is amenable to cross-user undo. (Though it's currently missing, I'm assuming single-user undo can be added easily enough.)

Undo works, just type control-Z (on windows) or command-Z (on a mac). It will undo only your own edits.

We haven't figured out a way to let people undo other users' edits. (We can perform the manipulation on the document, we just haven't figured out a sensible UI for it).

I don't think there is a sensible UI for it. I'd leave it out completely.

Paxos-style distributed consensus?

A master model is not necessarily slower. Everything goes through the server either way, and it comes down to how much data you have to transmit and how much the sender, server, and receiver have to process it.

At http://draftastic.com/ we keep a master but only broadcast changed portions -- a compromise between re-sending the whole document and handling a hail of tiny diffs. This uses automatic locking, which we thought was reasonable given that people rarely want to type directly on top of each other, but it means a much simpler implementation of undo and so on. Basically, we've made a heartbeat-based system with heartbeats so cheap that they can be fast, and we never need a merge algorithm.

In any case, I'll be watching the just-raw-edits approach with interest. Best of luck to Etherpad -- there's plenty of room in the market.

Look at MMO's or FPS games to see how fast they can make that model.

Agreed. Etherpad is sending a new POST request pretty much every keypress. Considering the HTTP headers and cookies, that's around 500-1k of data per letter typed. At google docs volume that would be a lot.

(Total data sent when I press 'j' was 1073 bytes of HTTP headers, and 123 bytes of program data).

The other reason may be that whilst it looks really cool to see each other typing, is it really needed?

Does look quite slick though :)

I think they're using comet for the requests so while the activity may be high when people are typing, during breaks it should be very low as opposed to ajax polling.

I do think that real-time is really needed for many applications. The few times I've tried collaborating on a google doc, I've given up because the lag introduced the possibility of overwriting each others edits. That and just a general annoyance with saying "Do you see it yet?" over and over.

Well that's a stupid way to handle it... keep the socket open... read about Comet. http://en.wikipedia.org/wiki/Comet_(programming)

Are you being sarcastic? Does anyone know what comet is? Please explain exactly how you can SEND multiple packets over time from browser to server without sending more HTTP requests, and without using any java applet/flash.

It's a shame people seem to have started saying "use comet" without understanding what it means. Did you read that link?

Connection: Keep-Alive

Without any java applets or flash?

Yes, you can specify keep-alive (Although some browsers refuse to do it in an XHR - Opera for one). However, you still send the full glut of HTTP headers every single request.

True, you still need the headers, what you don't need is a new connection every key stroke. Establishing a new TCP connection is a relatively high cost network item (between all the Acks and Syns you need 3 messages have to go back and forth for it to establish), keep the connection alive, the stuff will run a lot faster.

You can null out the default headers on a XHR so they don't get sent.

You're forgetting Google didn't develop Google Docs in-house they purchased Writely back in 2006 and that's the codebase Google Docs is built on. Depending on how it was built originally the change might require a massive rewrite...

Was Google Spreadsheets built on top of the Writely codebase? Because their Spreadsheet app is WAY better than their Document app.

No that's based on a different acquisition, 2Web Technologies

What's the difference between "something even Google hasn’t been able to do" and "their scaling requirements made it impractical"? When you get past the semantics, it's essentially the same thing. Either way, they're not doing it.

Lack of action doesn't imply inability. That's what's different.

A parked car is not a car that's not able to drive. It's a car that's not driving.

The end user only cares whether its driving. If it's not moving, the user doesn't really care why.

Reply on news.ycombinator.com