Google defends dropping chat federation with inaccurate comments
plus.google.com
plus.google.com
Google said: "For example, mobile has several requirements around bandwidth and battery that are simply not part of the standard"
He says: "This glances over the the fact that the X in XMPP stands for eXtensible, which still results in proposals for new protocol extensions every month."
His misinformation is this: XMPP is XML based. The bandwidth that Google most likely refers to is a result of verbosity of XML. You can't eliminate that by extending XMPP, something that a guy who serves on XMPP Council should know.
The only way to fix that specific problem is to change the protocol. Which is what Google did.
This is an example a bad advocacy: badmouthing companies that decided to no longer use something you work on.
I also find troubling the pounding Google gets for every little mis-step.
When it comes to using and promoting open technologies Google is seemingly from another planet compared to other tech giants (Apple, Microsoft, Facebook, Amazon, you name it).
And yet the furor is not about Microsoft using completely secret protocol (Skype), not about Apple promising to publish their secret protocol (Facetime) and then not doing it, not about Apple preventing fair browser competition in iOS etc. but about Google no longer using XMPP.
A lack of perspective.
Ehhh... well... if you're willing to put some work into the matter that may not be as true as you'd think. Yes, to speak the federated protocol to another server you're going to be speaking the XML protocol, but between a server and a client that come to accommodations with each other, you don't really have to speak XML.
You'd want to speak something isomorphic to the XML, but the XMPP XML is not that verbose on a node-by-node basis or anything (minus a couple things, see below), it really is the XML, not the semantic content. It would be fairly easy for a medium-to-high skill developer to come up with an isomorphism for use with mobile clients. (It is sufficiently tricky that there would be a few traps that could catch the inexperienced or unwary (watch those namespaces!), but it isn't that difficult.) The end result is a more bandwidth efficient protocol that is a drop-in replacement at the XMPP library layer in the client for XMPP. (Of course, if your client is programmed to fling around XML all over the place internally this won't help, but, well, that's the price of failing to abstract properly.)
Where you'd still have trouble is things like roster size, but yes, there are XEPs for that, and before long you're looking at a reasonably svelte protocol (not perfectly, but reasonably svelte) that is still largely XMPP and easy to write communication with the outside world.
What's really difficult is semantic IM translation between entirely different systems, not protocol translation. ejabberd, the server whose internals I am most familiar with, essentially works by internally speaking Erlang terms with itself, and translating to the XML protocol only at the edges. It wouldn't be very hard to drop in different serialization at the edges and get a different protocol; as proof of concept of the fact this is possible, one could easily simply serialize those Erlang terms with Erlang's term serialization, and indeed, when operating ejabberd in a clustered mode, that's actually happening across the wire as the cluster nodes communicate with each other.
However much effort that may be, it seems easier than starting from scratch.
See: http://en.wikipedia.org/wiki/Binary_XML
Fast Infoset is a good contender: http://en.wikipedia.org/wiki/Fast_Infoset
Google is in a position to define the course of internet and has used that power in several ways in the past. This move from an open communication network to a closed network, to me, is as defining as WebGL and WebRTC are, but in a very unhealthy way. Google's move therefore is clearly 'evil' (as per their own definition of evil) as it is a clear choice they made to safeguard their vested interests, stating that other corporations are unduly benefiting from their openness as a reason.(http://www.youtube.com/watch?v=AfK8h73bb-o [2:40])
They stated that major networks did not interoperate with them. But in 2007 AIM started interoperating with Google Talk. http://gmailblog.blogspot.in/2007/12/gmail-chat-aim-crazy-de.... Doesn't that count? What did Google do to promote federation, other than asking people to contact them to federate? It appears that when Google was small, they tried to be open to benefit from other big players. Now that they are the biggies, they don't see any value in being open.
I believe more than any other company you mentioned, Google benefited significantly by projecting themselves as the messiahs of internet, purportedly promoting openness and standardization. They captured the imagination of people like us by promising to be only doers of good, placing the objectives of internet and humanity ahead of the corporation's benefits. They did so, as long as it worked in their favor. Those initial adopters who stood with them for these objectives are either absorbed into Google to work as if Google's interests are internet's interests or they are too few and left with fewer alternatives to significantly influence Google and its corporate objectives. With many of the influential personalities deeply affiliated with Google, we have fewer voices calling a spade as a spade.
Sad but true. Corporations who have to worry about keeping their share price forever in the green will have to compromise on their ideals. They will still talk about their ideals but they won't put their money where their mouth is.
[1] https://developers.google.com/talk/open_communications#devel...
I'm not offering an opinion for or against the decision to drop XMPP, but I wanted to offer some corrections to what is being said.
I think you've misunderstood how scale works. At scale, problems do not shrink, they grow.
I do not work on anything even closely related to Hangouts, but I used to run an ejabberd server myself, and I've worked with SIP a bit.
XML does have problems, both with bandwidth, (even compressed), high CPU utilization, parsing code is more complex (which leads to more security issues). XMPP also has issues, and I'm not just talking about how hard it is to get multiple Audio/Video clients to communicate[1]. (e.g. kernel resources, the incredible number of failure states, etc.)
Cheers, Doug
#1 At least early versions of iChat used XMPP with SIP. For those who don't know SIP, it stands for Session Initiation Protocol, which means that it doesn't transfer the audio/video, it's just a language for making connections. You have to negotiate a transport layer (which was often but not always RTP or SRTP). You have to exchange connection ports via SDP. If you're lucky and manage to get by all firewalls at this point, you now have to hope you're able to use compatible codecs. Interoperability was a nightmare.
The client-server protocol was simply JSON. XMPP was not involved. (Well, we translated protobuf messages to JSON, so it was ugly JSON, but still JSON).
we translated protobuf messages to JSON
aside, any opinions on others choosing that as a standard way of communicating between internal services on disparate runtimes? (golang<->python, for example)
--
I think they meant the scale of Google (total) compared to the scale of just messaging.
i.e: If you could spend a year's worth of working cutting the costs of <sevice X> by 30%, that would be fairly meaningless if it was .000001% of the total cost of operation.
I don't know what Google's operating expenses are[1], but I'll use the public income numbers as a proxy, and you can adjust from there.
If you look at Google's first quarter 2013 numbers, they pulled in 13.97 billion in revenue, and I'll subtract the 2.96 billion in traffic acquisition costs to come up with 11.01 billion. We'll multiply by four[2] to get a rough estimate of yearly income: 44.04 billion.
If we assume that messaging is millionth of a percent = one hundred millionth[3] of Google's expenses, then the total cost to google is $440 per year, so saving 30% just doesn't make sense.
If, however, it's one hundred thousandth of Google's expenses (still a ridiculously small amount), then you're talking about $440k of yearly expenses, and you now need to compare a year of salary to 30%, or $146k.
At this point, depending on the availability of engineers (whether or not you have to factor in hiring costs) it starts to make a lot more sense.
Cheers, Doug
#1 and I probably wouldn't be able to divulge them if I did. :-)
#2 Each quarter is different, but x4 is a good rough estimate.
#3 Your numbers.
Now, XML itself is one problem, but the "culture" (eg, usage norms) around XML is another -- much like Java's cultural preoccupation with abstraction and boilerplate, XML-based formats have a tendency to veer towards the unwieldy and verbose. XMPP is no exception.
While XMPP has been instrumental in propagating standardized presence and IM, its age is showing. Unless we develop a modern, open replacement that is natively suited to the needs of today's Internet (voice, video, file transfer, group messaging, etc+), we will see companies continue to develop proprietary solutions that are locked behind walled gardens.
+XEPs and extensions are insufficient; too many optional extensions hinders interop and contorts the system beyond its original design specifications. Just ask anyone who's tried to implement chat/buddylists on top of SIP presence.
Youtube alone makes up a significant amount of the Internet's total traffic. Sending compressed XML isn't a problem for Google.
Sure, the page has really become a one-page app with some advantages that come from pre-loading, but jeez, it is still 1MB!
Then why comment? If you have nothing to input from either side. Doubt is not a valuable commodity.
Some of us need our thoughts clarified by the community.
In any case, I find that learning happens in an atmosphere of uncertainty, humility and exploration. Doubt, thus, is a valuable commodity in any community which encourages learning, and I'd like to believe HN is one of them.
If you respect people, then their opinions tell you things about the search space in which they live. Heck, even if you don't respect people it at least tells you nsomething about how widely known a particular thing is within comparable sections of society.
There are, potentially, some good arguments for filtering your information to keep newbies out of higher level discussion where people are trying to make progress on a problem - but in the context here it seems to me at the moment that you're just being somewhat mean; I don't see what you hope to gain other than to make someone feel bad.
Maybe you should refrain from commenting until you have something "valuable" to offer, too.
Optimizing parsing cost, at the expense of breaking foundational philosophy of their communication services, is hardly justifiable. XMPP has several overheads, XML will not top that list.
I hate XML as much as the next guy, but for a phone that can do 20 fps OpenGL scene the cost of parsing XML that is most certainly valid and as used by XMPP is effectively nothing.
Don't be a dick.
> for a phone that can do 20 fps OpenGL...
It's not just the phone, there is server overhead too, which everyone seems to be ignoring. While XML has a lot of problems, I never said it was the problem with XMPP (although it certainly isn't helping).
> Don't be a dick.
Oh, puh-lease.
> there is server overhead too
It's all going over the SSL, remember?
I have a life-long passionate hate towards XML, but you have to look at a larger picture to understand why Google is killing XMPP support. It's not the performance, it's not the lack of the protocol elegance. It's the fact that they are building a wall. Facebook got their herd walled and it seems to be working well for them, so it's only natural Google is doing the same. The XMPP was in a way, so it's gotta go.
But NIH/control is an unlikely reason for brewing their own protocol. If they simply wanted a wall, they could just disable federation. They've done it before, and it's working for WhatsApp, it's working for Facebook, it's working for Microsoft [1].
G already has a lot of experience with XMPP and it's probably just the engineering reality between what they want to accomplish with Hangouts and what it would take to retrofit XMPP or an existing protocol. Throwing out the baby with the bathwater? Maybe. But it wouldn't be the first time, and won't be the last.
[1] Remember that FBChat, Lync etc are internally proprietary, but they do expose XMPP gateways. Intentionally preventing federation is shitty, but dropping XMPP at the core is not.
He directly addresses the Bandwidth concern by noting that interoperability on the server side is irrelevant to mobile anyways:
"Of course none of these concerns for mobile are applicable for server federation. As far as I know, the Google Talk client on Android doesn't even use XMPP as the client-to-server protocol."
1) Compression does help keep the transmissions under the FACH threshold on most networks. This more than outweighs the CPU cost.
2) The bulk of the data to be compressed isn't XML, but text.
3) The chattiness of the protocol - ie, transmissions that are not needed at that point in time - will indeed change radio states, but not only did Google tackle this problem privately, but the XSF has also looked at it in detail.
4) Really, the CPU is a non-issue here. It's all about the radio power states. This is likely to be entirely different at the server end, however efficient XML parsers can and do exist to mitigate this, and make it more distributable.
Creating new and better protocols is OK. Making them closed and non interoperable - is not. XMPP is not ideal, but it's open for participation in its development and for its usage. Whatever Google decided to create is not open, and that's their main failure.
Google, on the other hand, got a free pass for a while because they had a lot of marketing to the effect of "Look at us! We're the openest openers who ever opened openness!" and... well, they have a history of spectacularly failing to follow through on that with, say, Android ("oh, it's totally open -- except for this release that only our partners get! And except for all the bits that most people don't realize are separate and that we'll C&D you for distributing! But it's OPEN, we promise!").
So people are starting to no longer give Google the automatic pass on this. Which should be expected.
(Not that it absolves Apple from failing to do it)
As far as I understand/interpret their policy: they only prohibit so-called copyleft licenses, and/or ones which would be violated inherently by the app store's binary-only distribution model.
The furore is partly also about Google moving away from an open protocol to a supposedly secret protocol. If Google were to publish the new protocol that Hangouts uses all/most of the animosity would vanish. And people would come up with a xmpp to hangouts bridge too.
Google has supported and even championed open source initiatives and it has served them well. And imo it will serve them well in the long term if they continue being open, instead of trying to monopolize a segment of the market.
Basically they co-opted the language of open source to rube nerds, while nobody is under any illusions re: Microsoft & Apple.
Chromium alone puts them in the plus column standard-wise. Usually apps are judged on merit, yet in Google's case there is the misconception that they have to defend every engineering decision they make.
I don't see how this is a hot topic seeing as every other IM is an island, and quite frankly if it means directing more resources towards projects like Google Fiber, personally I think it's more than worth it to sunset GReader or move away from a limiting standard.
Chromium will likely be less standards based now that they don't have the tow of the other WebKit companies keeping them in check. Case in point: SPDY.
Google is not the only company who has to defend every engineering decision. The reason it is being discussed is that it is fitting into this new narrative of Google being less interested in standards and more interested in their advertising centric walled garden.
And finally Google can do multiple things at once. It isn't an either-or scenario.
Google develops a proprietary protocol without ANY consultation. They then leverage the fact that they own the world's biggest website and popular browser to force it by stealth into popularity. And then afterwards decide to work to standardising it despite it not necessarily being the optimum choice for everyone.
The core protocols of the internet are fundamental. And they should be designed deliberately with as many parties as possible involved. This is not the way standards should come into existence.
(DISCLAIMER: I work for Google. All of my opinions are my own and not Google's.)
They solved a problem and made something better, case in point Facebook and Twitter among others have already deployed it.
If you do some research you'll find that most standards started as projects that came out of a single company or lab that were later adopted by industry.
The only reason companies are adopting SPDY is because it is an improvement than HTTP. SPDY is far from perfect. The issue is that with core internet protocols this is not the way it should happen.
I don't see the problem.
>The issue is that with core internet protocols this is not the way it should happen.
This is exactly how it does happen. Someone creates something, people like it, the creators don't enforce patents on it or try to keep it a secret, someone writes it up into an RFC. That's how internet standards are created. That's how they've been created as long as there has been an internet.
Look at a counterexample. Microsoft with OOXML. Here we have a so-called standard which says things like "do this the way X version of Microsoft Word does it" without actually specifying how to do it. It's not an open standard, it's a document claiming to be an open standard on paper so that Microsoft can claim their proprietary file format is an open standard. No one else is expected to use it for any purpose other than attempting interoperability with Microsoft Office and Microsoft has no particular interest in having anyone even succeed in doing that.
SPDY is not that. Google has published a reference implementation with source code. People other than Google use it for reasons having nothing to do with interoperability and Google services continue to be available to browsers without SPDY. "They didn't ask permission first" is just disingenuous griping -- who are they even supposed to have asked?
SPDY isn't a problem. Hangouts are. I don't see source code for a reference implementation. I don't see anyone but Google implementing the protocol. That's how you know something is wrong.
They use it because they don't have a choice. If you don't help and/or follow Google you don't get to play, right now, because they own a large majority of most technologies being in use by the public today.
That's the same thing people criticized Microsoft for.
Hangout is just much more wrong tho, as you pointed out.
They use it because they don't have a choice. If you don't help and/or follow Google you don't get to play, right now, because they own a large majority of most technologies being in use by the public today.
That's the same thing people criticized Microsoft for.
The way SPDY works is just like OAuth/OAuth2, HTTP, XMPP work: someone invent/improve a protocol, working on draft and get it open standard. I don't see any problem with that.
Here is the "Blink manifesto" if you will: A proposal for un-prefixing https://groups.google.com/a/chromium.org/forum/m/?fromgroups...
As for SPDY it's not proprietary and it's being baked into HTTP 2.0: http://tools.ietf.org/html/draft-ietf-httpbis-http2-00
And I'm not sure what you mean by "advertising walled garden" other that just throwing around some senseless hostile buzzwords.
Maybe I could get over the fact that it's so complex -- which puts an unnecessary strain on both servers and clients -- and maybe I could get over the fact that (on average) servers consume 40% more memory than their IRC counterparts. Maybe I could even get over the fact that the spec could change at any moment (the eXtensible nature of it isn't necessarily a good thing, you know).
But what I couldn't possibly in a million years get over is that... it's XML (shudder).
-- To the downvoters:
Here's some previous discussion about why XMPP sucks: https://news.ycombinator.com/item?id=2069810
I'm currently working on a project that deals with real time communication (think IRC for the new web) and after fidgeting around with XMPP for a couple of weeks, I gave up on it completely. It's awful. I doubt many people have implemented an XMPP client (or even worse.. a server). Try it some time.
I ended up making my own -- I based it on IRC but wrapped it in JSON. It's stupid simple because chat should be easy.
It's very good for its intended purpose, but it's not good for IM usage.
http://arstechnica.com/information-technology/2013/01/antarc...
best i:line ever.
I said IRC semi in jest before because it's very old and there really should be something lightweight that ought to take IRC's place in this web 2.0 world of ours.
It seems like the impact of the other big players not playing is down played here. Did it not allow facebook users to start conversations with google talk users but not vice versa?
Playing the cooperation card first is what google did, but in the iterated prisoners dilemma you often need to play the non-cooperative card if that is what your peers are doing.
That really seemed to be the core of the issue. Now 5-25 years from now maybe google will have an entrenched culture of playing the non-cooperative card and will not reach for the cooperative card first but I do not judge this to be the case yet.
Now those outside contacts are cut off, without any warning. Worse, they see their inside contacts online, can send them messages that never show up in Hangouts!
Those big players are hardly affected, though. Also, Google could simply block those entities that don't fully federate (only).
Do you have any evidence that federation helped Google grow the network in a significant way?
allowed them to do Google Talk for other domains
Are you referring to Google Talk integration with Google Apps For Your Domain? Outside evidence suggests that federation is not used between these domains. For example XMPP SRV records are not required and ignored if present.
That's exactly what the EFF has suggested, in fact: if they don't want to use XMPP, publish the new format for others to be able to use.
(I'm not a GOOG shareholder, so their business needs don't matter to me. I _am_ now going to be cut off from my contacts, though. Part of markets is the feedback that comes with mis-steps, and saying "it's just business" does a dis-service to those of us who feel that Google has been over-stepping their bounds.)
The [EEE] strategy's three phases are:
Embrace: Development of software substantially compatible with a competing product,
or implementing a public standard.
Extend: Addition and promotion of features not supported by the competing product
or part of the standard, creating interoperability problems for customers
who try to use the 'simple' standard.
Extinguish: When extensions become a de facto standard because of their dominant
market share, they marginalize competitors that do not or cannot support
the new extensions.
XMPP was essentially a non-player before Google used it, and it'll return to being a non-player after they leave. There's no big strategy needed to accomplish this.
This is not to say I support Google's recent decisions, but then again, I had already moved off their applications (Reader and Gmail) way before the shutdowns started, mostly due to the methods used to push Chrome.
Google's extensions to XMPP took place via the XEP process, and were published so other vendors could implement them.
It does not matter if the said modifications are called "G-XMPP" or 'Hangouts'. The result and logic are the same.
The rest is a useless fight on semantics.
SSH is not an extended version of Telnet.
Thats what Hangout does. Name 5 of your friends without a google account who use internet & mobile phones. See what I mean? They were using XMPP. Everyone could talk to them using XMPP. Not anymore. Wanna talk to them? Gotta use hangout, no way around it.
It does not matter if its based on the same _code_ what matter is that it offers the same functionality AND kills off the previous product by using their market share.
That's the freaking concept.
As for XMPP, it was their business decision. I'm betting engineering had a good reason for it, it just may not be shared with us.
XMPP does not require any open ports on a client.
source: I'm an ex-cofounder of a now very successful syncing solution startup who uses XMPP and I wrote the initial server and client code.
anecdotally:
% lsof -i4 -T s
COMMAND PID USER FD TYPE DEVICE
imagent 158 mh 6u IPv4 0x2969ca78b2e9b6bb 0t0 TCP mac:52812->pb-in-f125.1e100.net:5223 (ESTABLISHED)
XMPP is a connection-oriented stateful protocol. The connection setup process is very expensive: Connect, authenticate, advertise the list of supported client features, get the roster (contacts), and IIRC presence exchange, etc. Therefore, once you've established a connection, you really don't want it to break. Even just reconnecting is very expensive.
However, in the mobile scenario, regular connection breakage is the norm. What most mobile based clients will then do is proxy the XMPP connection on the server-side and connect to the client over HTTP instead. This is well defined in the BoSH spec (XEP:124). BoSH was originally designed for web-based clients, but happens to meet the need of the mobile use-case.
However, when it comes down to implementation, there are several optimizations that can be applied:
1. The roster exchange at the connection time is very expensive, and usually only sends data to the client that the client probably has already stored locally from a pervious session. Connection overhead can be reduced drastically if there was some way of versioning the roster and only transferring deltas. This was the OP was referring to. (Roster versioning, XEP:273).
2. Presence packets are one of the most chatty aspects of the protocol (x went offline, x is now online, etc). I've heard someone say that it constitutes about 80% of the packets on a typical connection, but this is anecdotal. In cases where the user isn't actively interacting with the client, having presence packets being sent all the time seems wasteful. One way to tackle this is to buffer up and de-duplicate the presence packets on the server, and send them in batches only when necessary. (For example, if x goes online, then offline, then online again, and then goes 'busy', the client only needs to know that x is busy.) I think this is what the OP was referring to when he mentioned the google:queue for delayed presence delivery. This alone can be a huge win for battery and bandwidth, since transferring data and processing it can be reduced drastically.
Aside: Presence is so chatty, that some mobile clients simply don't handle presence at all. Take WhatsApp for example. It doesn't show whether every person in your list of contacts is online. The assumption is that every user is online by the nature of the device, and this assumption works in the common cases. In the edge cases is where you'd need to buffer up messages. This reduces chattiness drastically, making the protocol seem much more lightweight. (WhatsApp is based on XMPP, but have spun it off into their own thing.)
Just thought I'd add these points to the discussion here, since there seemed to be some lack of understanding of the protocol. IMHO, the XML, as verbose as it is, isn't really the problem. The protocol itself is verbose and whether the transport uses XML or whatever else is of comparatively little consequence. This verbosity of the protocol (and not so much the format of data transfer) has its effect on battery life and bandwidth usage.
One could also take Hangouts for example. :)
anyways, very informative post and good anecdotes, thank you for sharing.
my understanding of the XMPP spec is that (assuming the server follows the server-client spec, anyways) there's no way to unsubscribe from your contacts' presence notifications and keep them on your roster [and therefore able to be contacted without reauthorization, usually.]
can you confirm/enlighten me? thanks!
In fact, this caused presence subscription requests to be the only attack vector in terms of spam in Google talk, which some argue makes it harder to combat. Mostly because there is less data available to detect malicious behaviour.
2. Universities, organisations? The number is too small to be bothered about, esp when compared with the likes of the two above^. They would rather innovate and try to lock-in users into their own platforms rather than hoping someday Cuck might turn the benevolent saint.
3. Whatever you say but unless you are short of manpower and talent it's actually easier difficult to innovate in-house sth proprietary and not when you've to conform to a "standard" and that too when the rewards are [1].
4. Even I don't like the move but it was a business decision and this one made a lot more sense than killing off GR.
The surprising fact here is, we hardly see anyone criticizing the providers that chose not to share back. We, keep on saying "no problems with XMPP" and "even though MS and Fb were not giving back, Google should have just let them free-load after all it's a do no eveil corp". Huh.
Seems you have not been paying attention recently. Certainly not in the contxt of microsoft+facebook+gtalk+xmpp+google+talk one way, just google it; but that's my assumption. This is what I meant by "took...3 times".
If it was sth else you wanted to know or were referring to then I obviously failed to grok that and you'll have to help me understand.
Why is Google the company important to the XMPP community? I scarcely see that it should matter.
Open protocols are not exclusive clubs or infinite universes, anyone can opt in or out at any time, the rest of us will live.
I think its great when big companies such as Google or whoever participate in open standards, but I also appreciate that they are in the business of providing the best product for their users (in their eyes) and are free to do what they want.
When they do participate, we are getting a gift from them. Its not their obligation, right or even necessarily to their benefit. When they find other priorities fine, maybe someone more committed will move the project forward. And in other cases their NON-participation can be a gift as well.
So whats the big deal?