Hacker News Reader

Comment by magic_haze | Hacker News Reader

Really? There are plenty of valid reasons why XMPP is not suitable for mobile usage (most notably, the proprietary push notification systems that all platforms insist you have to use), but XML isn't one of them. If you use TLS with XMPP (as gtalk does [1]), you get the compression protocol for free, but even if you can't, there's XEP-0138 [2], which specifies the use of zlib and has been in development for over 12 years now, and finalized since 2009. XML is fugly, but it works. At the scale Google operates, it is laughable that the use of XML in chat protocols could possibly impact their bandwidth accounting.

[1] https://developers.google.com/talk/open_communications#devel...

[2] http://xmpp.org/extensions/xep-0138.html

View on HN

Disclaimer: I am a Google employee, but I do not speak for Google.

I'm not offering an opinion for or against the decision to drop XMPP, but I wanted to offer some corrections to what is being said.

I think you've misunderstood how scale works. At scale, problems do not shrink, they grow.

I do not work on anything even closely related to Hangouts, but I used to run an ejabberd server myself, and I've worked with SIP a bit.

XML does have problems, both with bandwidth, (even compressed), high CPU utilization, parsing code is more complex (which leads to more security issues). XMPP also has issues, and I'm not just talking about how hard it is to get multiple Audio/Video clients to communicate[1]. (e.g. kernel resources, the incredible number of failure states, etc.)

Cheers, Doug

#1 At least early versions of iChat used XMPP with SIP. For those who don't know SIP, it stands for Session Initiation Protocol, which means that it doesn't transfer the audio/video, it's just a language for making connections. You have to negotiate a transport layer (which was often but not always RTP or SRTP). You have to exchange connection ports via SDP. If you're lucky and manage to get by all firewalls at this point, you now have to hope you're able to use compatible codecs. Interoperability was a nightmare.

And yet google wave ran on xmpp which was arguably a far far more sophisticated app doing way more communication than just chat

Google wave only used XMPP for federation. It was a nightmare to deploy and we never got it working reliably. Different jabber servers all had different quirks or implemented subsets of the XMPP extension spec.

The client-server protocol was simply JSON. XMPP was not involved. (Well, we translated protobuf messages to JSON, so it was ugly JSON, but still JSON).

(if you can't comment on this, no worries. thanks)

  we translated protobuf messages to JSON

just curious- is this the same 'protojson' I see exchanged in the current crop of APIs?

aside, any opinions on others choosing that as a standard way of communicating between internal services on disparate runtimes? (golang<->python, for example)

--

edit: offtopic, thanks for ShareJS.

>I think you've misunderstood how scale works. At scale, problems do not shrink, they grow.

I think they meant the scale of Google (total) compared to the scale of just messaging.

i.e: If you could spend a year's worth of working cutting the costs of <sevice X> by 30%, that would be fairly meaningless if it was .000001% of the total cost of operation.

Fair enough. But let's look at the scale of Google compared to messaging using your numbers to prove my point.

I don't know what Google's operating expenses are[1], but I'll use the public income numbers as a proxy, and you can adjust from there.

If you look at Google's first quarter 2013 numbers, they pulled in 13.97 billion in revenue, and I'll subtract the 2.96 billion in traffic acquisition costs to come up with 11.01 billion. We'll multiply by four[2] to get a rough estimate of yearly income: 44.04 billion.

If we assume that messaging is millionth of a percent = one hundred millionth[3] of Google's expenses, then the total cost to google is $440 per year, so saving 30% just doesn't make sense.

If, however, it's one hundred thousandth of Google's expenses (still a ridiculously small amount), then you're talking about $440k of yearly expenses, and you now need to compare a year of salary to 30%, or $146k.

At this point, depending on the availability of engineers (whether or not you have to factor in hiring costs) it starts to make a lot more sense.

Cheers, Doug

#1 and I probably wouldn't be able to divulge them if I did. :-)

#2 Each quarter is different, but x4 is a good rough estimate.

#3 Your numbers.

It's not just bandwidth, but the cost to process that XML, which is very expensive compared to other serialization formats. Then there are a whole host of other concerns that result from XML being text-based (cdata, preventing runaway parsing, etc). Not defending what Google has done, but the reality is XMPP is simply the most common among a lineup of really crappy chat/presence protocols.

Now, XML itself is one problem, but the "culture" (eg, usage norms) around XML is another -- much like Java's cultural preoccupation with abstraction and boilerplate, XML-based formats have a tendency to veer towards the unwieldy and verbose. XMPP is no exception.

While XMPP has been instrumental in propagating standardized presence and IM, its age is showing. Unless we develop a modern, open replacement that is natively suited to the needs of today's Internet (voice, video, file transfer, group messaging, etc+), we will see companies continue to develop proprietary solutions that are locked behind walled gardens.

+XEPs and extensions are insufficient; too many optional extensions hinders interop and contorts the system beyond its original design specifications. Just ask anyone who's tried to implement chat/buddylists on top of SIP presence.

Sorry but I just don't see how bandwidth and XML could ever be a problem for Google.

Youtube alone makes up a significant amount of the Internet's total traffic. Sending compressed XML isn't a problem for Google.

Just because Google has a lot of capacity doesn't mean they're wasteful with it. At scale, minor inefficiencies add up to a lot. That is the incentive for protobufs, etc. For a more concrete example of seemingly trivial bandwidth savings, check out the HTML source to their 404 page (it validates!): http://www.google.com/asdf.html

That 404 page is a relic from the past. Take a look at what makes up their search page and you'll see that it is almost 1 MB uncompressed. That's for a page that displays a logo, a navbar, a text input and a button.

Sure, the page has really become a one-page app with some advantages that come from pre-loading, but jeez, it is still 1MB!

> Sorry but I just don't see how bandwidth and XML could ever be a problem for Google.

Then why comment? If you have nothing to input from either side. Doubt is not a valuable commodity.

We not here for the sole reason of providing value to you. Although we may, incidentally, hope to do so.

Some of us need our thoughts clarified by the community.

In any case, I find that learning happens in an atmosphere of uncertainty, humility and exploration. Doubt, thus, is a valuable commodity in any community which encourages learning, and I'd like to believe HN is one of them.

Neither is being a snarky dickhead.

Maybe you should refrain from commenting until you have something "valuable" to offer, too.

Yes it is. For instance, if someone spends years trying to solve a problem and they can't do it and they tell you that they doubt that you're going to be able to solve it in a year and probably shouldn't base your strategies around doing so then that information is worth something - whatever the relevant entire costs are for your having done otherwise.

If you respect people, then their opinions tell you things about the search space in which they live. Heck, even if you don't respect people it at least tells you nsomething about how widely known a particular thing is within comparable sections of society.

There are, potentially, some good arguments for filtering your information to keep newbies out of higher level discussion where people are trying to make progress on a problem - but in the context here it seems to me at the moment that you're just being somewhat mean; I don't see what you hope to gain other than to make someone feel bad.

The problem isn't for Google, it's for the receiving mobile devices.

Oh, puh-lease.

I hate XML as much as the next guy, but for a phone that can do 20 fps OpenGL scene the cost of parsing XML that is most certainly valid and as used by XMPP is effectively nothing.

> Oh, puh-lease.

Don't be a dick.

> for a phone that can do 20 fps OpenGL...

It's not just the phone, there is server overhead too, which everyone seems to be ignoring. While XML has a lot of problems, I never said it was the problem with XMPP (although it certainly isn't helping).

>> Oh, puh-lease.

> Don't be a dick.

Oh, puh-lease.

> there is server overhead too

It's all going over the SSL, remember?

I have a life-long passionate hate towards XML, but you have to look at a larger picture to understand why Google is killing XMPP support. It's not the performance, it's not the lack of the protocol elegance. It's the fact that they are building a wall. Facebook got their herd walled and it seems to be working well for them, so it's only natural Google is doing the same. The XMPP was in a way, so it's gotta go.

Other than the unnecessary snark, I don't think we disagree fundamentally. I'm just as annoyed by walled gardens as most people here.

But NIH/control is an unlikely reason for brewing their own protocol. If they simply wanted a wall, they could just disable federation. They've done it before, and it's working for WhatsApp, it's working for Facebook, it's working for Microsoft [1].

G already has a lot of experience with XMPP and it's probably just the engineering reality between what they want to accomplish with Hangouts and what it would take to retrofit XMPP or an existing protocol. Throwing out the baby with the bathwater? Maybe. But it wouldn't be the first time, and won't be the last.

[1] Remember that FBChat, Lync etc are internally proprietary, but they do expose XMPP gateways. Intentionally preventing federation is shitty, but dropping XMPP at the core is not.

Let us say each message exchanged in XMPP is about 200-300 bytes. If in a session (time between app foreground and background on a mobile device), user exchanges 10 messages, we are talking about net data transfer of less than 3K bytes. Each of those characters might be compared to 10 tokens while parsing (thee is scope for significant optimization). I think, the battery cost of 30K comparison on CPU isn't comparable to the cost of rendering one of those messages on the screen and it might be sub 1% of the cost of keeping the screen lit for that user session.

Optimizing parsing cost, at the expense of breaking foundational philosophy of their communication services, is hardly justifiable. XMPP has several overheads, XML will not top that list.

zlib-compressed XML cannot hold a candle to protocol buffers in the department of space efficiency.

Indeed, its the wrong tool for the job. EXI compressions would be better.

http://www.w3.org/XML/EXI/

http://exificient.sourceforge.net/?id=performance

http://xmpp.org/extensions/xep-0322.html

Reply on news.ycombinator.com