Hacker News Reader

Comment by kkowalczyk | Hacker News Reader

Unfortunately, the guy responds with his own misinformations.

Google said: "For example, mobile has several requirements around bandwidth and battery that are simply not part of the standard"

He says: "This glances over the the fact that the X in XMPP stands for eXtensible, which still results in proposals for new protocol extensions every month."

His misinformation is this: XMPP is XML based. The bandwidth that Google most likely refers to is a result of verbosity of XML. You can't eliminate that by extending XMPP, something that a guy who serves on XMPP Council should know.

The only way to fix that specific problem is to change the protocol. Which is what Google did.

This is an example a bad advocacy: badmouthing companies that decided to no longer use something you work on.

I also find troubling the pounding Google gets for every little mis-step.

When it comes to using and promoting open technologies Google is seemingly from another planet compared to other tech giants (Apple, Microsoft, Facebook, Amazon, you name it).

And yet the furor is not about Microsoft using completely secret protocol (Skype), not about Apple promising to publish their secret protocol (Facetime) and then not doing it, not about Apple preventing fair browser competition in iOS etc. but about Google no longer using XMPP.

A lack of perspective.

View on HN

"You can't eliminate that by extending XMPP, something that a guy who serves on XMPP Council should know."

Ehhh... well... if you're willing to put some work into the matter that may not be as true as you'd think. Yes, to speak the federated protocol to another server you're going to be speaking the XML protocol, but between a server and a client that come to accommodations with each other, you don't really have to speak XML.

You'd want to speak something isomorphic to the XML, but the XMPP XML is not that verbose on a node-by-node basis or anything (minus a couple things, see below), it really is the XML, not the semantic content. It would be fairly easy for a medium-to-high skill developer to come up with an isomorphism for use with mobile clients. (It is sufficiently tricky that there would be a few traps that could catch the inexperienced or unwary (watch those namespaces!), but it isn't that difficult.) The end result is a more bandwidth efficient protocol that is a drop-in replacement at the XMPP library layer in the client for XMPP. (Of course, if your client is programmed to fling around XML all over the place internally this won't help, but, well, that's the price of failing to abstract properly.)

Where you'd still have trouble is things like roster size, but yes, there are XEPs for that, and before long you're looking at a reasonably svelte protocol (not perfectly, but reasonably svelte) that is still largely XMPP and easy to write communication with the outside world.

What's really difficult is semantic IM translation between entirely different systems, not protocol translation. ejabberd, the server whose internals I am most familiar with, essentially works by internally speaking Erlang terms with itself, and translating to the XML protocol only at the edges. It wouldn't be very hard to drop in different serialization at the edges and get a different protocol; as proof of concept of the fact this is possible, one could easily simply serialize those Erlang terms with Erlang's term serialization, and indeed, when operating ejabberd in a clustered mode, that's actually happening across the wire as the cluster nodes communicate with each other.

However much effort that may be, it seems easier than starting from scratch.

There are a whole bunch of competing binary XML formats that promise to be more efficient than XML (both in terms of space usage, and time taken to process/parse).

See: http://en.wikipedia.org/wiki/Binary_XML

Fast Infoset is a good contender: http://en.wikipedia.org/wiki/Fast_Infoset

The furor is not about dropping XMPP; it is about dropping the philosophy of open communication they championed: "Client Choice, Service Choice and Platform Choice" (https://developers.google.com/talk/open_communications). If Google chose to drop XMPP in favor of some other protocol that allowed others to adopt and interoperate, the discussion would have been about the effort required to migrate. But this move by Google is a huge blow to those who hoped for a world of ip based open communication solution.

Google is in a position to define the course of internet and has used that power in several ways in the past. This move from an open communication network to a closed network, to me, is as defining as WebGL and WebRTC are, but in a very unhealthy way. Google's move therefore is clearly 'evil' (as per their own definition of evil) as it is a clear choice they made to safeguard their vested interests, stating that other corporations are unduly benefiting from their openness as a reason.(http://www.youtube.com/watch?v=AfK8h73bb-o [2:40])

They stated that major networks did not interoperate with them. But in 2007 AIM started interoperating with Google Talk. http://gmailblog.blogspot.in/2007/12/gmail-chat-aim-crazy-de.... Doesn't that count? What did Google do to promote federation, other than asking people to contact them to federate? It appears that when Google was small, they tried to be open to benefit from other big players. Now that they are the biggies, they don't see any value in being open.

I believe more than any other company you mentioned, Google benefited significantly by projecting themselves as the messiahs of internet, purportedly promoting openness and standardization. They captured the imagination of people like us by promising to be only doers of good, placing the objectives of internet and humanity ahead of the corporation's benefits. They did so, as long as it worked in their favor. Those initial adopters who stood with them for these objectives are either absorbed into Google to work as if Google's interests are internet's interests or they are too few and left with fewer alternatives to significantly influence Google and its corporate objectives. With many of the influential personalities deeply affiliated with Google, we have fewer voices calling a spade as a spade.

> It appears that when Google was small, they tried to be open to benefit from other big players. Now that they are the biggies, they don't see any value in being open.

Sad but true. Corporations who have to worry about keeping their share price forever in the green will have to compromise on their ideals. They will still talk about their ideals but they won't put their money where their mouth is.

This raises the question of how much we users can trust any for-profit corporation, since at some point that corporation will value its own profit above our intersts. One possible answer is that we can trust them with short-term transactions, but not with any kind of long-term relationship, especially one that's hard to get out of.

Really? There are plenty of valid reasons why XMPP is not suitable for mobile usage (most notably, the proprietary push notification systems that all platforms insist you have to use), but XML isn't one of them. If you use TLS with XMPP (as gtalk does [1]), you get the compression protocol for free, but even if you can't, there's XEP-0138 [2], which specifies the use of zlib and has been in development for over 12 years now, and finalized since 2009. XML is fugly, but it works. At the scale Google operates, it is laughable that the use of XML in chat protocols could possibly impact their bandwidth accounting.

[1] https://developers.google.com/talk/open_communications#devel...

[2] http://xmpp.org/extensions/xep-0138.html

Disclaimer: I am a Google employee, but I do not speak for Google.

I'm not offering an opinion for or against the decision to drop XMPP, but I wanted to offer some corrections to what is being said.

I think you've misunderstood how scale works. At scale, problems do not shrink, they grow.

I do not work on anything even closely related to Hangouts, but I used to run an ejabberd server myself, and I've worked with SIP a bit.

XML does have problems, both with bandwidth, (even compressed), high CPU utilization, parsing code is more complex (which leads to more security issues). XMPP also has issues, and I'm not just talking about how hard it is to get multiple Audio/Video clients to communicate[1]. (e.g. kernel resources, the incredible number of failure states, etc.)

Cheers, Doug

#1 At least early versions of iChat used XMPP with SIP. For those who don't know SIP, it stands for Session Initiation Protocol, which means that it doesn't transfer the audio/video, it's just a language for making connections. You have to negotiate a transport layer (which was often but not always RTP or SRTP). You have to exchange connection ports via SDP. If you're lucky and manage to get by all firewalls at this point, you now have to hope you're able to use compatible codecs. Interoperability was a nightmare.

And yet google wave ran on xmpp which was arguably a far far more sophisticated app doing way more communication than just chat

Google wave only used XMPP for federation. It was a nightmare to deploy and we never got it working reliably. Different jabber servers all had different quirks or implemented subsets of the XMPP extension spec.

The client-server protocol was simply JSON. XMPP was not involved. (Well, we translated protobuf messages to JSON, so it was ugly JSON, but still JSON).

(if you can't comment on this, no worries. thanks)

  we translated protobuf messages to JSON

just curious- is this the same 'protojson' I see exchanged in the current crop of APIs?

aside, any opinions on others choosing that as a standard way of communicating between internal services on disparate runtimes? (golang<->python, for example)

--

edit: offtopic, thanks for ShareJS.

>I think you've misunderstood how scale works. At scale, problems do not shrink, they grow.

I think they meant the scale of Google (total) compared to the scale of just messaging.

i.e: If you could spend a year's worth of working cutting the costs of <sevice X> by 30%, that would be fairly meaningless if it was .000001% of the total cost of operation.

Fair enough. But let's look at the scale of Google compared to messaging using your numbers to prove my point.

I don't know what Google's operating expenses are[1], but I'll use the public income numbers as a proxy, and you can adjust from there.

If you look at Google's first quarter 2013 numbers, they pulled in 13.97 billion in revenue, and I'll subtract the 2.96 billion in traffic acquisition costs to come up with 11.01 billion. We'll multiply by four[2] to get a rough estimate of yearly income: 44.04 billion.

If we assume that messaging is millionth of a percent = one hundred millionth[3] of Google's expenses, then the total cost to google is $440 per year, so saving 30% just doesn't make sense.

If, however, it's one hundred thousandth of Google's expenses (still a ridiculously small amount), then you're talking about $440k of yearly expenses, and you now need to compare a year of salary to 30%, or $146k.

At this point, depending on the availability of engineers (whether or not you have to factor in hiring costs) it starts to make a lot more sense.

Cheers, Doug

#1 and I probably wouldn't be able to divulge them if I did. :-)

#2 Each quarter is different, but x4 is a good rough estimate.

#3 Your numbers.

It's not just bandwidth, but the cost to process that XML, which is very expensive compared to other serialization formats. Then there are a whole host of other concerns that result from XML being text-based (cdata, preventing runaway parsing, etc). Not defending what Google has done, but the reality is XMPP is simply the most common among a lineup of really crappy chat/presence protocols.

Now, XML itself is one problem, but the "culture" (eg, usage norms) around XML is another -- much like Java's cultural preoccupation with abstraction and boilerplate, XML-based formats have a tendency to veer towards the unwieldy and verbose. XMPP is no exception.

While XMPP has been instrumental in propagating standardized presence and IM, its age is showing. Unless we develop a modern, open replacement that is natively suited to the needs of today's Internet (voice, video, file transfer, group messaging, etc+), we will see companies continue to develop proprietary solutions that are locked behind walled gardens.

+XEPs and extensions are insufficient; too many optional extensions hinders interop and contorts the system beyond its original design specifications. Just ask anyone who's tried to implement chat/buddylists on top of SIP presence.

Sorry but I just don't see how bandwidth and XML could ever be a problem for Google.

Youtube alone makes up a significant amount of the Internet's total traffic. Sending compressed XML isn't a problem for Google.

Just because Google has a lot of capacity doesn't mean they're wasteful with it. At scale, minor inefficiencies add up to a lot. That is the incentive for protobufs, etc. For a more concrete example of seemingly trivial bandwidth savings, check out the HTML source to their 404 page (it validates!): http://www.google.com/asdf.html

That 404 page is a relic from the past. Take a look at what makes up their search page and you'll see that it is almost 1 MB uncompressed. That's for a page that displays a logo, a navbar, a text input and a button.

Sure, the page has really become a one-page app with some advantages that come from pre-loading, but jeez, it is still 1MB!

The problem isn't for Google, it's for the receiving mobile devices.

> Sorry but I just don't see how bandwidth and XML could ever be a problem for Google.

Then why comment? If you have nothing to input from either side. Doubt is not a valuable commodity.

We not here for the sole reason of providing value to you. Although we may, incidentally, hope to do so.

Some of us need our thoughts clarified by the community.

In any case, I find that learning happens in an atmosphere of uncertainty, humility and exploration. Doubt, thus, is a valuable commodity in any community which encourages learning, and I'd like to believe HN is one of them.

Yes it is. For instance, if someone spends years trying to solve a problem and they can't do it and they tell you that they doubt that you're going to be able to solve it in a year and probably shouldn't base your strategies around doing so then that information is worth something - whatever the relevant entire costs are for your having done otherwise.

If you respect people, then their opinions tell you things about the search space in which they live. Heck, even if you don't respect people it at least tells you nsomething about how widely known a particular thing is within comparable sections of society.

There are, potentially, some good arguments for filtering your information to keep newbies out of higher level discussion where people are trying to make progress on a problem - but in the context here it seems to me at the moment that you're just being somewhat mean; I don't see what you hope to gain other than to make someone feel bad.

Neither is being a snarky dickhead.

Maybe you should refrain from commenting until you have something "valuable" to offer, too.

Let us say each message exchanged in XMPP is about 200-300 bytes. If in a session (time between app foreground and background on a mobile device), user exchanges 10 messages, we are talking about net data transfer of less than 3K bytes. Each of those characters might be compared to 10 tokens while parsing (thee is scope for significant optimization). I think, the battery cost of 30K comparison on CPU isn't comparable to the cost of rendering one of those messages on the screen and it might be sub 1% of the cost of keeping the screen lit for that user session.

Optimizing parsing cost, at the expense of breaking foundational philosophy of their communication services, is hardly justifiable. XMPP has several overheads, XML will not top that list.

Oh, puh-lease.

I hate XML as much as the next guy, but for a phone that can do 20 fps OpenGL scene the cost of parsing XML that is most certainly valid and as used by XMPP is effectively nothing.

> Oh, puh-lease.

Don't be a dick.

> for a phone that can do 20 fps OpenGL...

It's not just the phone, there is server overhead too, which everyone seems to be ignoring. While XML has a lot of problems, I never said it was the problem with XMPP (although it certainly isn't helping).

>> Oh, puh-lease.

> Don't be a dick.

Oh, puh-lease.

> there is server overhead too

It's all going over the SSL, remember?

I have a life-long passionate hate towards XML, but you have to look at a larger picture to understand why Google is killing XMPP support. It's not the performance, it's not the lack of the protocol elegance. It's the fact that they are building a wall. Facebook got their herd walled and it seems to be working well for them, so it's only natural Google is doing the same. The XMPP was in a way, so it's gotta go.

Other than the unnecessary snark, I don't think we disagree fundamentally. I'm just as annoyed by walled gardens as most people here.

But NIH/control is an unlikely reason for brewing their own protocol. If they simply wanted a wall, they could just disable federation. They've done it before, and it's working for WhatsApp, it's working for Facebook, it's working for Microsoft [1].

G already has a lot of experience with XMPP and it's probably just the engineering reality between what they want to accomplish with Hangouts and what it would take to retrofit XMPP or an existing protocol. Throwing out the baby with the bathwater? Maybe. But it wouldn't be the first time, and won't be the last.

[1] Remember that FBChat, Lync etc are internally proprietary, but they do expose XMPP gateways. Intentionally preventing federation is shitty, but dropping XMPP at the core is not.

zlib-compressed XML cannot hold a candle to protocol buffers in the department of space efficiency.

Indeed, its the wrong tool for the job. EXI compressions would be better.

http://www.w3.org/XML/EXI/

http://exificient.sourceforge.net/?id=performance

http://xmpp.org/extensions/xep-0322.html

Re: "His misinformation is this: XMPP is XML based. The bandwidth that Google most likely refers to is a result of verbosity of XML. You can't eliminate that by extending XMPP, something that a guy who serves on XMPP Council should know."

He directly addresses the Bandwidth concern by noting that interoperability on the server side is irrelevant to mobile anyways:

"Of course none of these concerns for mobile are applicable for server federation. As far as I know, the Google Talk client on Android doesn't even use XMPP as the client-to-server protocol."

I should have made the link to the document on XMPP on Mobile Devices more explicit: http://xmpp.org/extensions/xep-0286.html. In short, the use of compression in the TLS layer negates the oft stated verbosity of XMPP. It really isn't the problem you think it is. More info: http://stpeter.im/journal/1384.html. I might be a fanboy, but I do know what I'm talking about.

Compression doesn't help CPU utilization, which in turn doesn't help battery consumption. In addition, if the protocol is "chatty" (i.e., sends lots of packets even if there isn't any communication going on -- say, when users change their status/presence information, for example) it means waking up the CPU, which in turn means less power efficiency.

Please read that first document and also my remark about SIFT, which basically allows for delaying or dropping presence to cut down on antenna use and battery consumption.

Hmmm... As the author of XEP-0286, I think I ought to chip in a bit.

1) Compression does help keep the transmissions under the FACH threshold on most networks. This more than outweighs the CPU cost.

2) The bulk of the data to be compressed isn't XML, but text.

3) The chattiness of the protocol - ie, transmissions that are not needed at that point in time - will indeed change radio states, but not only did Google tackle this problem privately, but the XSF has also looked at it in detail.

4) Really, the CPU is a non-issue here. It's all about the radio power states. This is likely to be entirely different at the server end, however efficient XML parsers can and do exist to mitigate this, and make it more distributable.

The only way to fix that specific problem is to change the protocol. Which is what Google did.

Creating new and better protocols is OK. Making them closed and non interoperable - is not. XMPP is not ideal, but it's open for participation in its development and for its usage. Whatever Google decided to create is not open, and that's their main failure.

Nobody expects better from Microsoft, but they still expect better from Google, hence the sense of betrayal and subsequent furor.

That's exactly the point, though- why should Google have to defend themselves so much more? Apple have the most tightly locked in messaging protocol, but no-one says a word about them opening it- even after they promised to.

Well, Apple gets incredible amounts of crap from HN and other places for their lack of openness. But at this point people just assume that Apple will not open up their stuff. And for the most part Apple doesn't make promises about that.

Google, on the other hand, got a free pass for a while because they had a lot of marketing to the effect of "Look at us! We're the openest openers who ever opened openness!" and... well, they have a history of spectacularly failing to follow through on that with, say, Android ("oh, it's totally open -- except for this release that only our partners get! And except for all the bits that most people don't realize are separate and that we'll C&D you for distributing! But it's OPEN, we promise!").

So people are starting to no longer give Google the automatic pass on this. Which should be expected.

I don't recall Apple ever promising that but if they did they wouldn't have explained why they can't or won't do it because they don't want to have it both ways. They are fine saying nope it's not open deal with it. Google seems to want it both ways which puts them under the microscope. Apple gets plenty of criticism from the same types of people for not being open. I don't buy into the poor lil' Google angle of this. They know exactly what they're doing.

"Upon the launch of the iPhone 4, Jobs promised that Apple would immediately start working with standards bodies to make the FaceTime protocol an "open standard." As of November 2012, it is not yet known to have been ratified by any standards body, and the extent of work by Apple with regard to this promise is unclear as Apple has not released technical specifications for the service. FaceTime is not currently supported on any non-Apple devices."

http://en.wikipedia.org/wiki/FaceTime

I read somewhere that the Facetime team heard about the open-source promise when Jobs announced it on stage. It could very well have been a spur of the moment decision made on stage.

(Not that it absolves Apple from failing to do it)

Perhaps because nobody can seriously expect Apple to commit to openness (this is the company that forbids open source licenses in its app store), where Google was already using an open protocol?

I don't think Apple actually forbids open source from the app store, do they?

As far as I understand/interpret their policy: they only prohibit so-called copyleft licenses, and/or ones which would be violated inherently by the app store's binary-only distribution model.

>And yet the furor is not about Microsoft...

The furore is partly also about Google moving away from an open protocol to a supposedly secret protocol. If Google were to publish the new protocol that Hangouts uses all/most of the animosity would vanish. And people would come up with a xmpp to hangouts bridge too.

Google has supported and even championed open source initiatives and it has served them well. And imo it will serve them well in the long term if they continue being open, instead of trying to monopolize a segment of the market.

Not a lack of perspective at all, more of an attempt at deflection because Google are being dicks. Skype's protocols predate Microsofts takeover, so attempting to 'blame' them is churlish, and there already has been a furore over Apple and FaceTime. The browser thing has absolutely sweet FA to do with this.

Google have spent years building a brand out of how """open""" they are; a lot of their marketing/fud is based around it.

Basically they co-opted the language of open source to rube nerds, while nobody is under any illusions re: Microsoft & Apple.

This.

Chromium alone puts them in the plus column standard-wise. Usually apps are judged on merit, yet in Google's case there is the misconception that they have to defend every engineering decision they make.

I don't see how this is a hot topic seeing as every other IM is an island, and quite frankly if it means directing more resources towards projects like Google Fiber, personally I think it's more than worth it to sunset GReader or move away from a limiting standard.

Sorry but NONE of what you said is true or frankly makes any sense.

Chromium will likely be less standards based now that they don't have the tow of the other WebKit companies keeping them in check. Case in point: SPDY.

Google is not the only company who has to defend every engineering decision. The reason it is being discussed is that it is fitting into this new narrative of Google being less interested in standards and more interested in their advertising centric walled garden.

And finally Google can do multiple things at once. It isn't an either-or scenario.

SPDY is an open protocol, and Google is working towards standardization of it. Plus, many other browsers have implementations of it based on Google's protocol specification, so it's not true that SPDY is an example of Google becoming less standards based.

Exactly my point.

Google develops a proprietary protocol without ANY consultation. They then leverage the fact that they own the world's biggest website and popular browser to force it by stealth into popularity. And then afterwards decide to work to standardising it despite it not necessarily being the optimum choice for everyone.

The core protocols of the internet are fundamental. And they should be designed deliberately with as many parties as possible involved. This is not the way standards should come into existence.

Yes. Standards should be designed by committee, ensuring as many cooks are helping the broth as possible. Much like UNIX and C came out of large standards consortia composed of many industry leaders, and like the HTML5 standards evolved incredibly quickly until those darn browser manufacturers slowed them down by going off and implementing things. A single engineer or group of engineers trying to solve a real world problem have never done anything good for the world.

(DISCLAIMER: I work for Google. All of my opinions are my own and not Google's.)

Yes. Standards must either come from a single absolute dictator, or from a mess of gigantic pointless committees. Those are absolutely the only two options available, and there is no middle ground of any sort whatsoever.

Calling Google a single absolute dictator seems like a huge stretch. I'd put Google somewhere between there and a mess of committees myself.

History does indicate that the middle ground is fairly sparse.

I don't think you know what "proprietary" means, you don't pay Google to use SPDY.

They solved a problem and made something better, case in point Facebook and Twitter among others have already deployed it.

If you do some research you'll find that most standards started as projects that came out of a single company or lab that were later adopted by industry.

No. You don't understand what proprietary means. SPDY is a trademarked technology owned by a private corporation.

The only reason companies are adopting SPDY is because it is an improvement than HTTP. SPDY is far from perfect. The issue is that with core internet protocols this is not the way it should happen.

>The only reason companies are adopting SPDY is because it is an improvement than HTTP.

I don't see the problem.

>The issue is that with core internet protocols this is not the way it should happen.

This is exactly how it does happen. Someone creates something, people like it, the creators don't enforce patents on it or try to keep it a secret, someone writes it up into an RFC. That's how internet standards are created. That's how they've been created as long as there has been an internet.

Look at a counterexample. Microsoft with OOXML. Here we have a so-called standard which says things like "do this the way X version of Microsoft Word does it" without actually specifying how to do it. It's not an open standard, it's a document claiming to be an open standard on paper so that Microsoft can claim their proprietary file format is an open standard. No one else is expected to use it for any purpose other than attempting interoperability with Microsoft Office and Microsoft has no particular interest in having anyone even succeed in doing that.

SPDY is not that. Google has published a reference implementation with source code. People other than Google use it for reasons having nothing to do with interoperability and Google services continue to be available to browsers without SPDY. "They didn't ask permission first" is just disingenuous griping -- who are they even supposed to have asked?

SPDY isn't a problem. Hangouts are. I don't see source code for a reference implementation. I don't see anyone but Google implementing the protocol. That's how you know something is wrong.

The point you're missing is that people don't use it because they think its good.

They use it because they don't have a choice. If you don't help and/or follow Google you don't get to play, right now, because they own a large majority of most technologies being in use by the public today.

That's the same thing people criticized Microsoft for.

Hangout is just much more wrong tho, as you pointed out.

The point you're missing is that people don't use it because they think its good.

That's the same thing people criticized Microsoft for.

SPDY isn't a trademarked technology, the name is trademarked (it should be), the protocol is not.

The way SPDY works is just like OAuth/OAuth2, HTTP, XMPP work: someone invent/improve a protocol, working on draft and get it open standard. I don't see any problem with that.

I don't see how you can jump to these false assumptions and then call my comment senseless.

Here is the "Blink manifesto" if you will: A proposal for un-prefixing https://groups.google.com/a/chromium.org/forum/m/?fromgroups...

As for SPDY it's not proprietary and it's being baked into HTTP 2.0: http://tools.ietf.org/html/draft-ietf-httpbis-http2-00

And I'm not sure what you mean by "advertising walled garden" other that just throwing around some senseless hostile buzzwords.

Reply on news.ycombinator.com