Hacker News Reader

Comment by scrollaway | Hacker News Reader

Here's what I don't get about Google.

A few years ago, they were one of the only players in the IM game, alongside MSN, AOL IM, ICQ... have you heard about any of these lately? Of course not, they're all dead.

Talk was built on top of XMPP, an open protocol, which helped its popularity as third party clients could connect to it. Talk was also built in to gmail, and at the time that was revolutionary: A fast and lightweight chat app, right in your browser. In (one of?) the most popular mail service providers at the time!

Google has failed to capitalize on any of that. They completely ignored Talk. But then when Facebook did the same thing, oh suddenly they had to compete. So they rebuilt Talk on top of a new protocol. This time, it's proprietary. This time, it's much slower in the browser. Oh and you lose half your contacts if you upgrade to it. But at least now it works on phones?

So they tried building this new closed chat ecosystem for no good reason, and they used their Android market share to do that. People didn't like it, still used Whatsapp, still used FB Messenger, still used Viber, and the now hundreds of alternatives there are, all incompatible with one another because everybody's gotta reinvent the wheel.

You know, I can get behind that XMPP wasn't up to the task - I tried dealing with XMPP myself and it's a frustrating piece of work. But the way Google has treated IM is appalling. Really backwards. They built a good product, then completely ignored it, then built another in an attempt to reinvent it and become more locked down, butchered the old one and are now losing everything. Who's making these decisions exactly?

Matrix is probably our best bet when it comes to open chat protocols, but it's honestly not mainstream ready. In the mean time, I use Discord (https://discordapp.com/) for essentially all my communications. I have completely moved off Talk, Hangouts, Skype and even most of IRC (which has frankly fallen way too far behind more recent comms tech, even as an open protocol). It's proprietary, but at least it gives me text+voice (+ soon video) and doesn't suck - and there is no open choice I can make at this point that is approachable enough that I can convert people to it.

View on HN

Not just failed. Google betrayed XMPP. You can blame Eric Schmidt for it. He gave some lame speech about how other instant messaging services aren't playing fair, so Google should also become a walled garden. And that was it. They made Hangouts and stopped caring about federated IM idea.

It's a wonder we can send e-mails between many servers. Imagine someone like Eric Schmidt driving it. We'd be stuck with incompatible AOL and Compuserve forever.

And about XMPP shortcomings - sure, it's not perfect. And if Google thought they can do better, why didn't they propose some IM-next as an open federated or P2P protocol? Because "don't be evil" is off the table I suppose.

People tried proposing to use Discord for me, but I'm really not interested in another walled garden closed protocol, without FOSS clients and servers.

If ever there was a software example of killing the goose with the golden eggs.

Imagine if they thought like that about email?

Google Wave was supposed to replace email.

But actually they replaced email with gmail. Now you can't send email from your own server because it will be blacklisted on gmail by default and noone on gmail will receive these emails. So you have to use "cloud email providers". On Android they changed "Email" client to "Gmail" client.

> Now you can't send email from your own server because it will be blacklisted on gmail by default

That's not correct. I tried sending e-mails using my own local SMTP server, and it comes to Gmail just fine. Not that it's a good idea in general.

Why do you say it's not a good idea?

Managing your own server is in general pretty hard (for sending or receiving). Unless you want to invest a lot of time in security and so on. It's simply way overwhelming. And if you don't invest in security, you are inviting trouble.

Sure. I've been running my own mail server for almost 20 years, but you seemed to imply that the bare act of sending email from a personal server to Gmail was a bad idea for some reason.

I've been running my own email server since... forever. In 1985 it was uucp ! paths and sendmail. I got a "for real" domain just for email in 1993, still over uucp dialup. DSL always on in 1999. But it does become harder as time goes on.

On the list of computing tasks I do every day, running my email server is WAY down there on the list of difficulty. Email servers are probably even easier than web servers.

Which doesn't necessarily mean you take necessary steps to seriously protect it. Or may be it means you are an expert and all that is trivial for you. But in general it's clearly not.

I guess my point is that you need to know your audience. Here on HN, running your own server is not a rare skill.

Ever heard of mail in a box[1] ? it does all this hard work for you.

[1]: https://mailinabox.email/

Stop spreading FUD.

I've been running my own email server for a year without delivery issues to Gmail or Outlook/Office 365.

Just make sure your VPS IP is not blacklisted, and play the DKIM/etc game. You'll be fine.

It’s not that easy.

I’ve had proper DKIM, proper SPF, my IP wasn’t blacklisted anywhere.

But I still ended up in Spam.

So I had to send for about a year emails to friends every few days, replicating our discussions on WhatsApp etc (so they’d have organic content), and I’d ask them to mark them as "not spam".

Now I end up in their normal Inbox.

EDIT: Why the downvotes? Please comment instead what you think is wrong – or how else do you think I should get my personal domain trusted?

That is correct, the age of the domain and the "not spam" click matter a lot.

If you want a new domain and have little traffic (e.g. personal domain), you're fucked.

> EDIT: Why the downvotes? Please comment instead what you think is wrong – or how else you think I should get my personal domain trusted enough?

I don't understand the downvotes either. So I upvoted your comment to counteract them.

I've found myself doing that more and more lately. I'll upvote a comment I disagree with if it was made politely and in good faith, and it appears to be getting downvoted unfairly.

UPDATE: In case someone ever wants to know more details about why my server failed verification, here are checks for that domain I took when changing DKIM keys last (today):

http://i.imgur.com/9zqILRk.png

https://mxtoolbox.com/SuperTool.aspx?action=blacklist%3akusc...

http://i.imgur.com/VIH4jlt.png

thanks for making me learn about mail-tester.com though you should probably obfuscate the email addresses in those images.

Don't worry — those email addresses in the images are public ones controlled by me.

I even have one of them in my HN profile if I recall correctly.

Thank you for this.

I plan to roll my own email server in the near future, due to legal and privacy concerns. As I understand it, in the US email archives aren't protected under the Fourth Amendment because they're asking for data from a third party, and the Fourth Amendment only applies to self-incrimination.

IANAL, but...

5th amendment is self-incrimination, 4th is unwarranted search and seizure.

I don't know the current law, but it used to be that e-mails left on a server for more than 6 months were treated like abandoned property, which has a lower-bar to clear for the 4th amendment. I seem to recall hearing that this was changed a while back though.

How the 4th amendment applies to any case is complicated though because it's very much a sliding scale of intrusiveness, with higher levels of intrusiveness requiring more cause for searching.

As I understand it, in the US email archives aren't protected under the Fourth Amendment

Currently, as I understand they are not. But, there is legislation [1] which proposes extending Fourth Amendment protection to e-mails and communication stored on your behalf on third party servers. I've already written to my Congressperson in support of that bill.

[1] https://www.congress.gov/bill/115th-congress/house-bill/387/

Yeah, people can still run their own mail servers without too much work. I've had one for a couple years but I ran into issues with Google tagging my mail as spam at first. It was a while back but I think they want domain owners to verify their ownership with Google's webmaster tools.[1]

My server worked better after adding a TXT record with google-site-verification + all the normal DKIM stuff.

EDIT: here's a better link https://support.google.com/a/answer/183895?hl=en

[1] https://support.google.com/webmasters/answer/35179?hl=en

The thing is, and this is the current state of email in general, aka not just google, using an email server other than from [big cloud], like say from my local ISP or a large shared hosting provider, you will run into issues. Usually fine, but every few months your (my) server's ip will randomly get blacklisted for a few days, and for a small biz you'll have customers occasionally not receive your email.

Random blacklisting maybe is less of an issue if you run your own server with a unique ip only you use, but for most people it's getting harder and harder not to use gmail/outlook/lycos.

So ya fuck spammers.

I built my own mail server, and use it for every day communication. If you do everything correctly (mainly setting up DKIM and SPF), you get pretty much trusted by gmail automatically. You might have to make sure your IP address isn't already blacklisted, though I didn't have that problem.

Might have to give it a month or so to settle down aswell and instruct users to mark email as non-spam for automated emails (which helps)

Same here. My personal mail server is running for 4 years. Never had problems being blacklisted by gmail.

I have been running my own mail server with no problem.

> Google Wave was supposed to replace email.

At least Wave was an open protocol including federation, even based on XMPP in fact. It maybe was the last thing before Google transitioned into a mostly walled garden.

> Now you can't send email from your own server because it will be blacklisted on gmail by default and noone on gmail will receive these emails.

I run a small mail server for some years now, no issues sending to gmail.

"On Android they changed "Email" client to "Gmail" client."

No, they didn't. Those are two separate apps. The Email one is part of AOSP. The Gmail one is part of Google Apps. Although many OEMs roll their own email client, as part of their custom skin.

On my Nexus 4, (Android 5.1.1), when I click on what used to be the email icon, I get a screen that says "Email has moved. The Gmail app now lets you view all of your email accounts." It has only one option (and a help link). That option is "TAKE ME TO THE GMAIL APP".

How do I get to the plain email client?

You install it

From where?

I run my own domain and mail server, no problems emailing major providers, including gmail.

> Imagine if they thought like that about email?

Exactly :) I updated my comment above.

What do you think the end-game for Google's various extensions for e-mail are?

...and people switched to slack.

... and end to end encryption would be a thing.

Open federated protocols with no obligation for providers to upgrade as the security landscape changes are a betrayal of user security and privacy of the highest order.

HTTPS is only becoming reasonable because Google can use its monopoly power and forced upgrade mechanism to bully website owners into adopting better practices. If we had a fragmented ecosystem of many open source browsers with user-consent upgrades and similar market share like the open web people wanted, certificate transparency would remain academic.

Personally, I consider the median advocate of open federated protocols to be more culpable for the wholesale violation of user privacy than the median NSA staffer.

No, closed ones are a betrayal. If someone doesn't upgrade, you can easily exclude them from communication and even build such logic in the protocol itself. It will pressure them to be up to date. I don't see an issue here.

You can't do that if the ones that refuse to upgrade are some of the bigger providers on the network.

You can if you care about security. That's exactly how the very same Google Talk was cut off from the rest of the XMPP universe, even before they decided to shut it down (they refused to support server to server encryption).

Isn't Hangouts mainly some kind of video chat? I've never used it because I'm not interested in video chat ...

No. It's a standard (and surprisingly shoddy) instant message service that also integrates video chat.

It does text and audio too. I mostly use it for text.

If you're looking for a future in federated chat: matrix.org. I think there's real potential in the protocol and Im slowing moving most of my chat activity there.

Matrix looks interesting, though I'm not sure why they use HTTP for transport. Sounds a bit counter intuitive, since HTTP was never built for proper duplex communication. XMPP (in the browser) had to use BOSH to work around that, then came WebSockets, WebRTC and so on.

Another intereting one is Tox.

Less problems with firewalls?

Perhaps this is the point we need legislation to require that IM services with above X users in the relevant jurisdiction must provide a full, open, protocol and enable interoperation via the published protocol? That's the only way I see the companies involved would move to enable cross-platform communications?

Aside:

>It's a wonder we can send e-mails between many servers.

I have a shared account on a small ISP, Microsoft won't receive our emails, even when replying to an email from a Microsoft user, even when user whitelisted. [It's probably the same with some other suppliers?] The only way to reply to people using Outlook/Hotmail/Live (I'm a user of hotmail for something like 16 years) is now to send via MS servers - basically had to make an account for this purpose.

The sending IPs aren't blacklisted with Senderscore (or other blacklists, I mention senderscore because they apparently provide domain reputation services for MS); some "related IPs" have medium reputation scores (senderscore 72).

Yeah, that's one of those problems which are hard to solve without global collaboration, and current major IM participants are too selfish and greedy to understand that concept.

I'm not a big fan of legislating technical standards, but in this case I think I could support it. IM has become a fundamental communication mechanism. Imagine how held back society would be if we had 50 different voice standards rather than the singular phone system we have now. IM needs to be forcefully standardized.

Don't we already have GSM and LTE, and who knows what else? And I'm not aware of any legislation for the use of those over others. As far as I know, as long as I have the right to use the frequencies, I could start my own cellular telco that transmits whatever format I want.

Right, the point is that you don't have the right to use whatever frequencies you want because the usage of the frequency spectrum is highly regulated, and for good reasons.

Basically none of which apply to IM.

> As far as I know, as long as I have the right to use the frequencies, I could start my own cellular telco that transmits whatever format I want.

Point I ("I have the right to use the frequencies") directly depends on point II ("whatever format I want"). When applying for using a certain frequency band, you have to specify usage and purpose, and you can't just change your mind, i.e. you can't apply for GSM frequencies and say you will be running a GSM network, but then run the PavelLishinOverAir protocol.

>When applying for using a certain frequency band, you have to specify usage and purpose, and you can't just change your mind, i.e. you can't apply for GSM frequencies and say you will be running a GSM network, but then run the PavelLishinOverAir protocol.

While this is mostly true, in practice (at least on the 450-470 MHz band I work with), you can give fairly vague purposes, like "telemetry" or "SCADA", and it doesn't matter what vendor or what protocol you use. In fact, there is no section of the licence that says "I am going to use protocol X on equipment Y manufactured by vendor Z" or any part thereof (emissions designators are a different story).

I don't have any experience will cellular licences, though. Given that carriers frequently repurpose spectrum as technology changes, I can't imagine their licences say "GSM only" or anything like that. The band that used to be 3G GSM is being used for 4G LTE on my phone right now.

This likely differs a lot between countries (though there should be a common core due to ITU of what is being regulated how by whom); in my country the cellular frequencies were auctioned off for dozens of billions of euros to the cellular service providers, with strings attached, e.g. that the networks had to at least use and support 3G.

The same regulatory body (BNetzA) is also responsible for licensing telecommunications providers in general. This is probably the crunchpoint - if you're not a licensed telecommunication provider, it's simply illegal to offer telecommunication services.

Huh, I didn't know that. Guess it's time to pivot PavelLishinOverAir to become some sort of drone taco delivery service.

GSM and LTE use public goods (spectrum).

Spectrum is owned by the public, and regulated as spectrum is a limited resource. IP messaging isn't by any practical means a limited resource.

> As far as I know, as long as I have the right to use the frequencies, I could start my own cellular telco that transmits whatever format I want.

No. Spectrum is regulated in terms of what technology you can use, and kind of usage is supported. You can't buy LTE spectrum and broadcast radio on top of it, and vice versa. You can't (in most cases) take 2G spectrum and deploy LTE on top of it.

> No. Spectrum is regulated in terms of what technology you can use, and kind of usage is supported.

Interesting! I was not aware of that. Thanks!

Also, bandwidth is a function of how much spectrum is available (for example between 1900MHz and 2000MHz there is a chunk of 100MHz) and the spectral efficiency (for example, 1 to 2 bits per Hz available).

For example:

If a carrier "owns" the block between 1900MHz and 1940MHz, and let's say LTE supports 2 bits/Hz using Frequency Division Duplexing (each direction, up and down, gets a chunk of spectrum), with 3 sectors per cellular tower, the carrier can support a total of 40x2x3 = 240Mbps per tower, or 120Mbps each way.

If the tower serves 1000 users (they serve far more in dense urban areas), each user will have 120kbps of capacity, and if they all use it at the same time, that's the speed they'll get. If there's a single user, the maximum down speed for that user would be 120/3 = 40Mbps.

> I'm not a big fan of legislating technical standards, but in this case I think I could support it

I feel another such example is IPv6. It could move much faster if it would have been mandatory already.

China has effectively mandated IPv6, and IPv6 certainly is popular in China (I think all mobile phones in PRC use IPv6, I may be wrong) - but let's not forget that China is promoting IPv6 because APNIC's allocated IPv4 space for China was comically undersized.

Nope. If mobile network uses ipv6 then people would have no problem accessing Facebook

You do realize how firewalls work, right? You have iptables and ip6tables. Sure you need entries for both, and sometimes ISPs forget, but it's easily implementable in all major routers and operating systems.

Firewall rules are not yet implementated on ipv6 network in China by ISP. It is a fact, and actually people are using ipv6 network as a means of bypassing the wall.

Unfortunately this is not the fact. While some ISPs are experimenting with IPv6, the vast majority are not on that train. Indeed, IPv4 address space has practically run out in China already, but many ISPs are taking the nasty NAT-like approaches, Carrier-grade NAT [1] for example. And it is particularly common in mobile. So far, IPv6 is only (relatively) well accepted among universities (CERNET [2]).

[1] https://en.wikipedia.org/wiki/Carrier-grade_NAT [2] https://en.wikipedia.org/wiki/CERNET

The Great Firewall likely affects the reliability of everybody's stats, but China seems to be way behind the US in terms of IPv6 deployment. (<3% versus 30%):

- https://www.google.com/intl/en/ipv6/statistics.html#tab=per-...

- https://stats.labs.apnic.net/ipv6/CN vs https://stats.labs.apnic.net/ipv6/US

And everything else would have slowed down because of resources (human and other) devoted to ipv6, and the complexity of ipv6.

Though I don't think the legislature even uses IM let alone understand it and understand it technically.

Keep it simple. All protocols for communication and data formats for storage must be published as open source or public domain. This is to prevent or reduce lockin, walled gardens, etc. No other change. They can still compete in numerous other ways.

Although this is an appealing idea, I don't think it is realistic. Lobbying would destroy any legislation almost immediately, and there would still be the issue of competing standards.

I think a more achievable version would be that any online service which maintains user data must provide a method for a user to retrieve that data in an open-source format. Some companies would deliberately obfuscate formats, such as calendar appointments as JSON rather than iCAL or another existing standard, but I think it would give users more control than they currently have.

"Lobbying would destroy any legislation almost immediately, and there would still be the issue of competing standards."

Oh, I agree. You're describing the effect of a corrupt government allowed to prevent or eliminate laws meant to protect its voters. I was talking about what law would be necessary if voters cared enough to push one.

And the result is OOXML, which is well known for being a trainwreck and which nobody - including Microsoft - has been able to implement properly, despite being an "open standard".

"And the result is OOXML"

The result of open-sourcing all data formats and protocols would obviously be all the data formats and protocols released as open-source. OOXML is not the only one in existence.

My point is that companies forced to do this would create abstruse formats, and then even misimplement them, to thwart those trying to interoperate.

Oh yeah. They'll probably try that. That will either be defeated with tons of sweat as it largely was in products like OpenOffice or the legal recommendation modified to attempt to deal with it. It might also be handled in the courts if the law includes language that specifically instructs to attempt simple, readable protocols to "facilitate interoperability." Obfuscations could be argued as the opposite of that in court.

That will either be defeated with tons of sweat as it largely was in products like OpenOffice

Then why do you need open standards? Every major proprietary IM protocol has been re-implemented by open source software, including Hangouts: https://bitbucket.org/EionRobb/purple-hangouts

It might also be handled in the courts if the law includes language that specifically instructs to attempt simple, readable protocols to "facilitate interoperability.

XMPP, the thing Google is being roasted for dropping support for, is often decried as being very complex and hard to implement correctly. If genuinely open protocols often end up like this, how can you possibly punish companies for writing complex, hard to implement protocols?

"Then why do you need open standards?"

"This is to prevent or reduce lockin, walled gardens, etc."

What I said. I'll add that it's hard to copyright API's to sue over compatibility when the API's and their implementation are released as open-source software.

"If genuinely open protocols often end up like this"

What are you talking about? There's all kinds of results from protocol design. You chose one of the worst ones. I was a PSYC fan over that if I had to choose an IRC replacement or open chat standard. A test for new ones might be trying to run it through LANGSEC's Nail, Cap n Proto or ZeroMQ to see if they handle it easily. Anyway, it's easier to do compatible implementations if you have source and can't be sued/imprisoned for imitating it.

Which would be obvious, flout the law, and allow for legal action to be taken.

The company itself has to use the format for live message exchange, have to publish a full working protocol document.

With OOXML Microsoft aren't required to be able to read & write a files correctly when it's written to the protocol. As I envisage it the requirement would be to open the protocol actually used "on the wire"; again with Word Microsoft don't use OOXML as primary.

I think it's workable except perhaps in requiring political will. It's an internet age equivalent of allowing third-party car parts manufacture.

Will it be obvious? How? Browsers often fail to implement open standards like CSS, even when they have nothing to gain, it's just that shit's hard.

Microsoft don't use OOXML as primary.

Sure they do, that's what .docx files are. They've been the default since Office 2007.

Microsoft was compelled to create and implement the standard so that governments can comply with their requirements to use "open standards". And they followed all the steps, including getting it past ISO, despite the mess that the format is. And so governments can now tick the "open formats" box when purchasing Office. It's all a sham, and I don't see how it'd be otherwise with this proposed law.

I don't think it needs legislation as such but could be worked from a licensing point of view.

Everyone moans about AGPL. Fine. Make something better that says if you're going to interoperate with a service, you better damn well actually interoperate. Make the license viral like GPL and hopefully when we all push in that direction, everyone needs to join up.

Korea mandated through legislation that all online shopping had to go through ActiveX controls, because that was the only way to guarantee secure encryption at the time. Now it's practically the least secure way to online shop and even Microsoft recommends not using ActiveX, but it's a huge amount of work to strip that infrastructure out and make everything work 'properly' so it's still an issue even today.

In short, legislating a solution is a workaround that can turn out to be a huge detriment not too long down the road. I'm no libertarian, but it seems like a bad idea to me.

That's a good point - especially important if someone's thinking about legislating a particular technology. On the other hand, maybe in this case it would be enough to legislate a requirement for openness of the protocol? I.e. whatever it is, if it has enough many users, it has to be fully available to third party integrations.

This feels reminiscent of net neutrality and also general competition laws. I wonder if there's something in that which could be drawn upon?

What about SMS? Everyone has it, it works, and is usually fairly instant. It's not very feature rich, but could there be an SMS 2.0 protocol that was backward compatible?

Isn't this RCS?

That was interesting to google.

The first link I got: https://www.theregister.co.uk/2015/04/02/south_korea_to_depo...

Sounds like a straw man, just because South Korea made a mistake doesn't mean legislating a standard is bad.

> IM needs to be forcefully standardized.

This is totally crazy to me. It would be like the government legislating when and where people have spoken conversations with each other.

No, it's not. What you're proposing would be like the government saying you can only use this specific service, which may be run by the government, to communicate with others. Standardizing IM is making is so that if you want to talk to your friend across the country you're able to using different applications but the same protocol. Forcefully standardized would be mandating any messaging app that's used in the country would have to follow the protocol. It's almost the same thing that happened to fax machines in the 70s and 80s before the Japanese government stepped in.

How would this look different to SMS, except that the protocol might be more feature rich.

It's not something needed. It's a plus, even if it influences society a lot, it's not something like food or healthcare, where if someone plays unfair people die. And it's not like net neutrality either, because you are not costrained by space (putting cables underground). People should be free to choose a service, and if the server want to stop providing the service or change the way it does ut, it should be free to do so. Anything else you can use contracts, or change provider, as long as there is net neutrality in place. And it's not something that messes with your health, so no justification for regulation aside from an arbitrary and unprovable "society being held back". Being unable to make something because of regulations, that's some real and provable holding back of innovation.

Except we still do have many different standards for the phone system. As an example the main codecs used for encoding voice today are μ-law (North America), A-law (Europe), GSM (2G/3G), AMR (3G/4G), iLBC (VoIP), G.729 (VoIP).

That is just for voice, now imagine you are a phone company and want to send a call to another phone company. There must be so many different things you need to cooperate on to make it happen - billing, switching, physical connections, numbering, codecs, etc. Yes the actual protocols may be standardised, but the main part is cooperation.

It's not standards we need, it's companies that are willing to cooperate and work together. None of the big tech companies want to do that now though. Everyone wants their own walled garden so they can say they have more users that X competitor.

(I'd argue that XMPP is good enough, or with a few extensions could be made good enough, for a company who really wants to push open IM)

And what if they don't publish the protocol? Do we block downloads of their client?

Even if we assume they are American companies, and we can sue them if they violate the rule; how on EARTH would we define an IM protocol broadly enough to avoid an easy avoidance of the law, but narrowly enough to not basically block all proprietary protocols?

Good questions.

If a company doesn't publish the protocol in use within, let's say 3 months of a valid request by a third-party then they'd be liable to a fine, perhaps something like $1 per day per registered user, plus costs of government and registered third parties in pursuing the interoperability request.

Small companies might escape, but if Facebook tried it they can't really run away. You'd need to be able to issue notices to ISPs to block those convicted of breaking the law, such orders are already part of UK law (against torrent discovery websites for example). Again you'd get some leakage but big providers like Microsoft couldn't feasibly expect enough customers to move to VPNs to bypass such a block.

IM client definition could be hard but in practice you'd have clauses requiring a judge to decide if the traffic exchange amounted to an IM communication. I see no problem with having a broad definition, if the definition were narrow then it would just get worked around. The definition would probably include email as a subset.

Were there other person-to-person communications protocols you are concerned would fall in the scope that definitely shouldn't be open?

I can think of a few... one example that comes to mind is a video game. The communication protocol to synchronize the game could fall in scope, and I think video game companies have the right to have their own proprietary game network protocol.

> You'd need to be able to issue notices to ISPs to block those convicted of breaking the law, such orders are already part of UK law (against torrent discovery websites for example)

And most of us in the US hope we never lose the right to open access in the way citizens of the UK have. I can't think of a worse thing to happen to the internet. The UK has proven that once you allow a government to censor the content of the internet they will aggressively and rapidly expand that power to censor and control.

Are you implementing spf?

I have a personal mailserver box in a cheap colo, and gmail won't put messages from me to my gmail-using friends in their inbox. It's shameful what email has become.

World you be happy to put the offenders in prison? Or if they didn't pay fines, to show up and collect?

i kinda doubt anyone sees protocol violation as a criminal offense.

it's wild to me how quickly "we should find a market-external pressure source for forcing interoperability given the profit motive seemingly guarantees lack of federation because it relies on competition as its motive force" becomes "we must frogmarch innovators to the death pits"

The correct way to solve this problem is through raising awareness of privacy issues and supporting efforts to create competitive and attractive open source communication protocols, not legislation enforced by a prison/military industrial complex. What you're suggesting is to hand the State further control over our choice in communications platforms and trust them to never abuse it.

I have been helping out a charity that uses a small web hosting company for email.

Any emails they send to Gmail automatically go into the spam folder.

Why? I reckon it is because there is no SPF configured, no DKIM configured, etc.

We could try to get that small web hosting company to configure all that.

Instead, the plan is to move to G Suite for Nonprofits.

I run my own e-mail server too and constantly run into issues:

http://penguindreams.org/blog/how-google-and-microsoft-made-...

If I send a new person an e-mail, I have to contact them on Facebook and tell them to check their spam folder. Fuck Google and Microsoft and their terrible e-mail filtering. Just because people are so dumb they will open any ransomware sent to them doesn't mean you drop all e-mail you don't trust with no way to account for it.

How long have you been sending mail from your latest IP Address?

I run some websites that have their own mail servers. One thing you didn't mention which can help is to be sure to use an email signature. Basically short emails are more likely to be flagged as spam, especially if there's an attachment.

In use 12 years, only ever used to reply (except when messaging businesses), never done mailings. Of course there is a footer with bricks & mortar address, phone, unsubscribe link.

Had problems with hotmail before, it's just associated IPs being factored in too strongly. Same ISP has an IP address that spam has been sent from; support responds with won't fix. In their favour at least they responded.

Domain has SPF but not DKIM.

Thankfully no other mailbox provider blocks us. We could pay to be added to the third-party "friendly" domains list of course ...

I'm surprised they wouldn't whitelist our domain but leave it on a tight leash.

Why don't you properly configure DKIM and SPF? Then you won't have this problem.

From the fine article he linked: SPF, DKIM and DMARC are all domain verification systems for validating e-mail’s origin to prevent spam. I have all three records set in DNS records for all the domains I send e-mail from, verified they were correct using testing tools, and I still get flagged as spam.

Google uses a lot of signals. The article above is on http, I saw a drop in spam flagging when the web host on my email domain was https only.

Historically giving the government more power to legislate the structure and content of digital systems has been disastrous.

They have already mail email incompatible. They haven't changed protocol but best of luck if you run your own email server.

The world made it impossible to roll your own server, not Google. The proliferation of spammers in email has necessitated a cottage industry of security and filtering and all manner of mechanisms, automated and otherwise, to deal with the avalanche of bullshit streaming through that protocol.

Google's the blame party for a lot of things, but not the loss of email freedom.

I don't know a whole lot about XMPP, but since it's federated, wouldn't it have the same avalanche? It seems like your choices are to use something interoperable and get spammed, or use a walled garden and you don't get spammed because the gardener has pitchforks.

> I don't know a whole lot about XMPP, but since it's federated, wouldn't it have the same avalanche?

It absolutely did. Everyone decrying how XMPP federation was killed conveniently forgets how useful it was to spammers.

Since this is IM the most you could do would be to generate a lot of requests to put the spammer on someone's roster list. In the unlikely event that a particular rogue XMPP managed to convince anyone to let them add some bogus users to their roster before it was blacklisted, it would take literally seconds to resolve the issue by removing them from your roster. So there isn't really any incentive to do spam over something like XMPP.

I have been on the federated XMPP network pretty much as long as it has existed and I have not gotten a single spam attempt. I guess I could post my jabber ID somewhere to make it more sporting (they need that to even try) but I really don't want to add random people to my roster. Email is intrinsically different that way.

This is not true. There are email standards out there to help prevent spam. But right now Google simply ignores all this even if everything is setup correctly. I think Google has a responsibility in helping me debug _why_ an email is spam and give tools to fix it. Currently, there is 0 information, 0 support. The webmaster tools simply say everything is awesome and dandy. And you can't do anything else.

Here is today' world work flow: 1. Buy a domain

2. Have to use Google because your mail will end up in spam with any provider (unless it's outlook or some special provider).

I've never heard of Google being particularly helpful in debugging literally anything. According to their "support", Google software is perfect and if it's not working for you you're doing something wrong.

You realize that if they did offer that support, every spammer in the world would be using it to make sure their spam gets through.

I got good mileage with the mail server on cloudron. Also, mail-tester.com is awesome to check spammyness.

Yes! Mail-tester is the best checker I've seen in a very long time. I'm amazed it's still free. Shush.

It was a product driven decision. The number of active federations was absolutely tiny and the number of messages sent over those links even tinier. The amount of XMPP spam was catastrophic, and for most users the only knowledge they'd ever have of XMPP was when they got a friend request from buy.herbal.viagra@cheap-online-meds.com. XMPP federation was a great idea, but it never had adoption, and at some point Google had to decide between UX and software purity - With one side having a clear high cost and unclear benefits.

(I was a Google SRE, sitting near the talk team. Everyone wanted XMPP to succeed, but the numbers didn't even come close to working out)

Product driven or not, that decision was backward, not forward directed. And as I said, if they thought XMPP was deficient and couldn't address spam or what not, where is their better federated and open alternative? They never made one.

Why spend time building a federated system when nobody was using the last federated system?

Ultimately most users just don't care about federation. Them's the facts. That's why everyone takes PayPal and only nerds use bitcoin.

Slack vs IRC, too. Twitter vs GNU Social. I'm sure there are more.

Firstly not nobody. Secondly, how can you grow it if instead of moving it forward, they roll it back? That's the opposite of progress.

> Ultimately most users just don't care about federation. Them's the facts.

Not really. Everyone is annoyed by this issue. But there isn't much they can do. People need to register on N different services and use N different clients to communicate with users of those networks when they need to. Do you think they appreciate this mess or find it highly convenient? But they don't have any other option (except may be not communicating with some of them at all).

People care about convenience, not federation. Google Talk and Facebook Messenger both supported it for multiple years and the number of people who used it rounds to zero. This is where we ended up. There was time for it, just no demand.

Launch a terrific, convenient new email service that doesn't federate and watch how quickly users will choose federation over convenience.

They did, it's called Facebook.

Slack if you don't like that example.

Those are IM services.

That work asynchronously even when I'm not online, that I can check later, that have functionally replaced almost all of the communication that used to happen over email...

It's technically and functionally identical to non-federated email.

Slack doesn't replace email. That's a silly marketing slogan. Imagine for a moment if it really did. What if Slack became so popular that everyone stopped using email? That would be horrifying. You wouldn't be able to communicate with anyone without using Slack because it doesn't federate with any other service. Every website would have to integrate with Slack for user account registrations. Etc. Decentralized federated communications is the backbone of the internet. The fact that IM services have never broadly adopted federation dramatically cripples the potential of the medium with horrible fragmentation and balkanization.

...and yet, here we are, where most people who wish to contact me will send me a message on facebook before they'll send me an email.

It's not fragmentation if there is a single one that the whole planet uses.

You're assuming here that federation = progress. That's the source of the disagreement.

Federation is clearly progress. Balkanized non interoperaiblility is not.

If you notice serious problems with federated messaging, it's an entirely valid choice to abandon federation instead of trying to make federation work. There's no evidence that a system that's (a) federated, (b) open to everyone and (c) non-spammy is possible in the first place; maybe it is, but even more likely you have to choose two out of three at best.

I don't see anyone repeating this fallacy w/regards to email.

The only difference between email and federated IM is email started out (mostly) federated and IM systems started out (mostly) not.

Only inertia is what drives this skepticism of federation in IM.

Just as there is no technical reason GateKeeper couldn't succeed on iOS just as it does on macOS, there is no technical reason federation couldn't succeed in IM as it does with email.

Just perceptions have to change.

> XMPP federation was a great idea, but it never had adoption,

And your alternative @ google was yet another closed source protocol that doesn't work with any other closed source protocol(obviously, because they are closed source). Now imagine if the internet worked that way, Google wouldn't be where it is today.

Surely the right answer would have been to enable users to report spam invites, and to kill spam-sending domains. Kinda like what is already done for email. Could probably repurpose a lot of the existing infrastructure.

It was done. Domains are cheap. User patience is not.

Why Google felt betrayed by Microsoft:

Larry Page commenting about it at I/O 2013: https://www.youtube.com/watch?v=9pmPa_KxsAM&t=2h54m20s

Strategist commenting about it: http://www.ucstrategies.com/unified-communications-strategie...

Microsoft's one-way integration was them not exposing presence and typing notifications to services they federated with (but they were "taking" that data from others).

Larry Page says there he would like to see more open standards used, and he is sad the industry isn't moving in that direction. So why isn't any of their new IM technology an open standard?

Because being constrained by existing standards, rather than being free to make the changes to the protocol that you think will make the product better, is a huge trade-off. The benefit is that your users get that interoperability with e.g. MSN Messenger or Skype or whatever.

However, if no one else is doing that, if no one else is playing fair (like Microsoft accepting things like typing or presence notifications but not sending them in return), then it turns out you're being constrained, and possibly doing additional work to add features, and the person benefitting the most is someone else's users and not your own.

Someone has to start doing it, otherwise you'll get the catch 22. So "others aren't doing it" is not an excuse.

But they did start doing it with XMPP and kept it that way for a while.

So. They had to persist, and address deficiencies too. That's the only way to break the cycle.

Then I assume their standards and implementations are completely open, and they allow third parties to federate?

I really doubt that Google were constrained for technical reasons. XMPP is completely extensible. I've worked with it for over a decade and there's absolutely no technical reason why Google couldn't have extended the XMPP specification and put out their own XEPs (XMPP Extension Protocol) which would have given them all the functionality and features they needed.

XMPP's only major downside at the time was mobile battery life due to XMPPs limitations over cellular. This could easily have been resolved by using a non-XMPP accepted handshake (as WhatsApp did) or by using a completely different protocol for mobile as Facebook did for mobile (MQTT).

Google decided to go for a walled garden approach as a competitive advantage. There's no technical reason for it.

Because their old one was an open standard, and they were effectively the only one open one (and the last open one standing). So I find it totally reasonable that they stopped caring about it (and probably would again if another major player did).

I find it a very lame excuse. If they care about it, it shouldn't depend on others. In the end, it shows they didn't care enough.

Probably costs a bunch for not that much convenience though. Tragedy of the commons in action.

That's the case with most of interoperability / standardization efforts. But without it, the whole situation ends up being the current mess.

I disagree. I can understand why you think so, but an analysis of the situation will point out something different.

Google got talk working almost by accident. And it was the most feature-complete chat app for years, at least. And then, they abandoned it for years. This isn't to say it had many features, it had few. But it worked well, was integrated with gmail, was compatible with XMPP and geek's IMs, ...

After that, they killed all the compatibility before making an app that was yet again a lot more feature complete : hangouts. And hangouts got the best features for quite a while.

And then Hangouts started wildly bashing it's weight around. Forcibly taking SMS was one thing. Refusing federation. Forcing Google+ account and lots of extra info. And so on, and so forth. Hangouts was a good app, but created a lot of ill will in the process.

And then it was seemingly abandoned. Feature frozen, with the excuse that all these features had resulted in an extremely difficult to maintain app that they couldn't add features to. Chatting without having a gmail being a big one (whatsapp allowing you to chat with "everyone in your phonebook"), status, reliability (and showing clearly and timely when it isn't/cannot be reliable because disconnected, not 3h after connectivity gets restored), video call quality, adding non-gmail users or just screens to video chats, not allowing bot interaction, apps in the video chat, ...). And this lasted for years.

Inside of China, some chat apps demonstrate how chat apps can be monetized in a way that users appreciate : allow chatting with companies and give those companies the ability to show interfaces for transactions. E.g. buy a coffee. Line at starbucks ? Open up whatsapp, because you're in starbucks it shows starbucks as a contact (you can add it permanently) and there's a button "order coffee". Select what kind of buvaranicpoppafrappadongieccino you want, you pay through the app, and you've just skipped the queue. Next time you do it 5 minutes before arriving. App takes cut of transaction value (just like credit card payment does).

Next, other apps turn up. Whatsapp, Lyne, ... and so forth. And they caught up with Hangouts. Surpassing it in some ways, behind in others. Mostly they're superior in letting you find the people you can chat with. And then they passed it by. And then they left it far, far behind. It's not (yet) the case that they truly dominate hangouts in features, but it's getting close.

And lo and behold : people switched to the (sorry) better apps. I'm not entirely sure why anybody is either surprised or complaining. These changes look a lot like they're making it worse, not better, but we should give them the benefit of the doubt.

Microsoft does support full federation. As long as you federate Skype servers. I know that's a bit lame, but since they bundle most of their products with most of their other products in licensing, no one in business seems to care.

Google doesn't do even that, as they don't offer anything on-site (anymore). They have seriously failed on business markets. I don't know about consumer market, because I mostly just send dick pics for the giggles to random people I don't know.

That's the big thing about SaaS, mobile and Cloud.

I don't get that people complain about Microsoft software being closed, and anti-competitive ... well any cloud software is more closed and anticompetitive than Microsoft ever was on it's worst days : it's not that you can't get good documentation that the application you're using uses ... you can't even get at the data at all, in most cases.

Google is the best at this out of all the cloud players, but it's still abysmal. And yes, you can download exports. And there's Google takeout. Is the data there all the data Google apps use to work ? No. Quite simply, no, it isn't. There's plenty more they won't ever give to you. And that's ignoring the fact that there's 1000s of developers making apps for Google and about 5 working on takeout. Why should Google be forced ? Well, that's the standard we're applying to microsoft, isn't it ?

And this is by far the most flexible cloud provider exporting data. They're not being jerks. I don't even think they're trying to be anticompetitive. Nevertheless, they're far more anticompetitive than Microsoft.

And of course, conveniently, the DRM-breaking is now a crime on things like iTunes and Hulu and ... because it involves actually creating access into someone else's computer system or at least changing their code (even though you're paying for it to do things and only getting it to let you exercise rights granted to you by law, like copying a movie). And the result is the same. iTunes, Hulu, even Play are ridiculously expensive, idiotically restrictive (buy movie on iTunes for a trip, go to Europe, log in to hotel wifi ... no movies. But no worries ! I copied that movie onto my device first. Nope, it won't play. WTF).

Same thing goes for app stores. There Apple is the big instigator of forced-incompatibility (or otherwise: your app won't run without Apple's permission hardware). Remember the flood of articles scaring people that microsoft might do this ? They're still at it, with the BIOSes that supposedly won't run anything but windows.

The dreaded "the right to read" scenario ( https://www.gnu.org/philosophy/right-to-read.en.html ) is far, far closer on the cloud and mobile than it is on your own machines.

Thanks.

(Talking about web and native APIs)

> And I think that we've really invested a lot in the open standards behind all that

> I've been personally quite sad at the industry's behavior around all these things

> If you just take something as simple as instant messaging

> We've kind of had an offer forever that we'll interoperate on instant messagoing

> I think just this week Microsoft took advantage of that by interoperating with us but not doing the reverse

> Which is really sad, right?

> And that's not the way to make progress

They gave up on Federated XMPP and never implemented video using XMPP. If they hated it so much, they could have proposed open changes. Now we're in a terrible IM landscape. No federation at all. E-mail is federated, but is been rendered terribly unreliable simply due to spam filters.

Sadly Facebook also used to do XMPP which is annoying as well. Used to be you could login to either one from any XMPP client, this means Pidgin, which most people used to use for MSN Messenger, which I miss so much when I think of what has happened to Skype. Even though MSN Messenger had it's own weird flaws (like locking people out of their own accounts because people would force login attempts to force lock your account so you couldn't login anymore). Now Skype lets anyone grab your IP and pull it into a booter. The gaming community is full of skids.

I absolutely agree with you though, Discord is one product done right, runs everywhere, and everyone I ask to "join my discord" I can't even get rid of them once they're on it, and if I stop going on they nag at me. I secretly wish Discord had been based out of some open standard because it is well done.

> I secretly wish Discord had been based out of some open standard because it is well done.

I want to highlight that this is not realistically possible for anyone, not right now anyway.

When Discord was created, XMPP was the only option if you wanted to adopt an open standard, and it was absolutely not up to the task. Matrix was also an option but far too new.

Discord is a startup, and is run as a business (just like Slack). If you were at the head of a startup in the messaging space, and there were no reasonable open source options for messaging, you'd design your own. This is a hard process; making it an open protocol from the get-go removes a lot of your freedom.

I used to say (even here on HN) that Google was our best hope at creating a modern open source messaging protocol, by improving Hangouts and making it open. They have utterly failed.

I am confident Discord is our next best hope... assuming they succeed. Otherwise, it's back to waiting for Matrix to finish playing catchup.

>making it an open protocol from the get-go removes a lot of your freedom //

Can you expand on that. Are Discord using someone else's closed protocol then in preference to their own (which they could open) or an already open one?

Or did you mean "freedom" as in "ability to use lock-in", or something else?

They're using their own protocol. By freedom, I meant ability to iterate fast, break compatibility, not have to support unwanted clients etc.

It's something you care about less once you're big, but it matters a lot when you're starting.

Presumably though even if you're iterating hard and breaking stuff, once you're out of alpha you're not going to introduce many backwards-compatibility-breaking protocol changes? I can see doing that for v0.1 to v0.2 and maybe from v1.9.x to v2.0 but nothing about making those changes seems incompatible with sharing the details of final protocols used in your production level clients.

Take a very real example: Discord is working on video chat right now. As part of that work, it's not unthinkable they'd need to severely rework the voice/text protocols because they want to bring all of them in line with one another, reduce their tech debt or what have you.

If it's an open standard, they need to check in with everyone, document the move, potentially have to explain it or depending on popularity won't even be able to justify doing it. And suddenly, you see they're losing their competitive advantage for the sake of pleasing a few people on HN.

If it's a closed standard, they do whatever they want, don't have to justify or explain it to anybody. They can turn the protocol into fairy dust, run two versions of it for a while, and require a client upgrade if you want video chat.

Now, a few years down the line once Discord is established, has a solid business model and the protocol is clearly not changing anymore, then we can talk about making it open, allowing third party clients and we can really seriously start bugging them about it.

I very much want this to happen but Discord does need to be successful first. You'll get nowhere by bugging a startup to spend time on what could potentially kill the business.

>If it's an open standard, they need to check in with everyone //

You're missing the point I think (or I got cross-threaded). It's Discord's standard, making it an open standard just means the protocol details need to be public such that a third-party client that uses their protocol can send and receive messages to & from their client(s).

The people to be pleased are not just HN readers but everyone who uses an IM system as each system could, if desired, then speak direct to others. In theory it wouldn't matter that I have Facebook and you have Google, I could still message you. If the market works effectively the best protocols could then win, as could the best UX, even if they came from different companies.

If I were drafting a law on this then it would have an exclusion for SMEs or new companies, it would as you say be an unnecessary regulatory burden to require protocol openness prior to establishment of a service. We're looking at million user plus systems.

and it was absolutely not up to the task.

In what way? The chief complaint I've heard about XMPP over the last couple of years is about excessive power consumption on mobile. And my own experience is that those concerns are highly over-stated.

My anecdotal experience is also that all the purpose built closed protocols like Facebook Messenger and Slack both seem to use way more power consumption "in idle" (no message traffic) than I recall ever dealing with for an XMPP client...

I've never seen users actually complain directly about an app's "excessive power consumption". Can we all admit that one has always been an excuse?

The only other complaint I've heard about XMPP was that it was too extensible. There wasn't enough mandatory features in the base protocol to expect good clients for fancy things and there were too many extensions to need to follow to write a good client if you tried to track "fancy thing du jour"... which is as much a feature as a bug (and the whole reason for that X in XMPP).

Has nothing to do with that. Put yourself in the shoes of the people behind a new messaging platform that needs a competitive advantage. Using XMPP is a massive technical restriction, and would give almost no benefits (Discord gets to be opinionated on how servers and channels are organized).

Discord's product isn't a hosting service, their product is the quality of the service as well. This quality would be impossible with XMPP as it exists today, within the constraints of a startup. Remember, they do voice and they want to do video as well. Google's own efforts on Jingle failed due to performance.

As for mobile, it's not just power consumption, it's connection lifecycle. XMPP last I heard had extremely poor support for lossy connectivity.

Has nothing to do with what?

Again, power consumption/connection lifecycle isn't directly a competitive advantage because your user doesn't care so long as it works. (Slack and FB Messenger seem to have terrible power consumption and connection lifecycles for mobile but also seem to be doing just fine.)

(Not to mention those are also things that could be fixed in an open standard, if people cared to contribute that effort. The unfortunate reality is that isn't a competitive advantage either.)

The competitive advantage is to find excuses to dismiss open standards, whether the arguments are technically or factually correct is another matter, and build walled gardens.

But more crucially to my previous point, being technically superior isn't really a competitive advantage if a user doesn't notice it and in fact, can slow one from getting first mover advantage by shipping something/anything faster and sooner. I really don't think that on a power consumption or connection lifecycle standpoint any of the closed source protocols are really all that much better, and we have mostly nothing but anecdotes to trade on that question because the companies want to maintain their secret sauces.

Facebook Messenger used to be an egregious offender here and chewed through battery life. I uninstalled it two years ago and that helped my phone's battery life noticeably.

React native is an 'open standard' :p

Don't forget Allo in the mix, from Google, that could replace Hangouts! Take a product people use, and replace it with something people don't, then replace that with something that people also don't use.

I tried Allo once, which was enough.

It was actually fascinating, because it was among the most viscerally negative reactions I've ever had to a piece of software.

It reminded me of that trick where you play back someone's speech to them in a very slight delay and it becomes very difficult for them to speak, only in this case with my inner monologue while trying to participate in a conversation.

Really impressively disturbing and uncomfortable. An amazing feat, but not something I will ever use.

Isn't Allo video only? Hangouts is first-text, with optional video.

Duo is video-only. Allo is text-only

I don't disagree with your points but your timeline fudges the timescale quite a bit. I've researched this [1].

Facebook Chat went live 2008-04-06. Google didn't view them as a threat in messaging but a threat in social networking; they thought they had messaging in the bag with their 2008-09-23 release of Android 1.0, and their 2008-11-11 update to Google Talk which brought voice and video calling, and their 2009-03-11 acquisition of GrandCentral, which was soon rebranded an invite-only Google Voice. But Facebook kept growing and growing and it had an integrated chat on a website where people went to spend their time, instead of Gmail, where they went to manage email.

To combat Facebook on social networking, Google launched Buzz with aggressive auto-opt-in on 2010-02-09. Buzz fizzled and attracted controversy for its aggressive piggybacking on Gmail, so Google tried again with Google Plus on 2011-06-28. That was a better effort, and it included the features "+Messenger", a text chat, and the video chat "+Hangouts". By this point Facebook had more than 700 million active users, and won messaging handily; its lead was cemented by the acquisition of WhatsApp on 2014-02-19, as Google continued to flail about.

In a post of mine last year [2] (which includes an older revision of the timeline linked in [1]), I speculate that it was Facebook Chat that killed the mid-2000s chat networks of old like AIM, Yahoo Messenger, and WLM, rather than Google Talk or any particular missteps of those incumbent chat networks. For example, I was surprised to learn that AIM was present in the iOS App Store at launch -- of course, there were no push notifications at the time -- not until 2009-06.

[1] https://news.ycombinator.com/item?id=13465483

[2] https://news.ycombinator.com/item?id=11114518

Honestly, what I think we're seeing is evidence of an existential crisis in some quarters in Google. There's a lot of people who still believe in "do no evil" and who furthermore believe that part and parcel of that is not building walled gardens. Then you have other people - including Larry - who think walled gardens are the only solution. The PM for RTC at Google wrote a long email about how basically interoperability is dead, XMPP is dead, federation is dead, and every attempt by Google to build open standards has been effectively exploited by competitors.

So a lot of influential management thinks going full FB is the way to go. But a lot of other people at Google think open standards are still the way of the future, or can be if a company like Google is willing to champion them.

The result of all of this is a complete and utter dumpster fire from a product perspective. It's the classic "if we're not pulling together then we're rowing against each other" problem. I think Google would be able to execute just fine if they could settle on what they want to achieve. Instead we have this.

> But at least now it works on phones?

This is actually really, really important to people.

You are saying it like if XMPP never worked on phones.

It didn't work well.

But who cares?

Even if there were, hypothetically, no possible way to make XMPP work, and Google really needed a custom protocol, they could still have released public documentation for that protocol, and open source client and/or server code.

Indeed, that's sort of what they did in 2009 by creating Jingle, an XMPP extension for voice chat, and releasing libjingle. But no such luck for the new stuff.

I could understand giving up federation, maybe, since it creates a spam problem. Though for Google to throw up their hands on spam is a bit ironic, given that Hangouts sits inside Gmail, which has world-class spam filtering. But federation was never really necessary: most XMPP clients support multiple accounts, and it's not like there were a lot of huge XMPP services - only one, Google. You just add that as an account in addition to whatever smaller-scale/organizational XMPP servers you use. (You know, the thing people used to use before Slack.)

Similarly, most XMPP clients support multiple protocols, so for Hangouts to use a custom protocol is not the end of the world. Annoying, but if the benefits are clear enough… it's fine.

But no. Like with most messaging services these days, Hangouts' protocol isn't documented, and nobody has bothered to reverse engineer it. So you're stuck with the official clients. Maybe not the end of the world, since unlike the bad old days of Windows-only clients that left Linux users in the cold, today any desktop OS can use the web client. (And non-top-two mobile OSes don't have enough market share for people to care about their not being supported, and many of them can run Android apps.)

Not the end of the world, unless you want all your messaging services in one place - people used to see a lot of benefit in that, and there are a lot more services now.

Or you don't like the official design. Or you don't like bloated web clients. Or you prefer free software.

Sigh.

XMPP doesn't work well on phones in part because Google, Facebook, Apple, etc. didn't want it to work well on phones because they wanted their own proprietary messaging services.

> Gmail, which has world-class spam filtering.

If by world class you mean huge amounts of false positives and destroying the ability for people to send messages reliably between one another, then yes. It's totally world class .. a world class failure.

As they say, Works For Me. Admittedly, I don't know how many important emails I've lost over the years, but I don't remember ever being contacted elsewhere about an email that turned out to be in spam. I have a bad memory so I might be forgetting some instance, but if so it must not have been terribly important, or I'd remember it… On the flipside, there have almost never been any false negatives. I remember being quite annoyed a while back when there was a brief period where a trickle of spam got through. Other than that, I've been spam-free.

(I also remember being IMed by spambots using XMPP federation a few times.)

This does come at the cost of letting Google see all my email. I'm not happy about that, but I'm skeptical that a fully encrypted solution could provide filtering as effective. Certainly a self-hosted solution cannot.

Are there any objective looks at this? I know personally I manually check the spam folder, and out of the hundreds of messages a month that end up there I've never seen anything important get mis-categorised. A few (<<1%) mass emails end up there inadvertently, but nothing that I would notice if I didn't check.

standard complaint was that it was a battery killer.

As far as I know, XMPP is still being developed further. The guy who wrote the Android Conversations (.im?) app is on HN and says it is a serious option for phones now.

(Except it costs money so most of my WhatsApp contacts will not switch.)

Very true! Although I think op meant it tongue in cheek. The joke being that the old version could just as well have run on phones, had they tailored the app for it, instead.

In fact, that's precisely what they did. The old talk app for android worked fine, even before they terminated federation.

I don't mean to diminish the importance of working on phones, however interoperability (at least as a third party) is also important and "not losing contacts" is pretty much at the top of the priority list.

Hangouts failed both.

Google had (no longer has, IMO) the clout to design and drive a solid, open source messaging protocol with a reference implementation, sitting in the inbox of hundreds of millions of users and on hundreds of millions of phones. Instead, they created a slow, mediocre, closed source product that didn't interact well with their own video/voice calling system and fractured their own userbase between Hangouts, Google Talk, Google Voice and now even Allo.

I forgot about Allo. Didn't they bring back the Google voice app? Yea it makes no sense at all.

Yes, if anything, some of us stopped using Hangouts not because it supported phones, but because it didn't support desktop machines.

(No, being forced to run chrome just to see chats in a crappy corner window isn't experience on par with iMessage / Telegram / Viber).

> So they rebuilt Talk on top of a new protocol. This time, it's proprietary. This time, it's much slower in the browser. Oh and you lose half your contacts if you upgrade to it. But at least now it works on phones?

Talk had a full-featured native phone app way back in Android 2.1 with multiple device support, which Allo still doesn't have today. I used it religiously on my Nexus One.

fun fact, ICQ is doing just fine. They just stopped bothering staying in the English speaking game, it's now mostly a Russian thing. Doing quite well, still under active support and development.

"Advanced" runet users dropped ICQ for telegram long time ago

Everything has consolidated, so it's all about walling off users to increase friction between the Google/Facebook/Microsoft/Apple universes.

My gf gave in and bought an iPhone 7. I haven't owned an iPhone since 2009.... iMessage is pretty neat... But still, I'm sticking with Android.

Yep, and I wish a bunch of HNers with lots of time would spin up an OSS Google Apps (for most of the apps at least) replacement. Then people could do one-click installs on AWS or their own infrastructure, and the like...

> there is no open choice I can make at this point that is approachable enough that I can convert people to it.

https://meet.jit.si/ works just in the browser. What's the problem?

Talk was built on top of XMPP, an open protocol, which helped its popularity as third party clients could connect to it.

By the time google talk came around, third party clients were easily connecting to all the other major IM services. I don't think the open protocol made any difference to its popularity.

Talk didn't have its own client outside of the Gmail panel when it launched. It would have been much harder for Google to bootstrap at the time, if they hadn't been able to just point people to existing clients that supported XMPP. Third party clients made Talk useful faster (thus popularizing it). Sure, several of the third party clients were pretty good at reverse engineering weird protocols while also supporting open ones, but there's no guarantee they would have bothered with Talk, especially if Talk existed only in a web browser.

Talk didn't have its own client outside of the Gmail panel when it launched.

Didn't it? I think it had a client from the get go

https://en.wikinews.org/wiki/Google_Talk_released_in_beta_fo...

Interesting, I seem to recall an entirely different order of release. Blame it on personal bias (was already using XMPP at the time of Talk's release) and/or aging memory, I suppose.

I remembered it exactly the other way round and also wasn't sure. It's curious/disturbing that events barely over a decade past are difficult to google. 'What was the first gtalk client'. 'When was gtalk integrated into gmail'. 'When did gtalk get an OS X client'.

Another example is iMessage which is very popular amongst the Apple fans. And Snapchat essentially is another great mobile to mobile coms app people don't quite get yet, but it's more powerful and entertaining than "Hangouts"

> You know, I can get behind that XMPP wasn't up to the task - I tried dealing with XMPP myself and it's a frustrating piece of work.

XMPP is weird, true; it relies on opening an XML document and sending fragments of XML for the duration of the session and only closing the document at session completion. Very odd. But it performs extremely well. And once you get your head wrapped around the constraints, it's not so bad.

This has got me thinking...

If the same GTalk was available on mobile, would I have installed WhatsApp?

Huge fan of Matrix, really hope it takes off. They've got the right idea and it plugs into other things like IRC very cleanly.

Regarding IRC, how do you feel about ircv3 [1]?

[1] http://ircv3.org

I have been following it closely but it feels like too little, too late.

IRC is just not the protocol of the future. There's far higher chances for Matrix to become popular or discord to be open sourced.

Are there any IRCv3 implementations? Is freenode planning on running it?

I'd love a totally OSS alternative to Slack. Maybe we'll see it with the recent Gitter acquisition?

Rocket Chat (https://github.com/RocketChat) is a totally OSS Slack alternative.

Many popular ircd's support different IRCv3 specs. Freenode, Rizon, etc all support different specs/capabilities. You can see a list here: http://ircv3.net/support/networks.html

(Warning: bias/plug; I contribute to Lounge)

In terms of OSS alternative to slack, I recommend you check out Lounge, a self-hosted JavaScript web client (think: selfhosted IRC version of Slack, minus/plus a few features)

https://github.com/thelounge/lounge

Slack, minus the ability to gracefully run in any sort of long term roaming context without babysitting. There's no decent reconnecting logic on either client or server, you just have to notice it's gone quiet, and then figure out the undocumented way to restore it without nuking your settings. Only figured out by accident "/connect" without parameters is valid.

I tried to fix it, but the client is a pile of terrible code and needs a complete rewrite.

Well, if you get disconnected and don't get reconnected, the server tab mentions to try `/connect`.

Additionally, if you want to rewrite the client, you're welcome to start and open a PR - it's a community-run project. I've had Lounge running for months now without issue. Contributions are always welcome.

Mattermost is an existing OSS alternative to slack. It's terrible.

Gitter is more likely to be good but to be completely honest, it will take a lot of work to not be mediocre.

Freenode is part of the IRCv3 working group and I believe uses it internally somehow. IRCCloud is the largest IRCv3 implementation, they are fully IRCv3 compatible afaik.

Hi @scrollaway,

Thanks for your feedback on Mattermost. Could you share more on what you'd like to see changed?

For folks who haven't seen Mattermost, here's a demo video: https://www.youtube.com/watch?v=AKqHWqrAgpk&t=1s

Also, I noticed from your HN profile links you're a UX designer.

We're actually working on our new generation of mobile apps in React Native (https://github.com/mattermost/mattermost-mobile).

If you're open to critiquing some of the screenshots from early builds, or even participating in the design discussion, we'd welcome your involvement.

Also, we'd welcome you to join our community server at https://pre-release.mattermost.com/

I'm on there as it33 if you want to DM me to share more of your thoughts?

PS: Regarding IRC, Mattermost connects to IRC, XMPP, Slack, HipChat, Matrix and other systems via Matterbridge: https://github.com/42wim/matterbridge

Hi Ian, good to see you here. I really shouldn't have been so harsh :) Mattermost isn't terrible as a product. The issue is that it's not going to replace Slack because, although it's FOSS, it's not an alternative to Slack. Let me elaborate:

Very few people know about Mattermost right now unless they use or used Gitlab. Whenever I hear Mattermost brought up, it's associated with Gitlab, which has a reputation of being "the slower, less-good GitHub" (and I love gitlab, don't get me wrong, but I agree with that reputation). Therefore, Mattermost, at least in my circles, has a reputation of being "the less good Slack".

And the main issue (unless I'm misunderstanding, but I did go through your pricing page again to double check) is that you do not offer SaaS except to enterprises. Which is super backwards. You're not going to be able to gather mindshare if you require people to set up your product themselves to even just try them out. Nearly none of your potential users have the expertise to do that, and those that do aren't going to want to do it unless they know what they're getting into beforehand. The only remaining ones that already do are Gitlab users, because they've had it set up for them.

Github, Travis, Slack, Discord, Sentry, half of the integrations you find on GH, what do they have in common? They're hosted and free to try. Make a free hosted version, make it very limited if you have to and promote upgrades/self-hosting, and then you have a shot at getting solid mindshare.

On a lesser note, since you asked for feedback, here's a couple more pieces of advice:

- The demo video you linked me is great. I see Mattermost has changed quite a bit since I last tried it, for the better. So I visited mattermost.com to try and get another look and it's super barebones on screenshots, I found no videos, there's no demo. After browsing the site, I have no feel for what mattermost looks like. And you don't link to your community server anywhere I could find, not even on your contact page! - Consider making public Mattermost instances optionally readable without an account - read only, and enable that for your community server!. This would differenciate you from Slack and Discord and would improve the situation above. I realize it's a lot of work. - On pricing again, it's irresponsible to put 2FA behind Enterprise only. Please don't do that. Push notifications could also stand to be in the free version, imho.

Cheers, hope that helps. My email is on my profile if you want to talk about this further.

Hi Jerome, highly appreciate your feedback. I can see where you're coming from.

You're 100% correct on website, we need an upgrade. We have someone starting soon to help and we'll show them this thread.

A couple of notes:

1. Push notifications are free. Paid is if you want to use Mattermost key for encryption (we choose not to encrypt things for people we don't know, but we document how they can do it themselves): https://docs.mattermost.com/overview/faq.html#are-push-notif...

2. Our intended users are organizations that need to deploy communication behind their firewalls (no 2fa from us needed) and can't use SaaS, e.g. manufacturing, banking, healthcare, public sector, security companies, companies with very strict non-disclosure agreements, etc.

Logos on our website are examples: https://about.mattermost.com/

PS: I read about HearthSim, you're running a really cool company. If you're ever in San Francisco for GDC, perhaps we could meet in person.

> Push notifications are free.

Might want to clarify that on your pricing page then; I thought the free version simply didn't have push notifications since they weren't in there.

> If you're ever in San Francisco for GDC, perhaps we could meet in person.

You bet. I'm usually in Anaheim for BlizzCon which is a safer bet, I don't normally do GDC. Feel free to shoot me an email if you're around :)

Google has ADHD and too much money for own good.

everyone get and will get IM wrong.

communication is taboo for ads. and nothing on the Internet can have a fee. see the incompatibility?

Facebook is spending on IM because it bought the first IM that dissociated itself from the Internet. every whatsapp user see it as a sms and phone company alternative.

everyone else that live by ads (Google, yahoo, etc) will try their hand at IM, realize they can't use it for ads or ad targeting, and will scratch it.

I don't understand what you don't get, this is how google works. Google does things like that constantly: buying popular softwares to kill them, make services such as google reader and kill them, etc.

Google has its own agenda and making an open and distributed internet that respect people privacy or wishes is not part of it.

With that said, despite being a poor product I rely on Hangouts daily because I can make free WiFi calls with Voice. I can't afford cell service and it lets me stay connected to my family.

So, what you don't get is why using an open chat protocol didn't turn into market leadership? Or...

Google's failure to turn Talk into a market leader has absolutely nothing to do with Talk being built on top of an open source protocol.

Reply on news.ycombinator.com