Hacker News Reader

Comment by mikeash | Hacker News Reader

Are we ever going to see unity between these services, or are we doomed to have little walled-off fiefdoms forever?

A phone number works anywhere in the world, to anywhere in the world, with anyone who has a phone. All e-mail providers work with each other. I don't care who you're with, and I wouldn't even have to know except it's usually part of the address. SMS works no matter which provider we both have.

It feels like we're taking huge steps backwards. Instead of sending a message to a phone number or e-mail address, I'll use iMessage or Google Hangouts or Skype or Slack or.... Video call? We can do FaceTime or Hangouts or Skype or....

Will these things start interoperating with each other eventually, or are we just doomed forever?

View on HN

FWIW, email only sorta interoperates...try running your own mail server and unless you get all the elements of server reputation exactly right, your email will probably not get delivered to people using the large email providers. Thanks to spammers, we've allowed our open, interoperable standard to be much more closed.

And SMS is also frustratingly non-universal. After years of having cell service with one of the large providers, I switched to Fi last year since I was going to be out of the country a lot. The biggest annoyance has been most short code SMS not working. Each SMS shortcode is only supported on a provider-by-provider basis.

I'm not disagreeing with you, I'm just pointing out that even your examples of open, interoperable protocols are instances where we've traded some of that open interoperability for convenience.

Good points, both! I think it's telling that the e-mail one is a relatively recent development. Is that also the case for SMS? I'm not too familiar with the history of short codes, but I have the impression that they're newer too.

I didn't even know what shortcodes were until this moment. I think this detracts from the main point, though. Of course automated commercial services will have their idiosyncrasies, but if you want to communicate with people, phone calls and SMS seem to work reliably, worldwide, in a universal way.

I run my own email, on a vps no less, and the only blocklist that has given me trouble is Symantec's (and a bit of Googling at the time showed it had a high false-positive rate).

Google, Hotmail, etc, all work just fine. Just make sure to set up DMARC (DKIM+SPF) and TLS by default.

That's a shame since I wrote the RFC/BCP on DNS blocklists while at Symantec.

Haha, well this is awkward..

It was a while ago, if I recall correctly the block seemed to be ip based (it reported my ip not domain-name). I host with vpsdime, while it is a vps and thus to be expected I don't seem to be on any other lists.

DMARC, PTR, no open-relay, encryption, etc., all set up on my end. The email I was trying to reach was in fact an (important to me) gov't adress. Ended up using my old gmail for that instead.

Short codes are managed per country, but within the US just about every true mobile operator should have the ability to work with the same US short code. If they don't support a given code, it is because they choose not to. Part of the approval process for getting a short code is having all of the major wireless carriers sign off on your application.

If Google Fi is using SMS-enabled wireline numbers as opposed to mobile numbers then there could be additional complications.

I have my own mail server and am running it from my home server for 15 years now, I did not notice having issues with delivery to large providers and I did not set up SPF, DKIM or whatever currently you supposed to use (I do have TLS enabled though).

I do have some mailing lists running from it, so it's possible it helped me that their users are making sure mails from it doesn't end up in junk folder.

Anyway, I would encourage anyone to run their own mail server, to prevent large provider doing what you're mentioning. Google already did this with XMPP. They made GTalk interconnected with rest of XMPP server, but as soon as they got a large base they disconnected from the rest and made their own proprietary network.

Same experience here. No special setup, no SPF or DKIM, and yet no particular challenge in sending e-mail. I feel the challenges of operating a mail server are often exaggerated. But then, I don't tend to send spam.

Wow, I am surprised. I recently had to migrate my mailserver from my datacenter to the cloud. I lost the original IP address as a result. The new IP address got flagged immediately on gmail and everywhere else. I setup SPF and DKIM, that seems to fix the issues with the big guys. However on comcast and others I still got throttled for quite a bit. i had to request csi.cloudmark.com to unblock the ip..

I wonder if you happened to inherit an IP address from someone who was up to something less than savory.

That would be my guess.

I run an email server with ten domains on it. Hotmail blocked me once, but I had no problem getting that lifted. No other issues for over five years now.

I have more of a problem with mail going the other way. I use a number of online spam traps to avoid spam, and my gf's Yahoo account often gets bounced by them.

Specific Yahoo servers regularly get flagged for spam, and if you're using a Yahoo account your mail will be filtered by many, many servers if it's sent from one of those IPs.

Oftentimes, poorly-managed VMs / VPSes get infected with scripts / botnet software and start blasting out spam. It takes a long time for IP reputation to recover, esp. under the same owner info on the netblock.

Regarding inbox deliverability from dynamic IPs (like you'd have with a residential connection), it might work sometimes for short messages with no attachments to people you've corresponded with before, but I wouldn't expect general deliverability to be very good. You have the same problem with botnet infections / malware-based spam here as well; home PCs are much more likely to get infected with that kind of stuff (though WordPress exploits are VERY common on servers), so many people running mail servers will just straight up block dynamic IPs from big ISPs and assume that nothing of value was lost. Personally, for low-volume mail such as what a home server might send regarding maintenance or alerts, I'd recommend using a service like mailgun and hooking up to that with SMTP. It's a lot easier than running your own mail server, you won't have to deal with inbound spam if you're only sending, it works well with minimal setup, they guide you through each step, etc etc.

It was a long time ago so my memory is a bit fuzzy, but if I remember correctly it was fairly straightforward. It was right before and as SPF really became worth doing. The thing that would get a lot of my customers running their own mail was the PTR not matching the name presented by the mail server and/or that name not being listed in the MX records for the domain it was relaying mail for. I did have to deal with the occasional IP that had been blacklisted on RBLs and that could be a pain the ass and typically the first order of business was actually blocking outbound SMTP except from their mail server.

Those were the days that I don't miss.

"And SMS is also frustratingly non-universal. After years of having cell service with one of the large providers" .. "Each SMS shortcode is only supported on a provider-by-provider basis."

Yes, each provider does support their own short codes, but this is not a great example of SMS being "non-universal"

A short code is just "dialling sugar" for {country_code} + {network_code} + {short_code}.

Think about it this way: It would be kind of like trying to access the following url: http://news.ycombinator

Do you need it on .com? .co.za? .io?

So in other words, if you are roaming and you want to use a short code. Just prefix with {country_code} + {network_code}.

The routing should work correctly, the switch (msc) routing tables will have the country code + network code, and route the message correctly to your network. If your network has been configured correctly, it will be able to respond.

There is nothing inherently in SMS that prevents it working. Though carriers can choose to block short codes to other networks (usually to avoid fraud). Your carrier would then need to include request the roaming partner white list a set of allowed short codes.

This isn't really correct. Short codes are hardcoded routing rules specific to each operator, not just syntactic sugar. They don't map to regular phone numbers (MSISDN) and are generally not available from other networks or overseas, unless the service provider (not the operator) makes an effort to replicate the code across operators and provide it internationally. Some countries have regulatory coordination to prevent the same number from routing to different services across different operators, but that's about it.

Source: used to work with SMS gateways for a living.

That said, short codes are rapidly becoming obsolete since data has become ubiquitous and everything is online now. The only one I ever use is voicemail.

> short codes are rapidly becoming obsolete

I wish. With the prevalence of using SMS for 2FA, short codes are very much in use. When you've got a provider that doesn't work with most short codes, you run into a frighteningly large number of difficult situations. Some providers will give you a "I didn't receive the text, call me" option. Some (grr...Venmo) don't. But it's one of those features you don't realize how often it's used until it doesn't work for you.

Yes they are hard coded on the home operator/carrier. Though this does not mean it will not route.

What country are you in?

The way I have seen them configured is they work from off-net if you prefix with correct codes.

Remember the foreign switch,smsc or ussd gateway is going to route the message if it has the correct prefix. The Routing (b number analysis) is looking up a prefix, to send it off.

It is up to your home operator to correctly handle this.

Source: 10+ years working with mobile operators

That's only true if you are in certain IP spaces that get lumped in with residential users or are public spaces that are heavily used like AWS.

I've run on-Prem email for years in different capacities. It isn't nearly as difficult as its often portrayed.

Postfix is not fun to set up and even less fun to debug if there are config issues.

But there are useful sample config files online, and once it's running it just works.

That's nothing - you should try running your own cell tower. Or just hooking up your own telephone exchange.

Email interoperability is at LEAST as good as phone interoperability, which was the GP's point, I think. But all of these things are more walled gardens than you might think.

And what you are describing are isolated instances that almost no one experiences with email or sms.

We ran our own mail server, didn't find it very difficult. If you run it off of IP blocks that have been abused in the past, might be harder.

I think the free nature of these services is what leads to this fragmentation.

Slack is an outlier because you don't actually need to communicate with everyone in the world on Slack.

But for others, it would be completely unreasonable to ask your acquaintances to pay to install Skype to talk to you. And you probably wouldn't pay for Skype if it didn't interop with whatever other people were paying for.

In a world where switching costs for consumers are so low, interop becomes less important because you can ask your contacts to install an app if it's useful enough.

> you can ask your contacts to install an app if it's useful enough.

Sure, for personal devices, but with business devices, installing an app can be a multi-month approval process.

Sounds likely. I'd say that there's still a decent cost in terms of setting up and maintaining all these different accounts, but I'm sure many users don't see it that way, or don't care.

No. I won't install a privacy invasive app (which they all are) just because you have it, never. SMS or nothing if we don't already use something else we'd rather use.

I'm sure you realise you're in the minority here.

None of these messaging apps care if you install their product or not because there are so few people who share your views.

Mobile operators can scan your SMS messages just as Google can scan Gmail (etc.)...

That is true but quite beside the point. If that was my only worry I'd have much less friction to install random spyw-ehm messaging apps.

(I believe that operators generally have much tighter regulations and laws governing what they could actually do with that, but regardless - besides the point)

how is it besides the point? It feels like your whole point...

There are other privacy concerns with messaging apps: e.g. harvesting of contacts, access to media and photos and camera on your device etc. Perhaps he/she means those.

Exactly, random example, whatsapp requires over 30 permissions on android. A fraction of those are reasonable.

Agreed, and that's why I think iMessage and (BBM when people were using Blackberry's) are the best version of this. Two people with Apple devices can seamlessly take advantage of the Apple ecosystem or switch to SMS when communicating with a person who doesn't have an Apple device.

I think you mean hangouts (as it was), which extended beyond the apple paywall onto PC desktops, mac desktops, linux desktops, chromebooks, android, and IOS. Too bad google can't get out of their own way and has started deprecating it with no replacement in site.

That is supposed to be what WebRTC is for. But major companies with their own motives are running the the major video chat solutions.

XMPP too. It can do video calling: https://xmpp.org/about/technology-overview.html

Sad that Google + Facebook pulled out of federation a long time ago. We could have had bring-your-own-client cross-network all of this stuff. We still can, but realistically we won't.

WebRTC defines how the browser supports audio/video/chat endpoints. A practical video calling system needs servers to manage firewall traversal, a directory of users, and ways to admin the system.

Who's going to provide that at no cost?

This isn't that complicated or sad. You can still call anyone in the world and still text anyone in the world. Its that the remaining spaces to compete in are feature based, and features don't move forward quickly on open standards.

There are plenty of working open standards to do the basics, its if you want to do anything beyond the basics that you end up in a walled garden.

Video calling and instant messaging have both been around for ages at this point, and remain stubbornly walled off. Both should qualify as "basics" by now.

Sure, you can still SMS worldwide, as long as they have a cell phone. But non-phone devices are becoming very common. What if you want to send or receive messages on your tablet or computer. (This works OK in the Apple world, but only because they hack it by routing SMS through your phone.)

Video calls don't even have that.

E-mail, for example, was different. I started using online services when you had AOL, CompuServe, Prodigy, etc. all in their own little worlds. That didn't last very long before they all bridged their internal e-mail systems to the internet and everyone could talk to each other.

Video Calling is a HARD problem that still has not been solved even remotely to the broader markets satisfaction. Everything from latency/compression techniques to hardware is in flux. It is a long ways away from a standard that can be easily and widely adopted and work in all scenarios.

It's hard, but I disagree that it hasn't been solved. As long as your device has a built-in camera and microphone, they pretty much Just Work these days.

Sure, bringing everybody together would be tough. You'd need a lot of discussion and probably glue/bridging code between different services. But it could be done! None of the big services are even trying to integrate with each other.

When IP video calling started being worked on in the 90s it was via an open standards process, specifically h.323. Back then the web was driven more by engineers, before money men truly tookover.

It is a miracle that the web itself has survived as one of the few open interoperable protocols. Though I think this golden period will soon end, strangled by the death of net neutrality on one side and proprietary walled gardens on the other.

Ultimately, a commons is not compatible with unrestricted capitalism.

Its easy to blame money (boogey) men when you don't understand the finite problems that have held back this space.

They are also very easy to blame when you do. Productization is ruining tech.

This. It's easy to forget that in the beginning, stuff like AOL and CompuServe offered their version of the internet, not "the internet". We've now paved the way for Comcast, Time Warner, ATT, and Verizon to do the same. Look at what the wireless ISPs already do to phones on their network today with locking, proprietary apps (many of which take advantage of zero rating), "free" phones, etc.

Sadly but somewhat hilariously, the thing which might prevent them a bit from offering exclusive-to-their-network content is the fact that in the US, you often don't have a choice of which ISP to go with, so if I really wanted some content which was exclusive to ATT's network, I simply can't switch off of Comcast (apartment building with no other option, past 3 apartment buildings were the same) since they've so successfully monopolized markets.

Edit: on the beginning: http://www.internethistorypodcast.com/2014/04/chapter-3-part...

Facetime works very well, my family uses it from 4 different continents around the world, from cities to villages, varying from toddlers to nonagenarians who didn't even go to school or know English. I don't know why something cross platform can't work as well as Facetime does.

I had hope for Hangouts, but Google dropped the ball big time in terms of user complexity and quality of product as they are wont to do.

Someone had me use Hangouts for a press livestreaming thing a couple of weeks back. I was actually surprised at how the UI had improved since I last used it.

I agree that video conferencing is rather fragmented but I'm honestly not sure how much that has slowed adoption. People just don't want to use video in a lot of cases. Essentially all my internal calls are on a videoconferencing system and I doubt I turn on the webcam 25% of the time. (We do use screen sharing a fair bit though.)

Google Duo is a vast improvement over Hangouts for quality and resilience. It has been almost as good as Facetime for me. Duo is also available on iOS and Android.

It is conspicuously absent on any other platform; even Android tablets can't use Duo, let alone a Linux, Windows, or macOS machine. So it's a hell of a walled garden.

> This works OK in the Apple world, but only because they hack it by routing SMS through your phone.

Actually, not necessarily. At least with my provider (AT&T), I can set up my Mac as a validated device for WiFi calling, so I can do SMS and actual phone calls without using the iPhone as a relay.

Nifty! I must not be keeping up with the latest developments. Apparently this is possible as of about a year and a half ago.

Yes, for those looking for more details, AT&T calls it NumberSync: https://www.att.com/shop/wireless/features/numbersync.html

T-Mobile supports this as well.

Matrix is capable of many advanced features, and can be used as the framework for others, while being an open standard.

It's all still quite new, and the drive to compete and gain market share keeps them separate, as it does in all industries. There will be consolidation in time. For now, it's a mixed blessing, but mostly a blessing.

What's new? Messaging?

> For now, it's a mixed blessing, but mostly a blessing.

In what way? You can accomplish all these things for free with your existing phone, Google Assistant, IFTTT, and very little effort. If you do it this way, your virtual assistant won't be as dumb as rocks either.

Sorry, but Alexa is still riding on the short bus.

Alexa is positioned differently. There are typically configurable DIY solutions to most broad appeal consumer products. Many just want something that works out of the box, with minimal configuration or learning required.

Practical video calling has been around for a decade, and instant messaging for two or three decades. How long is it supposed to take?

More optimistically, it could be a step toward solving the problem if it adopted a Roku-like model where all those apps are installed and logged in at all times and you can pick one from a standardized menu. On Roku you can go to Netflix, Hulu, and single-channel apps like A&E.

Bonus points if there is a standard way to 'get a call' or 'search for a contact' that cuts across the video conf apps.

iOS 10 supports this with CallKit. The "Call" action in the Phone UI can call through Skype, Messenger, WhatsApp, FaceTime, etc. and receiving a call looks like the native phone UI.

Email and phone spam also works everywhere. It's time someone did something about it with a whole new system. Calls that are not pre-whitelisted should be blocked and logged. Then at the end of the day, you should get an email summarizing all the blocked calls so you can whitelist the ones you want.

https://www.truecaller.com/ does this to a certain extent. Users flag phone numbers / caller IDs as spam, giving you the discretion to block, whitelist, or subscribe to lists to autoblock common spam callers similar to adblock plug-ins.

> Will these things start interoperating with each other eventually [...] ?

They used to. We have open or defacto standards for texts, chat, voip, and video calls. We had/have free and paid clients available. They withered on the vine, without support from the big players.

This is my biggest beef, thanks for pointing it out. All of this is great, but it adds to separation of market. In an idea l world, all of the 'big players' would sit together, and come up with a way to unify their services.

What, jabber?

I'm working for a startup that is trying to solve that problem :D

This problem was solved with XMPP/Jabber. Facebook and Google used to use it for their chat clients. You used to be able to use Google Talk to talk with other servers, but after a while they removed that, and then killed Google Talk. Facebook messenger used to have an XMPP-compatible API, but could never talk to other servers.

I don't know why more messaging services use XMPP to bootstrap their userbase. It would allow you to switch to a new service, while you could still talk to your friends that haven't moved yet.

No, no it bloody well wasn't. XMPP is a horrible protocol with any number of incompatibilities depending on which set of XEPs your particular implementation supports and which set your clients speak.

There's a reason it never took off, and it has nothing to do with corporate mendacity.

https://news.ycombinator.com/item?id=2069810

There are plenty of horrible, incompatible protocols that are widely adopted by industry (FTP, CalDAV, IMAP, DLNA...). Where there's a will there's a way.

Reply on news.ycombinator.com