HTTPS adoption has reached the tipping point | Hacker News Reader

HTTPS adoption has reached the tipping point

troyhunt.com

131 pointsdhotson83 comments

This is very misleading, because it's overwhelmingly skewed toward large websites run by Google, Facebook, etc.

The statistic is that 50% of page loads are HTTPS - the majority of those page loads are going to be visits to a very small subset of extremely popular (and well-resourced) sites, so this stat gives no indication that the remaining 50% is or will move to HTTPS anytime soon.

The real tipping point will be 50% of unique domains visited being HTTPS. And a more interesting statistic would where that figure is now, and trend data on that. I wonder can that stat be extracted from public Mozilla data...

In particular, it would be interesting to see what is the adoption rate in China, whose govt makes no secret of inspecting everyone's internet traffic.

If you control the network infrastructure it is possible to inspect the traffic while still allowing https/SSL. https://www.a10networks.com/resources/glossary/ssl-decryptio...

Only if you control the endpoints or have a compromised CA. The whole point of TLS is that you can't inspect traffic even if you own all the cables and routers between the two endpoints.

If I recall correctly all you have to do is deploy a cert on the client machine. The client connects to the machine doing the inspection and then it handles the final request under the real cert. They do this in corporate networks sometimes.

The way this works is that the nation state installs a proxy that does a man-in-the-middle attack on the HTTPS traffic. It receives the incoming TLS handshake from the endpoint, and completes it with a certificate rooted in a 'compromised CA'. Then they handshake with the real server, and proxy the traffic.

This allows them to intercept the traffic by shunting the plaintext off of the proxy to another system. Of course, it's plainly obvious to you, the endpoint, that this is going on because have to trust that compromised CA on your device to access the site. They'll, of course, helpfully offer instructions about how to do this and require it to be pre-installed on systems.

But if there's no other way to access any HTTPS services? Lots of people will do it.

It's not misleading because it's mentioned right in the article. The point is that there is more secure data than non-secure data being transmitted.

I just meant calling this a tipping point is misleading. It uses that statistic to paint a picture that's not representative of the reality.

The first line of the article:

> HTTPS adoption has now reached the moment of critical mass where it's gathering enough momentum that it will very shortly become "the norm" rather than the exception

This statistic doesn't logically follow through to that conclusion in any way.

I am hopeful that HTTPS is becoming the norm, but this particular stat just doesn't really give any insight either way.

You could define the "norm" in one of two ways though - the user experience norm, and the server setup norm. If a user sees HTTPS most of the time (seeing the lock in the URL bar, even if just from a few large sights), then that is their norm - they'd expect that all sights would do that.

Maybe, but - despite the constant back and forth by the tech community about the minutiae of address bar security indicator UX - nobody really notices whether they're on a HTTP or HTTPS site.

The new warnings for password fields are a promising step however.

Another tipping point is [1]: Chrome and Firefox now say "not secure" when password and credit card fields are displayed on a plan HTTP page. That's going to push a lot of sites to switch over.

1: https://blog.chromium.org/2016/09/moving-towards-more-secure...

google will also penalise sites with such fields without https in their search results, which is possibly an even more important thing for site owners.

Or they will engineer even less secure password-hiding fields. At least it breaks password managers.

Firefox Nightly has been saying that for almost over 8 months now. I think there were issues regarding the "too many warnings make people blind" stance. But I'm happy it finally made it into release.

What this is, is another accidental slide down the slope towards centralisation of the web. Yes, it's easy for a techie to add a letsencrypt cert, but browsing with https-by-default I still hit a lot of sites that lack https - mostly the sort that run with practically zero maintenance for a non-technical business or community.

Browsers are going to show scary warnings for these (indistinguishable to the average user from any other warning), and then stop loading them altogether, and the sites will die only to be reborn (if at all) on facebook.

The problem there though is not in allowing non-techies to easily install certs, but rather in incentivising cheap hosting companies to provide it by default, with zero configuration. Many companies are already doing this, but they're still in a minority. Making HTTPS "the norm" is hopefully the road to making this an expected feature from those hosts.

> What this is, is another accidental slide down the slope towards centralisation of the web.

Sorry, can you explain? (Not playing "gotcha", I really don't understand.)

He means "a small number of websites by big players" vs "a big number of websites by modest people".

I'm broadly in favor of HTTPS-everywhere, but this does seem likely to happen in a number of cases. Then again, I'm not sure it will be all that impactful in a situation where everybody's already leaving the open web for Facebook anyway...

That's still an improvement. Security is important, and I would prefer to not see sites which are insecure by default, even if it means losing some small minority of sites.

Going forwards I expect we'll see more webservers with Let's Encrypt-by-default, so it's a limited-time problem; it shouldn't affect many people after the initial shakeout.

On NixOS, enabling LE is no harder than putting "services.nginx.virtualHosts.<hostname>.enableACME = true" in my configuration file.

I don't think so. Thanks to Let's Encrypt, I expect that configuring HTTPS will soon be about as simple as "apt-get install apache". Hosting providers will make it a checkbox (that will soon be on by default.) We're making it easier for individual site owners to provide privacy for their visitors, leading to more decentralization.

We are getting closer. A few months back, I pulled the Alexa Top 1M: Of those, ~550K had port 443 open and were speaking HTTPS. Of those, ~270K had valid certificates (not expired, not self signed, matching hostnames, etc).

Planning to do a follow up to get a trend. Also some deeper analysis is needed: Many of the HTTPS sites just 301/301 to HTTP for the same domain, and often to other domains. So while those are websites that speak HTTPS, they aren't really HTTPS sites.

It's slightly better but taking the top 1M is still kind of the opposite of what we're looking for stats wise. I wonder what percentage of the Alexa Bottom 100M are on 443.

The long tail is still going to be large in cumulative terms

I just upgraded our blog site this week to a new Digital Ocean Ubuntu instance, and used LetsEncrypt to install an SSL certificate on it. I couldn't believe just how easy and QUICK it was!

Certainly a far cry from 15 years ago when I used to go through the Spanish Inquisition to get an SSL certificate, not to mention the cost. LetsEncrypt has been a game changer.

So too has AWS, with their Certificate Manager. I've been rolling out their own issued SSL certs to our various Elastic Beanstalk instances as they come up for renewal. Saving a pile of money, but more importantly, TIME, doing this.

For me personally, that is the tipping point - making SSL installs on server a couple of mouse click and less than a minute. Not surprised that it is becoming increasingly popular given this.

IIRC porn sites receive significant traffic and almost all porn sites serve content over http only. The post mentions that https is faster than http and also utilizes resources better than http. Is the statement still valid when streaming video? (Please note that I visit porn sites only to check if it has https enabled or not)

hahaha

-3! Nooo my precious points.

HTTPS is not faster than HTTP.

https://www.troyhunt.com/i-wanna-go-fast-https-massive-speed...

This is by the same author, and there are critics that could be made against it. In some cases it is faster it seems.

HTTPS is basically HTTP, on a different port (443 instead of 80), with TLS encryption on the socket connection. TLS encryption adds overhead, it doesn't reduce it. Therefore HTTPS is slower.

tldr: HTTP/2 with TLS allows for better speeds.

I think what OP was trying to convey in very few words: HTTP2 is much faster than HTTP, offsetting 50-150% (depending on what you're measuring) of the costs of encryption, as it's basically TLS only. For HTTP1, this is obviously false.

In the context of streaming videos TLS encryption overhead is basically the only overhead that matters.

In fact in the dev blog of Youtube they showed it wasn't true. Can't find the post right now.

You are incorrect, TLS overhead is too big to say something like that. Also on large requests, that you have in streaming, overhead of HTTP/1.1 protocol itself becomes negligible.

As someone who is implementing HTTP/2 I would also say that HTTP/2 has no advantage for high-bandwidth requests, e.g. video streaming. I would rather guess that even the unencrypted HTTP/2 variant might be slower than HTTP/1.1 in this scenario, since you now can't directly write into a TCP stream but have to interleave it with multiplexing info and have two flow control mechanisms working on top of each other.

HTTP/2 will be mainly a large win for short requests, where it can avoid the large overhead of creating additional TCP connections and make header exchange more efficient.

You'd be right, except HTTP is often tampered with, so in practice is slower.

That's true. On the other hand sometimes HTTPS connections are blocked.

HTTP/2 is faster than 1.1 though. And IIRC HTTP/2 enforces HTTPS

Troy's title could be argued to be misleading. The perfomance improvement he's discussing is actually HTTP/2 against HTTP.

Typically browsers that support HTTP/2.0 require the use of TLS, so with a little mental gymnastics he's claiming that HTTPS is faster.

Edit: Apologies, I agonised over the wording for so long that 4 other people answered before me.

What kind of mental gymnastics are you talking about? HTTP/2 is faster than HTTP/1. In practice, HTTP/2 requires TLS. Therefore, in practice, to get higher performance, you can achieve better performance with HTTPS than with HTTP.

Sure, but that's like saying "my wheels are faster than those", but they are faster because they only come attached to a ferrari. It's technically true, but it's also not really talking about what makes it faster. It is possible to do HTTP/2 without TLS, which would probably be faster.

Now, in the real world, HTTP/2 is going to mean TLS - which is good, so yes, HTTPS will be faster in that case.

None of what you've said is incorrect. HTTP/2 over TLS is faster than HTTP/1.1 over SSL/TLS.

I referred to Troy's assertion that "HTTPS is faster than HTTP", which is at best an incomplete statement and to some people completely misleading.

Since http/2 is only enabled for https connections in most browsers, sometimes https is faster than http.

I'm glad I'm not the only one it troubles that porn sites don't use https.

Do you pay for it? Maybe they would if people were willing to pay. https costs more resources at scale.

I know eporner.com went https long time ago and even asked opinion from users if everything is working as expected.

I don't know any other porn sites that have gone https, definitely not the big ones that dominate the market such as youporn, porntube etc.

Here's a question: how are captive portals going to present the login site if the user never uses HTTP? I know Android and other mobile OSs probe the network by requesting a HTTP site. On my laptop, I have to load something like `example.org` to get redirected.

I believe we need to work on a standard usable by those currently trying to MITM, which works with browser makers to present a nicer message to users, but maintains the privacy of their connection.

There was some discussion about that dilemma a couple weeks ago https://news.ycombinator.com/item?id=13369038

There's an RFC for a way to add this to DHCP https://tools.ietf.org/html/rfc7710

The solution is for captive portals to go away. Captive portals should absolutely not exist.

iOS already prints a security warning when connecting to such networks, my hope is that Apple will simply have phones disconnect from MitM'ed networks in future, breaking captive portals but making the situation right.

They only really exist where people have no other choice (e.g. hotel). They're often broken, or unusable on mobile devices with tiny UI. Captive portals are an abomination and misuse of technology, not to mention a terrible user experience. And any brand using a captive portal just diminishes my view of the brand.

I totally agree that captive portals suck and we should name and shame anybody using them, especially ones that are poorly built.

But, there has to be some reason that these exist right?

To get rid of captive portals, there needs to be this functionality in the underlying protocols:

- Requiring Terms of Service

- Showing a special site from the service provider (like the page Starbucks takes you to after "Accept and Connect")

- Selecting what kind of connection you want (like at airports and hotels where there's free and paid)

- Login to some system the service provider controls (also airports and hotels w/ paid plans)

- ... and, potentially other things I haven't seen

Edit: Somewhere else in this thread, kelleboo mentioned this [0]

[0] https://tools.ietf.org/html/rfc7710

> To get rid of captive portals, there needs to be this functionality in the underlying protocols

Not really - I don't think implementing legal considerations into technology is the way to go. Mainly because laws vary by country and laws also change independently from technology.

For example, in Germany, EULAs only valid if agreed on at the time of purchase of the goods or service. If presented after purchase, even if you have to click some "I agree" button, they are not valid. (And even then, they can't contradict German consumer law.) I don't believe anybody has sued because of WiFi, but then the question becomes when does service start? When you connect, or when you click accept?

Keep in mind that some devices like Nintendo DS' or Kindles can't use captive portals, better implementations of portals recognise these devices and let them connect.

In reality, what does the ToS really way, anyway? Don't do illegal stuff - but that's illegal with or without ToS. For free WiFi, you only have X amount of traffic - well, it's free WiFi, there's no obligation for for service.

(Put simply, EULAs and ToS' also need to die - but that's a different discussion.)

Billing is more interesting. One argument is it's 2017 - should we really encourage billing for basic WiFi when bandwidth is cheap? One solution could be to give users x amount for free, and only if they use all that use a captive portal. It's not ideal, but at least it doesn't happen right at the start. You could also simply have different SSIDs for paid and free.

> Not really - I don't think implementing legal considerations into technology is the way to go.

I think that's totally fair to say. But just know that the immediate answer from Starbucks or whoever at that point is "Ok, then we'll continue doing what we have to do".

With the ToS/EULA, I think it's really a cover-all protection measure. So, if there's something that the user is doing that is illegal, the service provider is not liable for the user. For example, I go to Starbucks and torrent Star Wars; Disney sues Starbucks for facilitating me or whatever.

But I think your last point is where the argument breaks down. Yes, WiFi should probably be free at this point. But, there are tons of reasons that are valid (even marginally) someone would want to have someone connect to some website before connecting to the rest of internet. Without providing some way to do that, the terrible hacks will continue.

Most likely desktop OSes will start doing the same detection heuristics as mobile OSes.

My Windows 10 laptop does this. As soon as I connect to a captive portal wifi, it opens up my default browser with a sign in window.

There are standard protocols for it, HTTP MITM was always (well, in recent history) just asshattery.

Google keeps a service on HTTP on purpose: https://www.chromium.org/chromium-os/chromiumos-design-docs/...

Try visiting: http://clients3.google.com/generate_204

If my little web page containing a few photos of flowers is never moved to HTTPS, is that bad in any practical sense at all?

Is there any plausible scenario whereby visiting such a basic site without any forms or data collection over HTTP is in some way a disadvantage to HTTPS?

As far as I understand, it leaves your user is vulnerable to man in the middle attacks, injecting malicious code and so on.

Sure - your site is vulnerable to a MitM attack where the attacker abuses the trust your visitors have in your flower site to exploit whatever the latest browser vulnerability of the week is and compromise their computers.

Sure. A MITM can inject javascript into the HTML the user receives from your site (just like the Comcast bandwidth cap example in the article) that causes the browser to open phishing or browser exploit pages.

Ok, fair enough. MITM attacks seems to be the consensus and good enough reason to move the flower page over to HTTPS.

The disadvantage is only that Google will punish your site in its searches.

That was kind of what I was thinking, but several replies above mention MITM attacks, which I thought were extremely rare and non-trivial to execute, but I don't read up on that stuff much.

Extremely trivial and common, see captive portals.

They're not malicious, but could easily be. You can intercept and inject anything you want if you control the network with HTTP. Also, e.g. spoofing an unsecured WiFi network is pretty easy. Even on wired networks, do you know what path your traffic is taking? With HTTPS, it doesn't matter.

An ISP or access point can inject ads in your page content.

An attacker can MitM your visitors, inject JavaScript and use it to DDoS third parties using your visitors browsers.

Yes. Somebody can change in transit your flower site to do data collection, or even to install malware. When you visit a site over HTTP you can never be sure you are really getting the site you wanted to visit.

There have been many recent cases of ISPs inserting advertisement on sites like yours. From there to malware it just takes a more ill intentioned actor.

Thanks, I suppose then the question is how rare these MITM attacks are. Would be interested to know stats and also growth of the attacks... like are they on the increase.

ISPs hijacking HTTP sites is very common. That DDoS some other comment was talking about was used by China's gov to attack Github just about an year ago.

I don't think script kiddie level attacks happen very often, because there are many easier avenues if you want to mass attack people. But MITM attacks can be very valuable for targeted attacks, because with it one can corrupt peers that your target trusts.

Do you actually have that web page with photos of flowers? Now I want to see them!

I do have a flowers page, although I was only using 'flowers' as an example of something very simple served from http, I wasn't referring to my page.

My flowers are actually hosted on Amazon so it's already automatically both http and https. I don't generally share my photos too widely, but since you asked I'll share my flowers...

https://ozimg.s3.amazonaws.com/macro-flowers/index.htm

Oh, wow!

What is the situation with available Adsense ads? Do I still have to except lower inventories of HTTPS enabled ads and thereby less income if I switch to HTTPS? I understand that this depends on the target audience. Did anybody experienced a drop at the beginning and later recovering?

I'll believe we've hit the tipping point when I type "example.com" into the browser address bar and it takes me to https://example.com rather than http://example.com.

For us this came with cPanel started including it by default. All you needed to do was a few lines in your .htaccess to forward all http requests to their equivalent https ones. Almost all our sites except for a few edge cases are switched over now.

It's definitely hit a tipping point for me. I installed HTTP Nowhere a couple of weeks ago and hardly ever run into a site I can't at least load up in the Internet Archive!

Services like cloudflare make it dead-easy to set up https. The more one-click https setups there are the faster we'll vault right over the tipping point.

Sorry but I disagree.

Please install PanicMode browser extension for Chrome (https://chrome.google.com/webstore/detail/panic-mode/lamdafc...) and try to surf the web for a day. You will know what I mean as soon as you go through this experiment.

Disclaimer: I am the author of this extension. I wrote the extension for personal reasons and it is very simplistic in nature. If you turn PanicMode on it will replace every outgoing http:// url with https://. It does nothing else besides that and unlike HTTPS Everywhere it has no exceptions list or special handling of the top 100 sites, etc. The site you are visiting should either support https:// or it will blow up in your face, which is exactly what happens 90% of the time.

Edit: Besides just because Firefox is seeing more HTTPS traffic means nothing if all the traffic comes from Facebook, Google, YouTube and a few others. Yes, there is more traffic and yes it is encrypted but does it really say anything about the state of the web? Someone needs to put this data out to make it clear.

"HTTP Nowhere" is a Firefox extension that blocks all non-HTTPS requests. It doesn't try to rewrite HTTP requests to HTTPS, though.

https://addons.mozilla.org/en-US/firefox/addon/http-nowhere/

I can easily configure NoScript to only use HTTPS on, say, ¤.cn . Sites that downgrade-redirects to http:// gets stuck in a loop. Annoying for everyone, so it raises awareness.

The prefs.js syntax is, noscript.httpsForced "¤.cn\n¤.ru\n*.uk"

Edits: ¤ characters are asterisks.

Reply on news.ycombinator.com