Delaying Further Symantec TLS Certificate Distrust | Hacker News Reader

Delaying Further Symantec TLS Certificate Distrust

blog.mozilla.org

111 pointslainon56 comments

If you look at the de-facto vulnerability disclosure standards where a company is contacted with details of a vulnerability and a timeline in which to fix it privately before the security researcher goes public, you'll see that a hard stance gets things fixed properly. How many times has a company ignored the timeline, only to have the vulnerability fixed within hours of of it going public? These companies are capable of fixing this cert issue, and are being lazy. They have been warned, and have been given a generous deadline. Zero sympathy for missing it and suffering the consequences. Empty consequences will only teach them to ignore future problems.

I don't think comparing this decision with a vulnerability timeline is very fair. But besides that, there are other motivations at work. I believe the core of this post is summed up by this one paragraph:

> we believe that delaying the release of this change until later this year when more sites have replaced their Symantec TLS certificates is in the overall best interest of our users.

They do not want to mess up the user experience. Which I can sympathize with, especially for a browser that does not have market share. Chrome on the other hand:

> Around the week of October 23, 2018, Chrome 70 will be released, which will fully remove trust in Symantec’s old infrastructure and all of the certificates it has issued. [0]

From my point of view this boils down to "We're going to let websites fail for Chrome first, which will get greater visibility, and then we'll follows suit once things have calmed down". If this is the true motivation, then I believe this is a wise decision as Chrome failures will get far more attention and action vs. Firefox failures.

Edit: Just to add some numbers. Chrome is currently at 60% usage while Firefox is at 5%. That is a significant difference in error visibility.

[0] https://security.googleblog.com/2017/09/chromes-plan-to-dist...

A coordinated stance would also be helpful, release both at the same time.

The Firefox release was already due to happen a few days after the Chrome one (this was just due to how the release cycles work out, rather than deliberate planning).

>while Firefox is at 5%

That is a real shame and I sincerely hope they are able to return to a health 25-33% mindshare.

For a long time Chrome was objectively the best choice, because it simply had better performance on almost everything. With the recent changes to Firefox (Quantum), that is no longer true. I have good hope this means that it'll climb back to a bigger percentage.

While its usage is certainly low, I also believe that its usage is under-reported at 5%. Many Firefox users today are privacy conscious and use plugins that block the sites that collect this data. Firefox even has built-in tracking blocker.

Burning people who could have just stuck with http and avoided the fuss will teach them a different lesson.

I didn't expect that from a dev of the "secure by default" OS.

Also, the main offenders aren't the small players, we're talking about the top 1 million. Sure any lambda person could use http if they wanted - but would they make it to the top 1 million?

Is the top 1 million so low a bar that it also contains one-person-orchestra that have no idea how to handle their site?...

So these 1% sites make Mozilla's certificate revocation plan 'Too big to fail list'? Sounds like a bad security plan. You are only as strong as your weakest link.

Why is TLS 1.0 and 1.1 still enabled by default in Mozilla? More of the same. Big players don't want to tighten up.

If I had to guess, I'd say TLS probably isn't the weakest link...

You ever wonder what's happening on the back end of some of these sites running with ancient certs?

They're not that ancient, a typical leaf certificate affected by this issue will have been issued in July 2015 or later (maximum certificate lifetime of 39 months, now it's 825 days)

Mozilla, as far as I know, isn't beholden to any consumer sites for revenue?

Instead of a generic "Something went wrong" TLS error message that's Greek to normal users and potentially causes blame to the browser, get aggressive.

At this point, it's laziness or incompetence.

"Warning: [WHOIS_COMPANY_NAME] is using a vulnerable certificate from Symantec. Criminals may be able to read anything you do on this site.

Please email [WHO_ADMIN_CONTACT_EMAIL] about securing their site. Click [here] for more information about why Symantec certificates are no longer trusted."

The actual Firefox message for a site using a Symantec cert on today’s nightly:

Warning: Potential Security Risk Ahead

Nightly detected a potential security threat and did not continue to [site].com. If you visit this site, attackers could try to steal information like your passwords, emails, or credit card details.

Websites prove their identity via certificates, which are issued by certificate authorities. Most browsers will no longer trust Symantec, the certificate authority for [site].com.

You may notify the website’s administrator about this problem.

I'd much prefer the browser continue to the site, but clear all cookies, treat the domain as an untrusted origin (so all CORS requests fail), and as soon as the user tries to type anything, pop up a big security warning saying "This site isn't secure! You shouldn't put any private data, passwords, or credit card numbers in". Then disable the keyboard entirely.

Displaying the site but mangling functionality runs the risk of shifting blame to the browser, which I think is the primary tightrope in their mind.

"Firefox won't let me do this" vs "ING's page is broken"

> Why is TLS 1.0 and 1.1 still enabled by default in Mozilla?

Because PCI DSS still didn't obsolete TLS 1.1, probably.

https://blog.pcisecuritystandards.org/are-you-ready-for-30-j...

One can always go to about:config and set security.pki.distrust_ca_policy to "2" to distrust Symantec TLS certificates. Reference: https://blog.mozilla.org/security/2018/07/30/update-on-the-d...

They are distrusted i. Nightly

"our latest data shows well over 1% of the top 1-million websites are still using a Symantec certificate that will be distrusted"

Is there a list somewhere? I'm curious to see which organizations missed what happened with Symantec.

Edit: Answering my own question. Found a list: https://scotthelme.co.uk/the-final-symantec-distrust-is-comi... There are a few I recognize, like solidworks.com, ferrari.com, and quite a lot of .gov. sites.

None of these businesses take their PKI seriously. They ought to be taught a lesson the hard way.

It's not that simple: many don't have the technical preparation or still don't know about this in spite of everything.

Rather than "teaching a lesson the hard way", but also rather than just delaying the distrust passively, I would introduce a visual warning in the lock icon, rather than in the developer console.

There are a few big banks on the list that don't have much excuse for not being on top of this.

<devil's advocate> Just because many people might not have "the technical preparation" or "they don't know about" driver's licenses, that doesn't mean society would give them a free pass for driving unlicensed and without the required training/testing... </devil's advocate>

While _some_ of those 1000+ sites may just be brochureware, I'd bet reasonable money that the majority of them are collecting user data of some sort (even if it's just email/password for site logins) - and the reality of website breaches and credential dumps in 2018 - those 1000 website owners are doing the Personally Identifying Information equivalent of unlicensed and uninsured driving.

I'd stop somewhat short of suggesting we should put the people responsible in jail, but if you run a top million website and have no clue your ssl cert is dodgy - I'd probably support calls to revoke your right to run a public website...

The lesson learned by many people if you don't delay is going to be that warnings about certificates can be ignored, making them useless.

Crying wolf about the security of sites too slow to upgrade their certificates isn't telling anyone accurately that they are in danger, it's just attempting to punish those who are slow. From the outside it just looks like political bickering and for people who don't care it lowers how much they care at all about anything being said about certificate security.

> However, given the current situation, we believe that delaying the release of this change until later this year when more sites have replaced their Symantec TLS certificates is in the overall best interest of our users.

If they haven't already, what reason is there to believe this delay will make them? Just curious if there is another catalyst that will push them beyond cert expiration.

Chrome will be distrusting soon too, and they have 60% browser market share. That will get far greater visibility then Firefox's 5%

Brand impact: If you think you use SSL/TLS certificates from "Thawte", "GeoTrust" or "Verisign" those are all Symantec. You need to pay attention and go read instructions from your reseller or issuer to find out what you need to do. Do it today.

In a few cases your systems might desperately care about those magic brand names, and DigiCert (who now own all these brands) can sort you out if this is the case. In most cases you don't need to worry, just follow instructions.

I'd love to know what you mean by caring about the magic names and what DigiCert can do there?

My worst case hypothetical is clients in retail pinned to Symantec root, upgrade url is https (pinned) and hostname matches your consumer facing website.

Sure, so here's an example. You've got this cert for api.example.com that's got "Symantec" branding on it.

So it's issued by "Symantec Class 3 Secure Server CA - G4" and that cert chains back to a root named "VeriSign Class 3 Public Primary Certification Authority - G3"

Well all that stuff is tainted so it's going away, right? Yup. But maybe your client checks for a pin to that "VeriSign Class 3 PPCA - G3" stuff so you need that!

DigiCert have your back, since it's being distrusted anyway there's no problem having their "DigiCert Transition Root G3" signed by that VeriSign root and that can issue its own Intermediate which they can keep online to sign for you.

So now the certificates aren't issued by the tainted Symantec systems, but your client trusts them.

They also have a bunch of brand new intermediates with the old branding, for cases where you need the branding to match (Yes, I know, that's cosmetic, but eh...) but the cryptography is all fresh.

You definitely should talk to DigiCert people about the specifics of anything you need, rather than take advice from random people on Hacker News like me, but for most scenarios they will either be able to advise you on how to transition safely to something that stays trusted in the Web PKI, or how to transition to something that's maintainable off the Web PKI.

The scenario they can't help with is if you need stuff directly (not via an intermediate) signed by old Symantec roots. That wasn't even supposed to be happening when Symantec existed, and in fact one of the smaller items taken into consideration when deciding to distrust Symantec was that they weren't able to resist plaintive cries (doubtless accompanied by hard cash) by banking customers who asked for this to be done even though it was forbidden.

Oh I see --- these are new intermediates that were allowed to be issued. That makes sense but feels wonky because these intermediates were signed after the announced distrusting. I didn't realize that browsers were going to be ok with the old CA(s) signing a couple more intermediates.

I don't know the details of this specific situation, but it's possible to have two signatures on an intermediate CA. One by the old Symantec root, and the other by a trusted root. Up-to-date clients will ignore the Symantec chain and only follow the trusted chain.

For example, Let's Encrypt maintains compatibility with old browsers by having IdenTrust (a widely trusted root) cross-sign their intermediate CAs.

>We prioritize the safety of our users

>However, given the current situation

"We march North!" says man marching South.

Is Chrome delaying too? M70 is supposed to contain the distrust change, and last I heard it's scheduled for an October 16 release.

They are not. I’m using Beta and received a certificate warning so I contacted the webmaster. They claimed it must be my computer though I was descriptive of the issue and cause, they then said they would look into it as they hadn’t heard about it. I forwarded them the article and I guess we’ll see.

Maybe Mozilla is hoping to get some people to try Firefox when Chrome does not want to visit their favorite website.

The error message in Chrome is loud and clear that it's a problem with the site using Symantec certificates. I'm sure Firefox wants to make the sites more visible (with Chrome dropping Symantec first), so there is more push from users to the web site.

It’s not clear for non-technical users. The error message is something like Symantec Cert Error and you can click a more info link where there is another link to get detailed info.

Even if it is clear, people will want to visit the site and if they don't find the button in Chrome or they don't want to bother with the nag screen (you can't "continue and remember" in Chrome)...

Oh no. I was expecting Mozilla to follow through and then use the fallout as a way to bring attention to the issue and use it as a marketing item for Firefox as a "more secure" browser.

The problem is that when this happens, users are going to blame the browser, not the site, because it "used to work fine". Especially when switching browsers is easy.

I ran into this issue while trying to visit Paypal on my Pixelbook (on dev channel). There was no "simple" workaround (ie. click to ignore the warning) so I just used a different device.

To test Chrome with the new behavior, launch with the following command line flags:

  $ google-chrome --flag-switches-begin --enable-features=LegacySymantecPKI --flag-switches-end

Feature flags are global, so make sure that you are launching a new instance of chrome and not opening a new window in an existing instance. Note that a domain's certificate validation status is cached, so you may need to clear history or use incognito mode to test the new business logic.

If you need to re-enable the old behavior after version M70 is released, use the following command line flags:

  $ google-chrome --flag-switches-begin --disable-features=LegacySymantecPKI --flag-switches-end

These command line flags are undocumented and may change at any time.

> We prioritize the safety of our users and recognize the additional risk caused by a delay in the implementation of the distrust plan. However, given the current situation, we believe that delaying the release of this change until later this year when more sites have replaced their Symantec TLS certificates is in the overall best interest of our users.

It would be nice if the sites were mentioned and contact information posted.

It would be nicer if that information was mentioned directly to the end-user to encourage end-user action toward the site.

And, is it not possible that the sites are fraudulent to begin with?

It is conceivable but unlikely, if you mean in the sense that the certificates should not have been issued.

The broad issue with Symantec was the lack of oversight delivered by its board and senior executives. People with the power to Make Shit Happen at Symantec either didn't know about, didn't care about, or actively permitted practices that were either forbidden, or which weren't specifically prohibited but a reasonable person should have known were unreasonable.

This is not like DigiNotar, where we knew actual bad guys had either the keys or something morally equivalent to the keys (e.g. root passwords to issuance systems).

It's not even like StartCom/ WoSign, where they were doing naughty things and trying to hide that so they wouldn't get discovered, but they fundamentally retained control of their systems, it's just that we didn't trust them.

But nevertheless this could not be allowed to continue.

Distrust was the mechanism by which it was made certain that it couldn't continue. However, Symantec decided to exit the business in 2017, and as a result their management practices are now largely irrelevant.

In theory we could imagine some James Bond style villain copying their old root keys and, before they can be finally distrusted, using them to enact some ludicrous over-the-top plan. Stealing a trillion dollars? Starting a war between the US and Russia? Something like that. But this is not a James Bond movie, most likely everything proceeded as intended and the final distrust is merely a precaution.

I've seen this issue in Firefox Nightly when trying to perform the HSBC UK credit card verification, so it makes sense not to roll it out to the wider public yet.

Or the opposite, that people will unknowingly transmit data over untrusted connections without this.

They can't. With most banks and even PayPal, their site is secured by HSTS which renders their site completely unusable with no way to get past the warning.

Your only option is to use a _different_ browser which trusts the old certificate.

> no way to get past the warning.

You are right, but there's always a chance that the various documented HSTS bypasses might filter down to normal people. People might be willing to find and use them if they /really/ need access to the site.

e.g. Typing "thisisunsafe" in Chrome on the error screen, or flushing the HSTS state in the browser.

PayPal just switched to a DigiCert certificate and should be accessible now.

Or to switch to a bank that actually cares about security?

link timing out for me, from archive.org:

https://web.archive.org/web/20181010180016/https://blog.mozi...

Thameswater in the UK still hasn’t rotated their certificates.

This actually shouldn't be something that a company that is giving away a product should even be enforcing or deciding on their own. Reason? It makes the assumption that end users are not willing to accept that in rare cases their traffic can be intercepted and it further assumes that it matters at all if that traffic is even intercepted. And most importantly it fails to recognize the impact of making a unilateral decision like this has on those who are not impacted at all. Like some random small site (not a top 1m site) that is just providing info and now either has to pay someone to fix and/or can't have a visitor to view their site.

Even in the case of credit card information (or other sensitive data) I would love to know exactly what the probability is of info being stolen. Not hypothetically. But the actual risk.

This idea of 'security at all costs' without looking at the actual cost and/or implementation is not realistic at all and can do more harm (some site not being accessible) than good.

Let's Encrypt certs are free. Most companies that sold Symantec certs that are being distrusted are offering free replacements. Other trusted certs can be purchased at minimal cost from a variety of vendors. There may be a cost associated with replacing the cert on the server if nobody in your organization has the (very limited) technical skills needed to pull it off yourselves. If you can't deal with a minor IT expenditure with over 6 months' notice, your business has bigger problems.

Business? Not all websites are for business or run by business people. Yet they provide info on the web.

Also Let's Encrypt (I am familiar and we use that btw) is great for certain types of situations but not all situations and doesn't work with all control panels in certain cases. For example using the plesk control panel you can't map a domain to a directory or sub page on a site. You can of course add it as an alias to the main page but not say /thisdirectory/index.html at least not as of last week.

Also installing a cert is non-trivial for most users and takes time and work. My issue is not with the people that care to do that or feel it's necessary. It's with this idea that 'everyone should do it period' being decided by the browser vendors. And in particular the way an error message is displayed and shocks a typical user not every gently at all. That is the issue that stops people cold in their track. And scares them.

Browser vendors have previously expressed their trust in the Symantec ecosystem. After Symantec repeatedly failed to hold up their promises, the vendors are now revoking the trust they had placed in Symantec ecosystem. Browser vendors can no longer tell their users they are connecting to a secured and validated website because of this.

To accommodate Symantec customers, browser vendors have given people (a lot of) time to change certificates, trusting the certs even though the party that signed them was no longer trusted. This placed users at somewhat of a risk, but made possible a grace period.

Now that grace period is nearly over. The companies at fault, who gave out unverifiable certificates, have not contacted all of their customers or their customers have not acted on their messages. Either way, this is not the problem of web browser vendors. End users trust their browsers to warn them if their safe connection is fishy and browsers will soon actually show those warnings.

If you disagree and want to ignore the recommendations of Google and Mozilla, feel free to disable the Symantec distrust flag or to use a different browser that still accepts the bad certificates. That is your choice and the choice of any end user. However, those who don't understand the underlying problem continue to rely on the safety their browser software provides, as they should. If a small website can no longer display simple information because of Symantic not being trustworthy, that website should take it up with Symantec. And it's not as if the certificate wouldn't have to be replaced within about a year from now anyway; there's no extra costs or effort required, just earlier costs.

Let's not forget that it is the web browsers that gave Symantec a business model to take away in the first place. If from the beginning non-commercial key distributions would have been set up (public keys over DNSSEC or something like that) this would not be a problem and the whole HTTPS certificate market wouldn't even exist. Sure, X509 certificates are used for other usages as well, but not as prominently as for web browsing.

Reply on news.ycombinator.com