Analysis of the Alexa Top 1M sites | Hacker News Reader

Analysis of the Alexa Top 1M sites

blog.mozilla.org

143 pointsokket15 comments

I'm an appsec person and security person. I'm surprised about the hundreds of people talking about CSP, HSTS, HPKP and it feels like I'm the only one talking about insecurity headers.

Within the Alexa Top 1MM I discovered about ~1000 sites that had turned SAMEORIGIN policy off with their CORS configs [https://ejj.io/cost-of-security-headers/] (including Atlassian/Bitbucket, Zulily, and several other big ones).

I asked Scott Helme to add looking for these problematic configs to securityheaders.io but didn't have any luck. I would love some help getting traction on this issue. CORS is fundamentally ... not good and people will keep getting it wrong unless people do something.

I'm actually just coming across this at work. I work for a large media company and I'm petitioning in this capex round to build assistant (alexa, google assistant, siri, snips, etc) apps for the different brands including radio.

The radio brands have streams that pass through Akamai into html and other sdk players that remove CORS protections, sometimes as the only way they function.

Can you list a few good resources for dealing with these situations, or point a guy in the right direction?

I am probably the right person to help having a ton of cdn experience, but I don't fully understand the situation.

Feel free to email me at evan at segment dot com

Thanks, I'll likely reach out if I can get the projects approved and initiated.

Hey Evan. CORS is on my todo list for Hardenize and will be added. We've done some work already, but it's hidden for now until I am certain our assessments are correct. Happy to work with you if you're interested. You have my email address!

Making a change to secure a site further is fairly invisible: you make the change, and hopefully nothing breaks. If you're the type of company to blog about your tech stack, you might make a post on it, but otherwise you just hope nothing goes wrong. And while a major security breach will get you in the news, an army of angry customers will hurt you too.

So, it's not surprising that the features that have gotten the most adoption (like HTTPS itself) are those that show up visibly in browsers, get direct publicity, or unlock access to other things. Features like CSP, while quite useful, don't make the news when you adopt them, and don't unlock any key functionality.

That's exactly right and it's a useful lesson. Browser vendors and internet infrastructure engineers need to do more than simply make security tools available - the tools need to be publicised and visible too or they won't be adopted. Sites need to actually break if they're not using necessary security options, or at least show a warning to the user.

Perhaps, just as browsers have started to take steps to show insecure HTTP as actually insecure, rather than just an absence of security indications, it might make sense to introduce such measures for other features as well. For instance, perhaps browsers could phase in requirements for the most critical security features (e.g. secureonly cookies) to be marked as secure at all, and some of the features (such as CSP) to be marked as EV.

Independent and unrelated to Observatory project, but if anyone is curious to dig deeper into Alexa top ~500K and how it changed over time, take a look at the HTTP Archive dataset. Some examples @ https://discuss.httparchive.org/latest.

E.g, to identify sites with CSP: https://discuss.httparchive.org/t/which-sites-have-content-s....

You can access raw HAR captures, as well as pre-aggregated stats via BigQuery.

How representative is Alexa data? Are people still installing dubious toolbars in 2017?

It's a fundamentally flawed metric and not really indicative of the top 1 million sites or anything else, all it shows is people who have that cr*p toolbar installed, and that's usually just actual website owners who should know better.

Implementing above the fold optimizations is not nice for a frontend noob like me when using CSP. You can use a nonce or sha256 for style and script, but not for inline event handlers (onload="..."). Which means for now I have still a 'unsafe-inline' in my headers (which also works as a fallback for browser that don't support CSP2, and will be ignored by browsers that support nonce-/sha256). But it seems there will be a way in CSP3.

Ideally there would be a way to get async loading css without onload. Is putting the style tag just before </body> acceptable? It's not valid html if I remember correctly.

Since I can't edit anymore: I meant the link tag before </body>, not the style tag.

Is the data available anywhere?

Wow, I thought I knew a few things about web security but it turns out I have a lot to learn.

Reply on news.ycombinator.com