konklone

This is fantastic news! And they decided to include the Chromium HSTS Preload list too. I've submitted something like ~30 domains to the list, and having them covered in newer versions of IE automatically is a very nice thing.

I tell this story all the time to people. It's so magically great, and the developers are a national treasure.

I think of 18F and USDS in part as pieces of an effort to make this kind of result less of an anomaly, and the kind of thing you generally expect your government to do. Cause after all, why shouldn't you?

(Disclosure: I work for 18F.)

You'd be surprised how much of a difference a single person, engineer or not, can make on the inside, with enough persistence and vision.

If you care about policy, you put in the time, and you know what you're talking about -- you'll make your mark.

https://www.federalregister.gov is a favorite of mine in the federal government, and an inspiration (they launched with a full JSON REST API in 2010!) -- but just so it's clear, that's not an 18F project.

The code for federalregister.gov can be found here: https://github.com/usnationalarchives/fr2

(Disclosure: I work for 18F.)

It's a product of 18F (of which I'm an employee). The US Digital Service folks are awesome, but a totally separate unit. 18F is in an independent administrative agency, the GSA, the Digital Service is in the White House.

For some extra clarification: this is the official list, as released by the General Services Administration, the part of the government that runs the registry.

Some of these -- the domains for the federal government's executive branch -- were already public. This includes the rest of the federal government (e.g. Congress, the courts), as well as all the .gov's in states, territories, counties, cities, and native tribes.

There's about 5,300 of them.

Yes. Not just inside government, either.

Er, so I take this back? Ilya Grigorik rang in and told me I wasn't distinguishing between request and response headers:

https://github.com/18F/tls-standards/issues/24

Sadly, we still do. Requiring SNI remains a tough sell. That's one of the biggest limiters in our HTTPS setup on saving costs and being better stewards of IPv4 space.

> By demonstrating by example and by explanation of what's already possible in current rules, 18F can boost the stock of the buried geeks hiding out in many of our government agencies.

You nailed it.

> You claim the second most important reason for using HTTPS is my privacy, yet you share all pages I visit with Google analytics.

I think that's a totally fair point. Our use of Google Analytics isn't changing any time soon, but we do plan to add a third-party disclosure page that makes it clear what third parties have some window into our visitors' browsing:

https://github.com/18F/18f.gsa.gov/issues/293

This includes otherwise invisible things, like our host, Amazon, and (until we implement OCSP stapling, which is happening soon) our CA during revocation checking (in some browsers).

FWIW, we do turn on the Google Analytics anonymization flag, which instructs Google to chop off the last IP triplet before they write the data into their database. Of course, that depends on trusting Google to keep their promise, but it's something.

> for very high traffic websites HTTPS costs a lot of money. .gov sites are not very high traffic.

.gov sites can be very high traffic. Think weather, social security, health care, immigration, visas...

> - When you have such high traffic you also have a bunch of old winxp and similar clients. These dont work fast on HTTPS and dont work without sslv3 and what not.

SSLv3 only kills IE6, which is doable. And it's okay if very old clients work slowly with HTTPS -- those clients have far more problems than just slow HTTPS.

> Exactly what the American people have come to expect from the government. Unless its been gift wrapped by a contractor they lack any ability to do anything technical.

Exactly the expectations that 18F would like to change.

Every site is going to have a different sort of userbase, so it's tough to make sweeping proclamations about support. I don't think we'll have trouble using SHA-2 in our deployments.

> Those aren't the only downsides. The other downside is dramatically increased first byte latency due to the handshake.

The latency increased, but it doesn't have to be drastic. Ilya Grigorik has a great piece on how to get the handshake down:

https://www.igvita.com/2013/12/16/optimizing-nginx-tls-time-...

We turn on as much of that stuff as we can for our servers, and I'm pretty happy with it.

Exactly - and that's why we have SPDY header compression disabled on our nginx config:

https://github.com/18F/tls-standards/blob/master/configurati...

When SPDY 4 or HTTP/2 make their way into an nginx module, we'll upgrade, and turn header compression back on.

> Do you have to encourage your colleagues to use it? I've enabled HTTPS on all our sites yet when the links are distributed they are the typical HTTP.

Yeah, sometimes. But that's also why we enable HSTS on our stuff, so that that's not relevant. We also just pushed the first .gov domains into the Chrome HSTS preload list a week and a half ago:

https://chromium.googlesource.com/chromium/src/+/af1870543d1...

Very glad to help. :D For those reading this, the best URL is https://github.com/18F/tls-standards/blob/master/configurati..., which is heavily based on the nginx config I maintain personally at https://gist.github.com/konklone/6532544

Thanks! But actually, it pains me that we're still showing a SHA-1 cert.

Because bureaucracy, we're still waiting on a replacement SHA-2 cert, which once installed will bring our score up to an A+ like this:

https://www.ssllabs.com/ssltest/analyze.html?d=staging.18f.u...

We're changing up processes internally that will speed things up. The subject of a future post!

EDIT: And yeah, we're aware of the notalone.gov issues. There's some legacy infrastructure there, but we'll try to get it fixed up. Our future sites will be stronger.

> Considering that latey there's been a pretty strong shift to using HTTPs everywhere, I don't see how using it on .gov site needs an explanation :)

If only the .gov space were shifting at the same rate! That sense of inevitability and momentum you sense in the private sector is not so widely present inside the US government. Posts like this have two audiences: outside folks like you, and my colleagues around the rest of the .gov world.

For reference, 18F got the first .gov domains added to the Chrome HSTS preload list less than 2 weeks ago:

https://chromium.googlesource.com/chromium/src/+/af1870543d1...

Right on, but tiny correction - 12.04 has over two years of life left, not three.

@MarcBodnick - I do understand this perspective. But I strongly disagree with its tradeoffs.

Quora should find other ways to protect its users, and/or make users aware of their own rights to get their old contributions taken down. The Archive respects takedown requests.

> The post is nothing more than an attempt to shame Quora into opening up their data.

My post is definitely an attempt to shame Quora into opening up their data, in at least the sense of making it available to the Internet Archive. No bones there.

> There are many people that don't want everything they post on the internet going into permanent and searchable databases.

We may just disagree to some extent on what the norms should be, but I think if you're intentionally posting public content to a public website, that's part of the permanent public record. Especially when that website is about accumulating a knowledge base.

Wikipedia, another knowledge base, records everything. Though unlike Quora, you're allowed to contribute fully anonymously (without even registering an account -- in fact, come in through Tor, if you like). They have no problem allowing themselves to be backed up on the Archive, and I'd be pretty worried if they did.

In fact, Wikipedia's robots.txt is really interesting: https://en.wikipedia.org/robots.txt

There's some brief bot exclusions, a brief, now commented-out section asking the Internet Archive not to archive user pages, and then a very, very long section blocking various pages from being indexed by anyone. That long section has a lot of thought and history in it, including notes about the Internet's memory about users, like "Folks get annoyed when XfD discussions end up the number 1 google hit for their name."

I think it's totally fair to argue with Wikipedia about the choices it makes in its robots.txt, but ultimately what we're talking about here are organizations making these choices on behalf of users, not the individual users themselves.

If individuals are concerned about their contributions being preserved, that should be something they take up with the Archive. The Archive respects take down requests, both because copyright is a thing and because they're not interested in harming individuals.

I don't think we're working in the service of humanity by blessing companies that gate the future's access to massive troves of knowledge that was freely contributed to public websites.

> What are some ideas about what Quora's long-term business model might end up being?

There's some good discussion on that on, how about that, Quora: https://www.quora.com/What-could-Quoras-long-term-business-p...

> Hey, I'm the random guy that asked you to answer on Quora. How I found you isn't very mysterious, I just asked all the folks listed here: http://18f.gsa.gov if I could easily identify a Quora profile.

And hopefully it's clear in my post that I'm really glad you did! :) It really is a good artifact -- which is exactly why I get so mad that it's not being preserved the way it should be.

> and if someone -- including the Internet Archive! -- reproduces it without your permission, you are legally entitled to ask them to stop. The Internet Archive won't even make you involve a lawyer.

That's right.

And this is important, because the Archive, like everyone else, is subject to copyright law. What Quora is asking for is something more: an API that allows them to go in, without any sort of human process or judgment, and sync up the Archive's records with what Quora wants those records to be. That's much different, and much worse.

> And their mission is certainly not to "share and grow the world's knowledge".

Well, I obviously agree that it's not their actual mission as executed. But I'm putting it in quotes because I copied it from Quora's official About page: https://www.quora.com/about

I could have said "practically unique", instead of "in theory is unique" there, but I make the distinction more clear just below that and note that there are always collisions out there.

In fact, Heartbleed helped a lot: http://news.netcraft.com/archives/2014/05/05/sha-2-very-cryp... But there's a long way to go.

I have, it's one of the links in there. I truly loved reading that discussion.