HTTP Immutable Responses | Hacker News Reader

HTTP Immutable Responses

tools.ietf.org

150 pointsokket89 comments

For more on the rationale behind this feature, see:

https://www.ietf.org/mail-archive/web/httpbisa/current/msg25...

https://bitsup.blogspot.co.uk/2016/05/cache-control-immutabl...

Rough summary:

> At Facebook, ... we've noticed that despite our nearly infinite expiration dates we see 10-20% of requests (depending on browser) for static resource being conditional revalidation. We believe this happens because UAs perform revalidation of requests if a user refreshes the page.

> A user who refreshes their Facebook page isn't looking for new versions of our _javascript_. Really they want updated content from our site. However UAs refresh all subresoruces of a page when the user refreshes a web page. This is designed to serve cases such as a weather site that says <img src="" ...

> Without an additional header, web sites are unable to control UA's behavior when the user uses the refresh button. UA's are rightfully hesitant in any solution that alters the long standing semantics of the refresh button (for example, not refreshing subresources).

What's the hurry to optimize away the revalidation requests when the user clicks reload? Is it just beancounter mindset about saving a few "304 Not modified" responses? In that case they shouldn't count the percentage of requests, but percentage of bandwidth or CPU seconds. Tiny responses are much cheaper with HTTP/2, so be sure to benchmark with that.

I'd be happier if my browser made fewer requests, however small they may be. Even if they make up a tiny percentage of my internet traffic, it all adds up. Making computers do less unnecessary work is a good thing.

People are notoriously poor at predicting what increases their happiness; see Daniel Gilbert's book[1].

[1] https://en.wikipedia.org/wiki/Stumbling_on_Happiness

At Facebook scale, the sum of all those "304 Not Modified" responses is probably a significant amount of resources.

I'm not sure it's a good argument to take up the biggest companies and then tally up effects of a micro improvement. You could argue for all kinds of complexity increasing changes resulting in %0.01 efficiency improvements this way.

At my company, 304s account for 3% of our CDN requests.

304 requests are so tiny that you probably end up in the order of 0.01%.

Very few entities operate at "facebook scale".

But they still account for a lot of the Web traffic.

Think dropped requests. If you're operating with 99% packet loss like many people around the world minimizing the absolute number of requests can have dramatic impact on load times.

You can't do anything with TCP or the web in a situation like that.

What do you mean? I've operated at 99% packet loss plenty of times in the rural parts of Vietnam. The key is you can't use any of the websites or apps that are popular in North America / Europe.

The likelihood of even opening a TCP connection is going to be very low. For phase 1, stacks typically send 5 SYNs before giving up. So 95% of your TCP connection estabilishments will slowly fail due to timeout before you even get to phase 1 of the 3-stage connection estabilishment. To say nothing of actually transmitting or receiving any payload data successfully. Which you would have to many times to open a web page.

I understand their rationale for it, but I don't think it should be implemented. Having something immutable like this allows it to end up being used for tracking purposes. Just add a <script src="trackingcookie.js" /> that calls a function with the cookie and all the sudden there's yet another place to covertly store an ID for tracking a user.

But you're removing HTTP requests, so how can it make tracking easier? Any ID that a company puts in their immutable content can also be put in their normal content. The change doesn't make tracking any easier than it already is.

It makes it another place that they can store the tracking cookie and have your browser give it back out to them. Deleting the cookies or other such things wouldn't remove the tracking ID as long as the browser continues to use the cached immutable resource. Similar to how this[1] works by using multiple storage methods.

https://github.com/samyk/evercookie

I think a better way to handle this would be to use subresource integrity [1]. Then, if the browser's cached version matches the hash, it can be sure that it doesn't need to do any requests.

[1] https://developer.mozilla.org/en-US/docs/Web/Security/

It would be far better to have user agents handle caching headers correctly instead of creating another configuration option (which will likely suffer from the same implementation problems).

cache-control: private with either sliding or concrete expiration time already handles this.

cache-control: private doesn't seem to imply that a resource won't ever change and on page refresh the browsers have to check if the resource has been updated, immutable would avoid the 304's responses cascade.

That's the entire point of the expiration time. Use a 2 year range and it's effectively immutable. No content will stay on device forever anyway and headers can easily be set to a smaller time-frame or must-revalidate if the content owner wants it.

Browsers mistakenly continue checking for new copies when they shouldn't within the expiration time. Fixing poor implementations with more standards never works well.

The problem is that servers are allowed to update their resources at any time without waiting for any specific expiration time. So when a user instructs it's browser to refresh the page, usually expecting to get the most up to date version, the browser has to choose between giving the still valid, but maybe not completely updated, cached version or actually checking if the resource has been updated.

Immutable makes it clear that the server won't update the resource in place and will handle updates by generating a new one so the browser can happily avoid checking those resources on page refresh.

I dont see a problem - browsers should honor the expiration time and choose the cache copy if it's still valid.

It's up to the server to use proper headers. Why say a file is ok to cache for years if it actually isnt? If the same url will change content then use shorter cache times or require active revalidation and/or etag checks - or just use the typical cache busting querystring parameters.

This "immutable" flag is unnecessary.

You disagree with the choices the specs have made.

From the cache control rfc:

   When a response is "fresh" in the cache, it can be used to satisfy
   subsequent requests without contacting the origin server, thereby
   improving efficiency.

From the immutable rfc:

   Clients SHOULD NOT issue a conditional request during the response's
   freshness lifetime (e.g., upon a reload) unless explicitly overridden
   by the user (e.g., a force reload).

What exactly do I disagree with? The specs are fine, it's the implementation (the browsers) that are broken which is what I've been saying with this whole thread.

If the implementation is faulty, what is another spec going to solve? Again, there is no need for an "immutable" flag because existing cache headers already express everything that's necessary.

Browsers are free to make a request "just in case" with cache control. With immutable, they are strongly discouraged from doing so. My point is, browsers aren't broken according to the spec if they make those just in case requests.

That is 100% on the browser to optimize itself. Otherwise it's diluting the point of the existing cache header if we need to add a flag to say "we're actually really sure about this expiration time"

And many people are almost certainly going to find that they actually need to either recall an old immutable thing, or mutate it.

Also, I will certainly want to clear out my browser's cache on a regular basis. I do not want it keeping immutable things just because they shouldn't ever change.

You can't 'recall' something you already sent out to browsers, and if you need to mutate then it's easy to make a new URL.

This header won't make browsers cache data any differently. It skips a step when the cache is being read from.

But in the current world, you can serve new content on the conditional check that caches currently do.

That said, I am ultimately for this. I think. There is plenty of data showing that this is a low hanging fruit to hit.

The conditional check that they do sometimes. Now half your users see the new version and half see the old version. Not much of a recall.

Still more of a recall than will be possible in the new world. And you can always detect the old code and prompt users to refresh. (Typically happens on a restart.)

Again, though, I am ultimately for this. I just remain skeptical of any panacea.

You sound as if people who consciously set an immutable header, or set cache expiration time header to 5 years, do not know what they are doing.

Should we optimize the web for clueless server operators?

With the numbers of folks that fix local development by clearing caches... Yes, I am comfortable claiming that. :)

Doesn't that indicate though that the cached resource can't be shared?

Then use cache-control: public with a sliding or concrete expiration time. We can even set different expirations for middle proxies and end user clients.

What does immutable really mean when you only rent a domain name? I think about this occasionally with domains and email addresses, they've become a trusted piece of information, but over time the ownership of that thing changes, does make me wonder about fraud in the future when sizeable companies die off and their domains free up.

In this case, if a company dies and a malicious actor gets the domain, there's not much they can do besides tell the browser to load those assets - but they could probably just take a copy of the original site and serve a copy of those assets themselves.

The attack might work the other way around: the attacker buys a bunch of domain names, serves "sleeper" malicious JS files with this on common paths (say, the paths used by Wordpress and other common CMSs), then releases the domain. When the new owner installs a CMS and start serving their site, the browser loads the malicious JS instead, which is now running under the new site's Origin (security context).

But to make this attack work, browsers would have to visit this site before the new owner takes it over, in order to receive and cache the malicious JS. And if you can make people receive malicious JS, you've already got your attack vector - immutable caching isn't needed.

if you can make people receive malicious JS, you've already got your attack vector

No, because a malicious JS file by itself can't do much. The attack vector is the malicious JS running on the new site, with permissions to steal session cookies and interact with the application. That's why caching without verification is important: to make sure the browser uses the cached malicious JS instead of fetching the new one.

That's an interesting attack vector, in the section 3 of the RFC they recommend to ignore the directive unless it's a secure connection which would mitigate that kind of problems.

Another solution would be to use an unpredictable versioning scheme so the attacker can't anticipate the name of the resources.

The attacker who buys a bunch of domains and legitimately owns them for a period of time wouldn't have any issue getting SSL certificates for them.

Correct, but, even if not explicitly said, the cached entries should be associated to the certificate's fingerprint and immediately discarded once the certificate expires or is changed.

Certificates often change for legitimate reasons, e.g. Let's Encrypt certificates which must be changed every 3 months.

That would be ok.

Isn't this also true for long cache control durations as well?

Yes, although an immutable guarantee makes it more likely that the browser won't check for a new version.

It is no different than, say, incorporating. If you fail to perform certain rituals each year, you can lose your legal status. The odds of this happening any time soon with Coke are slim, similarly with ownership of coke.com, but it could.

Nothing is 'immutable' if you look at it hard enough. The Bible's teachings have changed over time. On the right scale, the Earth is a temporary novelty and questions about proton decay become relevant.

Tl;dr: set your expiration dates appropriately and you won't have a problem.

The issue is that an attacker could buy a domain, serve content from it with the cache-control: immutable response, and then later someone else buys the domain, puts their own content on it, but a lot of their users (who had accessed the domain when it was owned by the attacker) have a lot of the attacker's immutable content permanently cached.

Even if the second owner follows the advice of setting expiration dates appropriately, nothing helps them.

That ignores attacks on dns and certificate authorities.

I could readily see a fun novel about a nation state or isp using this trick to seed malicious code to a lot of public sites. Imagine setting it for index.html on any site with a source that merely bootstraps to the intended site. Along with the malicious code.

In an attack where DNS and CAs are compromised, immutable caching is going to be be least of our problems, frankly.

Hasn't this happened? Several times?

I'm not trying to say it is everyday. But it is easier than you think.

Browsers are removing bad authorities. Which is good. There are a lot of them, though. Which is a larger attack surface than many folks acknowledge.

Nice to see this has made it through the standards process -- the `immutable` keyword is tremendously useful for systems that store and provide actual immutable data, e.g. content-addressed distributed systems.

It would be nice to have strong resource pinning. I've been in contract situations where coordinating with in-house IT for a new server deployment would have been a massive, go-nowhere headache, while throwing a webapp together on my own and self-hosting would've been easy but a big problem wrt company policy on data export. Resource pinning solves this by bringing us into the realm of "auditable". Strong resource pinning would look something like this RFC, except the browser would refuse to accept new resource deployments without deliberate consent of the client. (Something that can be bypassed with a hard refresh, as in this RFC, is not strong enough.)

Other situations I imagine would benefit from this are web crypto and HIPAA compliance.

I have a feeling this will end up like HSTS. It sounds really great at first, then a bunch of folks will get burned by it (just wait until somebody accidentally sets it for /index.html or whatever), and finally the general recommendation will be to stop using it altogether ("more harm than good").

Forever is such a long time.

Besides, aren't there already ways to say "cache this resource for <acceptable timeframe>"?

This is also known as key-based cache expiration, detailed by DHH here: https://signalvnoise.com/posts/3113-how-key-based-cache-expi...

You never worry about when to expire your cache entries if the key changes every time the item does. It's nice to finally see cache-busting coming out of the woods.

You can already use that kind of cache expiration with HTTP of course, and many of us do -- but you can't _tell the user-agent_ or other client that you are using it. The best you can do is set a far-future expires date. Some agents/clients will still do HEAD/If-Modified-Since requests to check if it _really_ changed before your expire date.

So this is a new thing, properly called immutable responses, to tell the client that they really can treat this (in various ways) as a completely immutable response.

I like the idea of this. This would be very useful for versioned resources (e.g., images, JS, and CSS files) because it would eliminate unnecessary requests for them after they're cached.

The example of a 1 year cache time seems a little extreme, though. I think a month would be better.

A 1 year cache time doesn't force the browser to hang on to it for a year, it just lets the browser know that if it's seen that resource in the last year it shouldn't bother getting it again. The browser is still free to dump whatever it wants to reduce its footprint.

Of course the browser/agent still can with a response tagged immutable too. There will never be a standard that _forces_ agents to hold on to cached content regardless of their disk space availability and needs.

But the immutable keyword gives an additional clue to the agent about semantics, to inform caching.

Why? What matters when deciding how long to cache immutable content?

Instructing the browser to cache a resource for a whole year just seems wasteful to me. I know it's completely arbitrary but if your browser hasn't used something cached for over a month, you likely don't need it cached anymore. Now, I do realize browsers have their own rules for maintaining their caches and may not even cache something for a year. That just seems like a very long time.

For example: Say many websites start adopting immutable, version all of their resources, and ~25% percent of their resources are updated monthly. I could easily see the browser cache growing to an excessive size over a year before things begin being purged from the cache. Excessively long cache times are sloppy. That's why I think it matters.

Just because the server gives it a year timeout doesn't mean browsers must hold onto it for a year! The max-age is just a maximum age, not an exact age. Browsers can of course discard any cached responses that they don't think they're going to need again.

Browser caches can hold on to things longer than the freshness lifetime, and do.

They can also evict things before the freshness lifetime is up, and do.

The freshness lifetime is one of the things a browser can use to decide how long to keep something in its cache, but not the only one.

Chrome is not going to get this:

https://bugs.chromium.org/p/chromium/issues/detail?id=611416...

Instead, when the user attempts a full-page reload, Chrome will revalidate the resource in the URL bar, but not subresources (they will come from the browser cache, if the cache-control header checks out):

https://blog.chromium.org/2017/01/reload-reloaded-faster-and...

Yeah I thought that was the current behaviour, and shift-cmd-r actually invalidates all

Section 3. Security Considerations

"Clients SHOULD ignore the immutable extension from resources that are not part of an authenticated context such as HTTPS. Authenticated resources are less vulnerable to cache poisoning."

This must NOT read SHOULD. It must read MUST! Otherwise, your computer will be subject to an executable planting vulnerability.

I'm surprised they don't have a much larger list of security considerations. There are many other issues that can happen.

If "an authenticated context such as HTTPS" isn't being used -- regardless of the presence of this extension -- isn't your computer already "subject to an executable planting vulnerability"?

Maybe. That isn't exactly true. This adds the ability to persist a new threat.

There is a temporal difference. An attacker may wish to plant something today, in a coffee shop, that would execute in a protected environment, tomorrow. Immutability of caching can only help an attacker.

Yes, there are other ways for an attacker do this, but there is no reason to add to more ways! That's why the web is in its current state.

I see how you put "executable planting vulnerability" in quotes. Sometimes (in the current marketing), these are called APTs, but they have been around forever. e.g. Think of dll planting in Windows and the millions of attacks and three or four new API sets from Microsoft that resulted from that single ability to plant a dll in the search path.

This type of persistence can also be called incubation.

It's sad that there's still no good way to do deployment based expiration of assets without horrible hacks like sticking the asset checksum in the URL. I know that none of my assets will ever change unless a deployment occurs, and even then, most of them won't change. HTTP doesn't seem to support this use case well at all.

We're almost there with ServiceWorkers. A site should soon be able to push a new manifest that updates or invalidates the cached resources.

Great pointer on the ServiceWorkers, thanks!

You don't need to use a checksum, just add some irrelevant URL parameter like "?sitever=X" to all asset URLs, then in the packaging/build script just replace it with the current site version:

    find ./ -type f -exec sed -i s/?sitever=X/?sitever=$VERSION/g' {} \;

That will unnecessarily break caching of resources that haven't changed between versions. (Hence, hashing the actual content.)

I suppose if you deploy a lot, that can be wasteful.

> without horrible hacks like sticking the asset checksum in the URL.

What's wrong with this approach?

It requires a way more complicated toolchain because you have to use dynamic methods to generate references to the assets in HTML/CSS/JS.

With that more complicated toolchain comes some added benefits though. See https://www.drupal.org/project/advagg as an example that just about does it all.

You don't really need a toolchain, it can all be done lazily be the web application server.

What do you mean by "deployment-based"? You want things to expire each time you "git pull" on the server?

Wait you're actually running git on production boxes? Doesn't that mean your entire build toolchain also lives on production?

The last company I worked for did that and everything was much slower and fragile than it would've been had we deployed packaged artifacts instead.

Yeah this just doesn't scale, especially if you're pulling from GitHub.

Eh, heroku seems to handle it fine.

No, Heroku has a separate build step before it deploys to your dynos.

I'd like to have a way to force If-Modified type checks on cached assets after a deployment, but not if no deployment occurred.

I mean, there's no way to tell remote agents which aren't in contact with you to contact you, or that the cache should be busted. There's no way to broadcast this "NOW you need an if-modified check, cause I just did a deployment" to all possible caching agents. Not even really any reasonably sane way theoretically/hypothetically, on the internet we've got. Either you've got to tell the remote agent how long to cache for without checking in, or it's got to check in regularly to see if it's cache is still good, I think that's a basic constraint that can't be gotten around.

You can use an ETag of a checksum, instead of a checksum in the filename. Now a user-agent can just check with an if-modified-since and get a response quickly. But it's still got to check regularly.

That or guaranteed changed URLs when content changes are about the only way it's ever going to work in the HTTP client-server architecture, I think. Maybe there's a creative way to come up with guaranteed-changed-url that isn't as inconvenient in development for you, but most people find the current practices a pretty good spot I think.

It can be done if resources are associated, perhaps by site, or by path within a site, and if there's at least one non-cached resource. For example:

Add a HTTP header on resource responses that is 1:1 with the deployment. When this header changes on any response, treat any associated resources in the cache as needing an If-Modified checkin.

Then add that header to a non-cached dynamic page, like the user's home page. When the browser checks this page and sees a deployment change, it'll know to check the static assets.

What happens when wrongly configured servers or frameworks with default settings implement this too aggressively? Browser vendors want their product to work as expected when the user hits refresh, so will be forced to either voliolate the standard or show stale content.

Per foota [0]:

  Clients SHOULD NOT issue a conditional request during the response's
  freshness lifetime (e.g., upon a reload) unless explicitly overridden
  by the user (e.g., a force reload).

The server is still going to serve up the resource when requested. This behavior is for the client-side of things (browser).

[0]: https://news.ycombinator.com/item?id=15262108

Reply on news.ycombinator.com