HiDPI/Retina image upsizing by clients & servers, not javascript | Hacker News Reader

HiDPI/Retina image upsizing by clients & servers, not javascript

github.com

27 pointsskeltoac28 comments

I really like this approach. Is there a plan in place to implement this on a server and client? A simple node server with a hacked up moz might be a good start for a prototype.

I hope to implement server support on WordPress.com if browser hackers are interested. Our image serving architecture is mostly in PHP and has access to originals, so this should not be terribly hard.

Why is this information put in the HTTP headers rather than the HTML for the page in question? It just seems like a relatively obscure place to hide what's bound to be a common bit of functionality; not the most noob-friendly solution.

I submitted a different potential solution to whatwg forums and a few relevant mailing lists, without much significant feedback, using meta tags to indicate availability of 2x image assets. It does seem more practical to me from the point of view of an author, but maybe I'm missing something important:

http://forums.whatwg.org/bb3/viewtopic.php?f=3&t=4998

I think it's so that on the individual image requests the server could be 'smart' and decide whether to return a 2x image or not.

With the solution I propose, it seems like the server could be blissfully dumb, and the client could be smart and choose whether to request 2x images if the page author indicates they're available. I don't see the advantage in this regard.

Adding new HTML conventions puts too much responsibility on authors. HAIR requires very little of authors: store your images according to some new convention. This burden exists in other systems, too. HAIR is unique in that this is the only burden placed on authors. They don't have to worry about making their HTML or CSS match their uploads. They don't have to compare dozens of competing JS solutions every time they start a project.

Blissfully dumb machines are a less worthy goal than blissfully dumb authors. Raising the bar for millions of people--especially without a standard to bind them--is impossible. OTOH it takes a handful of people working on server and browser software to add HAIR to millions of clients and servers. Another handful of people will make macros to help designers export images into the new format.

I guess I set up my proposal the way I did so that authors could explicitly flag assets as being available or unavailable in 2x in the most convenient way conceivable to me, because it would add inefficiency to force a choice between assuming all assets are available in 2x and that none are, as some requests will fail if they're not all available in 2x. I will mention that for my proposal, if you want to assume all assets are available in 2x, the extent of the work for authors would be to add '<meta image-scaling={2:"@2x"} />' to the page. It is potentially more work than HAIR, but I figured it would pay off with server resource efficiency, accessibility to authors, and powerful flexibility.

That said, I could totally get on board with HAIR, assuming my concerns are unfounded (which is likely). How does that system affect server capacity and performance? Is there a significant performance penalty for having the server determine whether each asset is actually available in 2x or not before fulfilling the client's request? Is this information which would automatically be cached, and thus not have a significant impact? Or will each image request have an additional operation to perform?

Also, does this system force you to have your 1x and 2x assets share the same exact filename? In the example, are the 1x images in the 'img' directory, and the 2x in 'image' with the same filename? I'm sorry if my confusion is making me ask the wrong questions; maybe evidence of the challenge of explaining this system to clueless people like me.

[Upon further review, it looks like an img src link to '/img/bob/' would load the 'index.jpg' file from inside the 'bob' directory by default, and then your system would have it load the 'dpr=2.jpg' file if the client wants that. So this imposes even more stringent file naming and organizing restrictions than I had in mind, and it works nothing like what I thought upon first glance (never knew you could even link to a directory for an image asset, and have it load index.jpg, though it does make sense). I am embarrassed to admit that I'm not completely sure where I'd even put those parameters, though I assume it would be in httpd.conf for apache, or maybe .htaccess otherwise. Also, does this require a separate entry for each image on the site?]

I am working on an implementation in Apache/PHP. The first part is an internal redirect in .htaccess to a PHP script which decides what to serve.

Obviously a PHP script is not optimal for broad use. A better implementation would be an Apache or nginx module. But PHP is handy for experimentation.

Because your markup doesn't need to change to support HiDPI. The beauty of this solution is that the browser can be DPI aware and use HTTP content-negotiation behind the scenes to ask for higher resolution images. This is one of the main, albeit underused, use cases of content-negotiation with Accept headers. The main problem is that it requires both servers and browsers to add support. Whereas with a markup based solution that pointed out how to find hidpi assets existing servers can serve two files just fine.

It seems inflexible and potentially inefficient though. What if you have 2x assets available for some images but not others? Will the client be making two requests in all those cases? Or will it simply break?

Server logic can figure it out just fine. For a familiar example check out nginx's static gzip compression support [1]. This works today. Browsers can say they support gzip compressed assets in a header. You don't have to add markup to say "get the smaller version". When nginx gets a request it looks to see "is gzip on the accept-encoding list"? If so, it looks to see if a compressed version of the file is sitting on the server and serves that directly. If not, it compresses on the fly.

Same story could apply here with a few different details.

[1]: http://wiki.nginx.org/HttpGzipStaticModule

I don't understand how that resolves the issue. Let's say you have two img elements on your page, linking to their respective 1x image assets, and you have a 2x version of the image available for the first but not the second. You indicate 2x availability in your HTTP header, and a client which supports this protocol wishes to display 2x images.

Does the client specifically request 2x image file paths for both images, and when the second one fails, as no 2x version exists, the client makes another request for the 1x version, or does it simply fail to load the image?

Or does the client request the 1x asset like usual, and when the server determines that one is unavailable in 2x res, it sends the 1x transparently? If that's the case, do we not care about the server using resources determining availability of individual 2x assets because it's insignificant? And could the client easily choose to request 1x assets rather than 2x in a non-hackish way even if it does support 2x resolution (maybe it's concerned about bandwidth)?

You indicate 2x availability in your HTTP header

This part never happens. The client simply signals that it would like 2x images if possible with every request, and the server sends the 2x images when it is possible, and 1x images when not. This is the exact same thing as has been done with gzip compression for years.

The client can choose to get 1x assets by just not sending the header indicating that it wants 2x assets.

And what about the performance penalty for the server to determine 2x availability for sites where only a fraction of images are available at 2x res?

I'm also not sure I fully understand how we clue the server in on where to look for the 2x version of a 1x asset. In the examples given, do the 2x versions have the same filenames as the 1x versions, but they're in the 'image' directory rather than 'img'? Seems like a significant limitation to force different assets to share the same filename, if that's what's going on here.

You send back the 2x assets where possible and otherwise return the standard asset that is not 2x.

This is needlessly complex. I really don't see pixel ratios of anything more than 2x being useful. Having two versions of assets is painful enough.

The apple convention, of having images with "@2x" appended, works quite well. The server could look for a http header indicating DPI and serve accordingly. This gives the developer to provide either one single HiDPI that's served to all clients, or separate files for HiDPI and normal resolution.

It is short-sighted to assume that 2x will be the limit of high-resolution images. If we're making browser developers add support for this, we might as well have it be at least somewhat future-proof.

Meaning of "1x" has changed from 72dpi to 100-130dpi over time.

It's not unreasonable to assume that same will happen with "2x" as screen densities gradually improve, and over time it will shift from 220dpi to 300dpi-400dpi (for desktop viewing distances).

Density doesn't need to improve indefinitely, it just needs to cross the "retina" threshold (which arguably 2x does already).

There are already ratios of 1.5, 1.33 and 2.25 in the wild.

See Android

Wait, are you saying there's a version of Android somewhere that ships with a web browser which actually specifies its device pixel ratio as a non-integer? I am dubious, since I assume that would force rough anti-aliasing on everything and look like crap.

Some power users with 2880x1800 MBPs are using them at 1920x1200. Sounds crazy but I looked at it and found it almost bearable.

The device pixel ratio is simply the conversion ratio between device pixels and css pixels. It is on the device makers to determine how to interpolate image data into CSS pixels in the browser. There are many devices that have non-integer based DPRs. From what I recall, earlier devices even have DPRs of 0.75

I mistakenly included a great deal of peripheral detail in the first draft. It's not as complex as all that. It is sufficient to provide only 2 versions of each asset: the 1:1 default and a higher-res version, whether it's 2x or 15MP.

You mention an HTTP header indicating DPI. Sending device pixel ratio is already a major part of the HAIR proposal; see Example 2. If that's all that got added to browsers and servers, it would eliminate most of the app-level solutions out there.

We actually talked to some Safari engineers about this and the response was that image-sets were the way forward. I am not sure I agree. The key argument was that the accepts header puts the burden entirely on the server and does not account for multi-resolution environments, such as multiple monitor setups. Content moving from one device/monitor to another could make use of image-sets to load in the appropriate image for its display.

I wish image-set was more efficient to use. I'm way too lazy to maintain explicit lists of every variant file path for every image if I can avoid it (which should be possible if we take advantage of systematic file naming conventions already commonly in place). I'd take this proposed solution over image-set, simply because it avoids that extra work, not to mention being more backwards-compatible; image-set would require you to list the path for the 1x asset of an img element in both the 'src' attribute and the image-set attribute in order to maintain backwards-compatibility.

It won't succeed. The closest parallel is the Accept-Language feature in the HTTP headers, which never took off. By the time this could be implemented in many browsers and servers, everyone will have already solved the problem with javascript.

JavaScript is not a real solution to this problem, since it takes the control away from the client and condemns web developers to maintaining those horrible hacks forever as clients with unforeseen capabilities or expectations emerge. We need something that allows developers to add support for 2x images without having the server make any assumptions about what the client wants. If the client wants 1x images for whatever reason, it doesn't matter what their screen resolution or pixel scaling factor is, or whatever your script is using to determine support for 2x image display.

How would this effect sprite sheets and images used in with the canvas 2d API?

Reply on news.ycombinator.com