Wide Color Photos Are Coming to Android
android-developers.googleblog.com
android-developers.googleblog.com
Here's a video of me moving around on a static background, taken from a tripod. Evidently bending the fabric of the universe as I move:
https://www.youtube.com/watch?v=qSlgOwm_eXU
That's the stock camera app on my brand new Pixel 3a. Since this was taken, I've been experimenting with every setting possible to try to convince it that I don't want to "pop" or be "motion stabilized" but no, there's nothing that will stop it doing this.
I'm going to have to return the phone.
It’s not really noticeable on a phone unless you zoom in, but impossible to miss on a big screen.
I haven't found anything about anybody having the same issue on a regular Pixel 3.
No luck thus far. I can make it slightly less noticeable, or glaringly obvious. But I can't get rid of it.
Please let me know if you can (or better still can't) replicate it. I'm returning the phone tomorrow, but if it's just my device I'd consider buying another one to replace it.
Though that doesn't help when you can't adjust the quality.
If it were video compression, you'd expect it would try to not alter pixels that didn't change from frame to frame. But it clearly updates large swathes of them.
Also, changing the compression mode in the camera from h.264/AVC to H/265.HEVC doesn't fix it.
For instance, in the video provided in another comment, when the climber swings his legs up a bit, there's a portion of the wall that stretches behind him, before eventually un-stretching back into place. It flows behind the moving limbs. I suspect that if good video were taken of the scene, the leg would also be somewhat distorted while moving, but we can't really tell because it's supposed to be changing, unlike the rocks.
The reason that today's cameras on phones look so much better than those of 10 years ago (especially for static photos) is due to advanced methods of image processing. The information available to the image is limited by the diffraction of the lens and by the thermal noise of the captor, and these are two physical constraints that can not be loosened. Thus, any increase of resolution is due to heavy image processing, that denoises and de-blurs the images so that they look as if they were better than they can actually be acquired by this captor. This is an ill posed problem since denoising and deblurring essentially act in opposite directions! The solution to this problem inevitably goes through projecting the low amount of information that you have into an image model of higher resolution. This necessarily leads to artifacts in some cases.
For the case of video, the motion artifacts that you observe in the rocks are often due to video compression and the correction of camera shake. The first is done in hardware so there's not much that you can do about it. The correction of camera shake may have a software part that you may be able to disable. Have you tried different camera apps?
In the same vein I wish that Google would just let me view all the photos I had taken on my phone.
I almost exclusively use the "Camera" Album in the Photos app on my Pixel 2, but some update seemed to irreversibly back up and delete most (but not all?) of my photos. Now when trying to view all these backed up photos online (as there doesn't seem to be a way to get them back onto my phone) the feed is polluted with other random images from Whatsapp, etc. with no way to filter any of them out. And now I can't easily show anyone anything I took on my phone from a few months ago!
It's annoying because I quite like the Pixel 2 camera, but definitely agree with you that it should have more "dumb" options.
It would be useful to know which mode was used and if this occurs in more than one of them.
My armchair guess is that the 3a isn't powerful enough to capture 4K without creating compression artifacts.
But if 1080p is "good enough" I would say that's going to be the 3a's story - just don't use the most taxing settings.
720 would be unacceptable for a camera in 2019, so I didn’t try.
I’ve already initiated the return with google and printed the shipping label. Guess I’m on the market for a new phone if anybody has a suggestion.
I gave up on pestering them about it 10 years ago after they were largely unreceptive about the topic for several years, but it seems like it’s still a complete shitshow.
10–14 years ago I was grumpy because my laptop display had a smaller gamut than sRGB and everything looked bad (undersaturated). Then there were a few years in between where most of the displays I used were close enough to sRGB that missing color management more-or-less worked out. Now many devices have wide-gamut displays which basically make everything on the web look like oversaturated garbage.
Apple had color management more or less figured out on the Mac in the mid 1990s. It is absurd that it is still so hard for software made by gajillion-dollar companies to get the bare basics right >20 years later, especially now that we have superfast GPUs to do compositing in linear space and arbitrarily fancy gamut mapping using floating point arithmetic, brighter and wider-gamut displays, better cameras, ...
To the browser vendors: all untagged content must be assumed to be sRGB. If viewers have wider-gamut displays this content must go through a CMS. If viewers want to view wider-gamut images or other content, this must be opt-in. The current behavior breaks color on the web. (Buy a new phone or computer and suddenly nearly every color relationship you see on the web will be different than the designer’s intention.)
What is X?
2) Let Y = The color profile assigned to your display.
When X does not equal Y, which option should the browser choose for converting colorspace X to colorspace Y for display purposes?
a) Relative colormetric b) Absolute colormetric c) Untranslated integers d) Other
Then,
3) Let Z = The color profile assigned to CSS HSV/LAB color codes.
What is Z? Is it the same as X?
I don't envy browser developers any aspect of this problem. If they do it right, no one will notice care. If they do it wrong, the Internet will burn them alive. I can't blame them all for being scared, but I hope that now that we've finally exited the "wild west" of "My display could have any ICC profile under the sun" and are instead approaching "My display is either sRGB or Display P3" that they will, at the very least, decide what to do and then do it and make us all adapt.
ps. If you answer question 1 with "X has no assigned color profile", then here's two extra credit questions for you:
4) Should a CSS box with "no colorspace" #ff0000, when displayed on two calibrated displays of equal luminosity but differing (sRGB, P3) color profiles, appear visually identical to the human eye (Safari) or should it be more saturated on P3 (Firefox)?
5) What is the correct CSS declaration to specify P3(r=1.0,g=0.0,b=0.0)?
pps. It looks like the CSS working groups are being pushed by both Microsoft and Apple to solve this problem, so eventually they will clearly all standardize on some way to specify colors in non-sRGB, and likely default unspecified colors to sRGB. To which I have to say, "Finally", because maybe then the one outlier will finally align with what I think is right. This is a hard problem, don't get me wrong, it's just frustrating having waited a decade for them to solve this and it's still not solved.
ppps. The answer to all of the above questions can likely be found here, once they exit the draft stage someday. The above is all designed to convey the problem in a nutshell to people who aren't familiar with the core problem. If you truly understand and care about this stuff, here you go:
CSS Color Module Level 4, W3C Working Draft, 5 March 2019: https://www.w3.org/TR/css-color-4/#predefined
11.2. Predefined colorspaces: srgb, image-p3, a98rgb, prophotorgb and rec2020.Here’s a 300 page monograph discussing more or less the state of the art 10 years ago, https://amzn.co/0470030321
But here’s the thing... it doesn’t really matter. Their current choice is to just treat everything untagged as being in the display’s profile and do no color management. This is pretty much the worst possible choice.
The specification is very clear. Untagged colors in CSS/HTML are sRGB. Period. Doesn’t matter if they are specified as RGB or HSL coordinates (P.S. it was a huge mistake to add the latter to the web; HSL/HSV should never be used for any purpose).
Currently most of the browsers flagrantly violate the specification.
The browsers have been doing things wrong since forever. People who know anything about it have been annoyed since forever, but their feedback seems to have very little influence. Everyone else is blissfully ignorant, except for often encountering weird color bugs that they don’t understand the cause of.
> Should a CSS box with "no colorspace" #ff0000, when displayed on two calibrated displays of equal luminosity but differing (sRGB, P3) color profiles, appear visually identical to the human eye (Safari) or should it be more saturated on P3 (Firefox)?
It should appear more or less the same on both. The spec explicitly specifies that it is sRGB. Under no circustances should it be naively treated as P3. However some gamut mapping algorithms might make it slightly more colorful when going from sRGB -> P3 (without significantly altering lightness/hue relationships between colors).
> What is the correct CSS declaration to specify P3
This is something to take up with the CSS spec authors. They can add some ability to opt-in to tagging non-sRGB colors. If the spec doesn’t support it, then there is no way.
* * *
Edit: Your linked draft spec clarifies that something like relative colorimetric intent must be used. “Implementations may choose to implement these steps in some other way (for example, using an ICC profile with relative colorimetric rendering intent) provided the results are the same for colors inside the source and destination gamuts.”
So your answer about the untagged #ff0000 is that it should look pretty well identical on both displays.
Glad they are adding some facility for non-sRGB colors to CSS. Of course it wouldn’t be the W3C if they didn’t also add some useless clutter in every new spec. The “HWB” color space? Who comes up with such nonsense?
(Compare to SVG trying to add turtle graphics and Catmull–Rom splines to their path specification mini-format, for basically no reason, in the latest draft spec.)
HWB (like RGB and HSL and HSV) does not make sense to “average human beings”. It only makes sense to people who have spent years training on the specifics of RGB displays (e.g. your proposed “lighting technicians”). For everyone else it is a confusing mess which produces bad results which differ significantly from their intentions.
If you want a color model related to NCS, specify something which properly follows its principles. https://en.wikipedia.org/wiki/Natural_Color_System
> magical shortcut
This is neither magical nor a shortcut. It’s a superfluous hassle that anyone who wants to implement the spec needs to implement, debug, and maintain, and anyone reading the markup for arbitrary web pages needs to know about one more weird complication.
If individual web designers want to use this idiosyncratic tool, they are welcome to put it in their page’s javascript, or add it to their CSS pre-processor. But it should not be foisted on the rest of us.
HWB: https://commons.wikimedia.org/wiki/File:GTK_color_chooser.pn...
NCS: https://commons.wikimedia.org/wiki/File:Animation_of_the_NCS...
The NCS is a similar concept, but executed based on a plausible theoretical model and (more or less) sound empirical study of human perceptual response to visual stimuli.
“HWB”, like HSL, is just someone’s trivial half-assed transformation of the RGB cube, completely ignoring more than a century of color science research. It’s what you get when a person off the street were tasked with redefining the NCS, without understanding it. It reminds me of people’s bicycle sketches, http://www.gianlucagimini.it/prototypes/velocipedia.html
Personally I am not the biggest fan of the NCS system for designating colors; I find Munsell-like coordinate systems to work better for me, in practice. But at least it has some kind of theoretical and empirical basis.
https://pdfs.semanticscholar.org/7833/f96c09a1201f0e7f54e0de...
CSS, having already standardized on Munsell hue circles (page 5) in the past, appears content to continue forward with them as the hue circle of choice for HWB. While I respect that you would prefer to see them use the NSL hue circle (pages 10-11), it appears that they are unlikely to do so at this time.
One bonus find here is the abstract for the cited Whitfield (1988) paper, where the paper's authors attempted to replicate the assertion that NSL is better than Munsell, and failed:
https://onlinelibrary.wiley.com/doi/10.1002/col.5080130210
The study reported here is a replication of experiments cited as support for this claim. Subjects were trained in the NCS using NCS training material and carried out colour‐identification tasks. For comparison, a further subject group was trained in the Munsell system and carried out identical tasks. The results indicated (a) a lower level of accuracy than previously reported for those subjects trained in the NCS and (b) little difference in performance between the NCS‐ and Munsell‐trained groups.
CSS has nothing to do with Munsell hues or NCS hues. I have no idea what you mean by “NSL”.
The “HSL” / “HWB” hue circle is just a trivial transformation of the RGB cube (namely: zigzag along the 6 most colorful edges of the cube at constant speed), designed by non-color-expert computer programmers based on what was convenient to implement on workstation computers from the late 1970s (note: these considerations are not relevant 40 years later), and has almost nothing to do with human perception.
It is nowhere near perceptually uniform (unlike the goals of Munsell hues), and also does not have human-perception-meaningful landmarks like the NCS cardinal directions.
If you follow a circle of constant “W” and “B” in the “HWB” model, at constant speed in terms of “H”, then a typical human observer will see wildly varying “lightness”, “colorfulness”, “saturation”, “whiteness”, “blackness” with respect to their own perception, and will see perceived hue change at highly irregular and unsmooth speed.
https://arxiv.org/abs/1802.06067
(Implementation: https://github.com/nschloe/colorio )
And this (read the Appendix on the BACKLIGHT of the Sharp Quattron, and take a look at Figures 1.3, 3.5, 9.2, G.3!):
Note however, that all of this completely ignores the fact that humans seem capable of some form of HDR color fusion via binocular vision, as impressively demonstrated in this paper:
http://arxiv.org/abs/1703.04392
Given that many people seem to report slight differences in monochromatic color differentiating between eyes, we can speculate that binocular vision influences color perception in some way.
Edit:
See also my comment here: https://news.ycombinator.com/item?id=19983432
(Make sure you watch for typos in your link.)
But frankly using CIECAM02 or CAM16 (or whatever) vs. CIELAB for doing gamut mapping is a relatively minor improvement in most cases. There can be serious issues when trying to convert RGB images to CMYK or similar for printing, especially hue shifts in the blue–purple region. But for converting colors from sRGB to a wider-gamut display there shouldn’t be any significant issue.
The big problem is that historically most browsers have done no color management of CSS colors. Looks like that is now changing, which is encouraging, whether or not the gamut mapping algorithm they use is state-of-the-art.
> ignores the fact that humans seem capable of some form of HDR color fusion
This seems largely irrelevant to the discussion in this thread.
Nice, thanks for sharing! You seem to work on a lot of stuff I find highly interesting! :) entirely off topic, but, you might find this of interest to your logarithmic spirals work:
https://link.springer.com/article/10.1007/s00004-015-0285-1
>But for converting colors from sRGB to a wider-gamut display there shouldn’t be any significant issue.
I agree, but I'd prefer it if eventually, we treated sRGB colors the same way we treat CGA colors. Especially due to the issues outlined in the other paper I linked. (Namely, that our 'red' subpixels ain't red, they're more orange-red, and that the 'green' ones ain't green either, they're more yellow-green. Really a shame Sharp messed up their tech in that regard by just using a standard back-light, it put a huge blemish in the quite sane idea of having yellow subpixels.)
But, at the moment, we don't even have Chrome properly supporting ICCv4 profiles, so, those dreams of the far future will have to wait.
>This seems largely irrelevant to the discussion in this thread.
>largely
Yes, I agree, I placed it there as a note of caution about:
1. Limitations in the works linked to;
2. The fact that color science doesn't seem to account for stereoblindness and monocular vs. binocular stimulii more often than not;
that's all.
That's also why I linked that PhD dissertation, which also goes beyond CAM16. I in fact neglected to mention some other limitations, namely, that it also doesn't fully factor in:
1. The works on color perception by Brian P. Schmidt¹ et al.: https://bps10.github.io/ (Note: It does factor it in /in part/)
2. More esoterically (by which I DON'T mean pseudoscientific.) the ones noticed by John Medeiros: https://www.youtube.com/watch?v=4Oj0hweYE00
¹ NOT the Brian P. Schmidt who won part of the 2011 Physics Nobel price
Edit: Almost entirely off-topic, but, I try to keep my thematically related Hacker News comments at least indirectly hyperlinked to each other, so, here, another color related comment:
https://cameratico.com/guides/firefox-color-management/
EDIT: That's the desktop - can't speak for the mobile version. I really hope they turn it on by default in the new mobile browser they're building. Also, thanks, I had enabled the color management a long time ago but it apparently was disabled again in a recent upgrade or something.
First of all: Chrome still has broken color management. DisplayCAL has a way to generate profiles that look like garbage in applications which process color profiles incorrectly, and this happens with Chrome (but not Firefox.).
Besides that, even if it had feature parity with Firefox, we'd still remain stuck in the land of ICC color profiles, and those have a few issues.
I'd love to see web browsers support this modified version of this:
https://arxiv.org/abs/1802.06067
For which an implementation exists, here:
https://github.com/nschloe/colorio
…but no operating systems and hence no browsers support it.
Edit:
See also my comment here: https://news.ycombinator.com/item?id=19983604
I hear you, but at the same time this relevant xkcd comes to mind:
Colorio looks great, thanks for the link! Also has Jzazbz support I see, neat!
App authors and OS builders have refused for a long time to make this a priority - I’m looking at you, Slack - and now finally with both iOS and Android moving forward, I really hope it’ll be enough to finally force them all to suck it up and implement proper ICC handling.
After reading the description, I can see that the colors are a little brighter.
But, now that I can see it, I can't bring myself to care. I'd rather forget about it. This won't make my life any better. It's just a way to sell more expensive hardware, much like audiophile gear.
It's the same for most consumer tech, and is very noticeable on phones/tablets/laptops. The only thing we can improve on is shaving of .25mm every gen and calling it a revolution. Or adding machine learning + a 8k screen to our smartphones to explain why you need to spend 1000$+ on a 10 core cpu and 8gb of ram.
The complexity skyrockets, the price doubles but the user experience barely changes, and sometimes it's not even a positive change. Because at the end of the day the vast majority of people just want a basic camera, spotify, netflix and google maps.
https://2.bp.blogspot.com/-LqRniYeVeEk/XOLYEiOcKCI/AAAAAAAAJ...
If on a Pixel 3 or Samsung Galaxy S10 or another P3 capable display, it's a faint-but-visible red logo. Otherwise it's just solid red.
Edit: If not on Android Q, try using Firefox.
There seem to be some odd behavior depending on the software and this may not be always consistent.
It might be included only in the Android Q update.
Googling around shows that it might be restricted to custom apps only?
"There is one way to bypass this issue, and that is to explicitly tell Chrome to override the default provided system colour profile and just tell it the display is Display P3. This enables colour management in the app, and short of writing a custom application, is the only way to get wide colour gamut content to work on the Pixel 3."
https://www.anandtech.com/show/13474/the-google-pixel-3-revi...
Take a screenshot of the full-screen image. Do you see the logo flash?
Zoom in to the image. Do you see the logo appear, but only near the edges of the display?
Computers!
On my Pixel 3 XL, the behaviors I described occur, and also the logo is invisible unless the overlay is shown (in which case it's drawn only near the edges, unless you're taking a scree...oh.
I get it.
Image math is getting done internally in a wide gamut. Including alpha blending. But it's converted to sRGB before being displayed.
The things I described all, I suspect, modify the image- mostly by just dimming it, and the resulting dimmed wide color values map to different sRGB colors (where the undimmed colors all map to the same shade of red.)
as to what's actually happening: not a clue! just pointing out possibilities :)
If anyone is curious: https://i.imgur.com/IGTAWZF.png
I'm looking at this from a macbook pro 2015 so I assume no wide colour support (?), yet I see the logo. Which conversion, at which step, caused that effect?
The clipboard here is probably copying the image buffer in the untransformed state, and pasting it into MSPaint would dump those values incorrectly.
The issue would be the mechanics of the copy (attaching metadata) and pasting (interpreting the metadata in conjunction with the source image buffer.)
So technically, no, there's no conversion on the source data, it's the representation in the pasted context that's the issue.
I then saved it as a PNG (still showing the logo in Windows image viewer) and uploaded to Imgur. I assume pasting it into paint compressed the colors?
I saved it as a PNG and uploaded it. I don't know the technical details behind this but I'm assuming Paint compressed the colors.
Not sure about Chrome as I set the color flags to be sRGB a long time ago as, compared to Firefox, colours in Chrome are unsatisfactory on Linux.
I also tried on a consumer grade laptop that runs the latest (19.04) Ubuntu with one of those glossy screens, definitely not designed for Display P3. This failed in Firefox, Chrome, Gimp but the thumbnail in the file browser came good. How come when the hardware isn't there for it?
It seems like 25% of the colours are missing with a lot of that being green, green being quite an important colour. It must be possible to have some creative fun with this. If you can get images that only kids can see on their phones but adults not see on their boring PCs then you could have some variant of 'satanic backmasking' going on.
'Satanic backmasking' was where you could hear stuff about Satan on Beatles records if you played them backwards, listening whilst stoned. Politicians bothered themselves about young minds being corrupted by such things back in the day.
Maybe DCI-P3 images can court a similar controversy, shame April Fools Day is a lame day of corporate gags where nobody is fooled, otherwise DCI-P3 is ripe for moral panic, terrorising the adults with another thing for them to get their politicians frenzied about.
I hope Chrome gets to do colour a bit better, to Firefox standards. Otherwise we will be with half the world's screens not showing all the colours. Although not the same gap as between colour and monochrome, it is a huge difference to anyone that works with colour and needs subtle colour variations.
Yet I cannot get the android logo to display on Chrome or when the picture is downloaded and viewed on the default photo viewer.
Oh well maybe a future update will solve this issue. :p
Actually I'm a hobby photographer, and I generally favor accurate color representation over any other special color modes. I do remember playing around with the settings when I got my phone and I think I just forgot to switch the color mode back to normal from vivid.
tl;dr: (╯°□°）╯︵ ┻━┻ color is hard.
Edit: answered below - download to device, open in photos (menu -> device folders)
However, donwloading the png file and opening it on any image viewer it shows two different shades of red. I guess firefox is applying a silly color contrast change to its images before displaying them.
It's also solid red in GIMP 2.10, which surprised me.
Which would mean that the red logo is just a result of mismapped colors.
I was pretty surprised to see the logo on a phone from 2011 with a rather meh screen…
I tried setting the flag (chrome ://flags/#force-color-profile) and choosing P3, but I still see just a red square in Chrome.
EDIT: muro's tip (open in photos (menu -> device folders)) worked. Not sure why opening the same image from the Photos home screen doesn't work.
Chrome displays a red square, Firefox displays the wide gamut Android logo. Good job, Google.
The blog post mentions "8 bits per color channel is not enough", but as far as I can tell everyone using Display P3 for consumer-grade photos (including the iPhone) is still using 8 bits per color, just over the wider gamut.
It seems weird that this hasn't become a thing with the big marketing push behind HDR10 and others for TV. You might think "backwards compatibility", but Apple was fine making the HEIF break and converting when sharing with incompatible devices.
Encoding in a larger volume colour space may require additional bit depth to prevent posterization, but that's somewhat tangential. The larger issue is the number of vendors and operating systems that can't get it right. Windows is horrific, for example, with Android being at least as bad, as given by the accounts in the comments here.
Are you trying to post the same thing in every top-level thread to get your comment seen? This has nothing to do with what I was asking, which was pretty much "everything about gamut and display aside, what's the deal with bit depth?". Hopefully you can see how your response is a distraction.
Why is this important? Most of these cameras images end up on the web. VP9, HEVC and HEIC all support HDR10 so with the right processing this should come through.
1. You have no color management: you see the logo, but it is desaturated
2. You have a working CMS and standard gamut: no logo
3. You have a working CMS and wide gamut: logo is correct
#1 is true because the image has different values. #2 maps all the colors to the monitor's gamut and so you can't see anything on a standard display.
Funny enough I can see it just fine on my Macbook Pro running the latest Firefox stable.
Haven't looked at the image too deeply, but this is likely the rendering intent being set so something other than the colorimetric offerings, such as Perceptual or Saturation. It could also be a poorly encoded image as well.
https://www.anandtech.com/show/13392/the-iphone-xs-xs-max-re...
See chrome://flags/#force-color-profile
It has no adblocker.
So Chrome is using the APIs, but actually using that profile is still behind a feature flag, it seems.
Edit: I just tried it in Firefox and the log is much clearer. This makes me think that Firefox is doing something to change the image before displaying it, which is why it "works" there.
On Chrome, the image was incredibly faint.
My (probably flawed) understanding was that wide colors means there's more than 8 bits. So stuff that would be say rgb(254.5, 0, 0) would now be distinguishable from rgb(255, 0, 0)? If you screenshotted it back to 8 bit you would just lose the extra bits and I expected to still see a red square? Or is that not how it works?
Safari on my iPhone X passes.
This also ignores the shortcuts that Chrome and other browsers have taken on their road to colour management.
Maybe surprising to some, but Windows actually supports HDR10 (which includes "wide-gamut" color) better than other OSes. You can, for example, play HDR10 video games with a connected HDR-capable monitor. It might be thanks to MS's Xbox platform.
I think the ICC profile is the only reason I can see it and it isn't a very strong difference on my monitor.
Edit: When I say 32bit, I mean 8 bit per RGB channel/24 bit "True Color"