HDR‑Infused Emoji
sharpletters.net
sharpletters.net
As the early versions of the images emerged we thought we could used HDR to provide more or a aura to some elements. We tried to make it subtle and not overwhelm.
This example is my favorite:
https://restofworld.org/2024/divinity-altered-reality-muslim...
I think it worked well - and this technique would have been useful. We tried something similar but could not get it to work.
Our method was to use a stretched HDR video in the background.
Here are the steps I used:
In Photoshop create white image to proportions required. Save as MP4:
File > Export > Render Video
With the MP4, generate a HDR version in WEBM:
ffmpeg -i sample.mp4 -pix_fmt yuv420p10le -color_primaries 9 -color_trc 16 -colorspace 9 -color_range 1 -profile:v 2 -vcodec libvpx-vp9 sample.webm
ffmpeg -i sample.mp4 -pix_fmt yuv420p10le -color_primaries 9 -color_trc 16 -colorspace 9 -color_range 1 -profile:v high10 -vcodec libx264 sample.mp4 .religion-atf__nav-chapter--current .religion-atf__nav-chapter__book {
box-shadow: -4px -4px 50px 0 #fff,4px 4px 50px 0 #fff
}Like in the famous case of the Apple logo in the 1990s. Steve Jobs, when asked why he uses a black and white Apple logo instead of a color one, said - "color will only distract the eye from what's important".
I can immediately tell if anyone has messed with any knobs or buttons on my receiver or if any of the speakers seem off / wrong sounding. Maybe I'm that 1% but I can remember many multiple times people have been over for movies / TV and someone asks out loud "does the sound seem off?" and sure enough a kid or a clueless friend was messing with the knobs.
Selectively deployed, a glint of extra brightness, above and beyond the “100%” baseline, simulates the glints and shimmers that draw our eyes naturally—in this case, in the same manner as gilt on the physical counterpart to the books they’re depicting. It fits in cleverly with a long tradition for that specific context.
Where I agree is with the idea that brighter-for-brighter’s-sake is not better after a certain point, any more than color-because-we-can. And it seems, as far as I can tell, that uniformly cranking up the full frame brightness into the HDR range is not The Done Thing, at least in film and design, at least so far. Possibly for compatibility with the wide range of displays stuff will end up on.
If the tech does seep into ads, as I guess it must eventually, I’ll be right behind you in turning it off…
Usually the first rule of web development is to not touch scrolling, however, I’m on the iPhone and it’s seems to be faster than native scroll, and surprisingly it feels very good!
For this one, we did not interfere with scrolling behavior.
One thing I was attempting to do was create an emoji that only had partial portions that had their brightness maxed, while the rest of the emoji displayed "normally". It turns out this was quite hard to do - most filters or scripts such as the one on this page simply max out the channels, which creates a "fried" look to the image.
Gimp luckily has support for this, but getting an existing image to display the "same" with the new color space proved fairly difficult. I found this curve to be the best starting point, after which I edited things from there manually: https://image.non.io/a3c227d4-56d6-40bd-a898-3879fd062cf3.we...
I used this to create a companion emoji to the :mean-jacob: emoji (https://html.non.io/emoji/mean-jacob.png) my team is quite fond of using (I'm Jacob). I made :angel-jacob:, which has a halo that shines with the brightness of a thousand suns. I don't expect my team will use it, but still it was a fun exercise: https://html.non.io/emoji/
Or you're using Safari because my hardware absolutely does support this (tested in Chrome and I am thankful that Safari does not support it because good grief.)
Safari absolutely will support HDR images if it doesn't already. It might not support this PNG hack, but it's inevitable that it'll support HDR HEVC or JPEG images since those are what's produced by iOS and Android cameras respectively, and they obviously aren't going to just ignore them.
iPhones have not captured HDR images until much, much more recently. No earlier than iPhone 12 at the soonest (when they first could capture HDR video), although they keep fiddling with which format they use for the result. iOS 17 was when they added support for displaying these images in UIKit & Swift, which was only like 2 years ago give or take. WWDC '23 was similarly when they started talking about handling HDR images. And they just recently announced they'll be adopting ISO 21496-1 at WWDC 2024. ISO 21496-1 being the gainmap style approach that Google & Adobe adopted with UltraHDR in 2023.
Here is a recording of this happening for those who can't experience it for for themselves:
https://logandark.net/files/2QN2R1P3-26295O2O-9045N31P-P2POQ... (6.7 MB / 6.4 MiB)
(sorry for the terrible quality, it was a very lazy recording)
And will continue to see for quite some time when my eyes are closed.
I should not have been clued into this power.
The first emoji here is just too bright. Even with my phone on low brightness, it's too blinding compared to everything else.
Demo: https://notes.dt.in.th/HDRQRCode
Interestingly that one worked on iPhone, while the new emojis one doesn't
My apologies to your co-workers
Maybe some day.
Related: there is also a CSS property coming which allows sites to control which page content should be clamped to standard range or not. Worst case you can just add an * !important override in your Safari->Preferences->Advanced->Style Sheet if nobody else considers it problematic but you still wanted to clamp things (in Safari, otherwise you can just disable HDR).
The only reason Safari lacks HDR image support on macOS right now is it's their lowest priority platform for the feature. It's coming, it's supported on their other platforms, and it wasn't an unreviewed accident they've been working on it.
Could you provide a link to any recent communication from them about this, or are you just speculating based on that other platforms support it?
I do believe that Apple generally has plans to implement HDR support in Safari, but I wouldn't be surprised if they immediately walked that back once we see abuse of the technology annoying regular users.
I don't think it'll be problematic though because a site can already choose to show you images a lot more bothersome than a bright light (I say this as I type on a 1600 nit HDR monitor) already and that's not a particularly common problem to worry about either. Same for videos, which already HDR support in browsers.
But this is hardly true. There are some complicated heuristics (like Chrome's "Media Engagement Index") but many websites can and do autoplay video and audio. And browser policies are even more relaxed for playing audio on user events (like clicking).
I am not talking about a slight brightness increase, I am talking Ill be scrolling youtube and suddenly this video is like a portal into another dimension its so bright.
Can anyone explain how its done?
Yes, there are formats that able to store a higher contrast ratio so that's why it doesn't happen on non-HDR content but the actual brightening of a portal on your screen isn't because of the format but because of your hardware (and software) choosing to interpret the format that way.
For more a practical example, if you had an 8-bit HDR image, 255 on the red channel (after inputting this number through a math function like HLG[1] to "extract" a brightness number) might mean "make this pixel really bright red" whereas 255 on a SDR format would mean "just regular red." However, each red channel is still a number between 0 and 255 on both formats but your hardware decided to make it brighter on the HDR format.
(Although in reality, HDR formats are often 10-bit or higher because 256 values is not enough range to store both color and brightness so you would see banding[2]. Also, I have been using RGB for my example but you can store color/brightness number many other ways, such as with chroma subsampling[3], especially when you realize human eyes are more sensitive to some colors more than others so you could "devote fewer bits" to some colors.)
[1] https://en.wikipedia.org/wiki/Hybrid_log%E2%80%93gamma
There's no system that does that. The only thing that's kinda similar is at the display level there's a concept known as the "window size" since many displays cannot show peak brightness across the entire display. If you've ever seen brightness talked about in context of a "5%" or "10%" window size, this is what it means - the brightness the display can do when only 5% of the display is max-white, and the rest is black.
But outside of fullscreen this doesn't tend to be much of any issue in practice, and it depends on the display.
You mean the darkening of everything else to highlight bright HDR areas? All recent Macs do, including the one I'm typing on right now. It's a little disconcerting the first time it happens, but the effect is actually great!
Thanks to everyone trying to help me understand this. I have heard of HDR for years but Ive never witnessed my macbook darken and brighten a video before like 2 months ago.
if you record in HDR, uploading that raw footage to YouTube should produce an HDR video. to get the raw footage, you can either upload the file directly from the phone, or AirDrop it to your Mac from Photos (you should get a .mov), or sync it to iCloud (or connect the phone over USB, maybe) and then use Photos' "File > Export Unmodified Original"
HDR10(+) & Dolby Vision, for example, encode content at absolute luminance, so they are basically completely trash formats since that's an insane thing to expect (the spec for authoring content in this format literally just goes "lol idk do X if you think it's going to be seen in a movie theater of Y for TV and hope"). Sadly, they are also quite common. Mobile phones (both Android & iOS) are instead pushing HLG, which is better. Although then hilariously MacOS's handling of HLG was atrocious until the latest update which fixed it but only if the video contains a magic flag that iPhone sets, but isn't standard so nobody else sets it (the "avme" tag https://developer.apple.com/documentation/technotes/tn3145-h... )
There's then also just how your eyes & brain react. When HDR shows up and suddenly the white background of a page looks like a dim gray? That's 100% a perceptual illusion. The actual light being emitted didn't change, just your perception of it did. This is a very hard problem to deal with, and it's one that so far the HDR industry as a whole has basically just ignored. But it's why there's a push to artificially limit the HDR range in mixed conditions, eg https://github.com/w3c/csswg-drafts/issues/9074
Can someone else confirm I am not mad?
PS - I am not trying to shut you down, you clearly know alot in the space I am just explaining what Im experiencing on this hardware.
This is almost certainly your eyes playing tricks on you, actually. Setup that situation where you know if you scroll down or whatever it'll happen, but before triggering it cover up the area where the HDR will be with something solid - like a piece of cardboard or whatever. Then do it. You'll likely not notice anything change, or if there is a shift it'll be very minor. Yet as soon as you remove that thing physically covering the area, bam it'll look gray.
It's a more intense version of the simultaneous contrast illusions: https://en.wikipedia.org/wiki/Contrast_effect & https://en.wikipedia.org/wiki/Checker_shadow_illusion
Eyes be weird.
SDR brightness is not reduced to "add contrast". The blog post doesn't seem to say that anywhere, either, but if it does it's simply wrong. As a general note it seems wrong about a lot of aspects, like saying that Apple does this on non-HDR displays. They don't. It then also conflates EDR with whether or not HDR is used. EDR is simply the representation of content between apps & the compositor. It's a working space not entirely unlike scRGB where 0.0-1.0 is simply the SDR range, and it can go beyond that. But going beyond the maximum reported EDR range, which can be as low as 1.0, the result is simply clipped. So they are not "simulating" HDR on a non-HDR display.
> The SDR content is dimmed proportional to the increase such that SDR has the same emitted brightness before & after the change.
That's the intent, but because things aren't perfect it actually tends to get darker instead of stay perceptually the same. It depends on which panel you're using. MBPs are prone to this, XDR displays aren't.
Your layman summary is wrong, though. Brightness stays the same is the summary, whereas you said it gets darker.
> MBPs are prone to this, XDR displays aren't.
On my M1 16" MBP it doesn't have any issue. The transition is slow, but the end result is reasonably aligned to before the transition. But yes MBP displays are not Apple's best. Sadly that remains something exclusive to the iPad
Although to be fair it is not like Windows in general supports HDR well for desktop use anyway, SDR colours are way too DIM like brightness is at 25%
For an image or video:
- https://github.com/ccameron-chromium/hdr-headroom-limit/blob...
- https://drafts.csswg.org/css-color-hdr/#controlling-dynamic-...
For a canvas element:
- https://github.com/w3c/ColorWeb-CG/blob/hdr_canvas_r2/hdr_ht...
Resource: https://www.w3.org/TR/mediaqueries-5/#dynamic-range
And this explains how Apple implements this feature on non-OLED/mini-LED screens (and, in my observation, at least still to some extent even on mini-LED): https://prolost.com/blog/edr
Edit: Maybe my hardware doesn't support it. I'm using an LG monitor with Windows. There's also a good chance I've never actually seen anything in HDR.
Then you are probably not viewing this with HDR-capable hardware and software. Otherwise it'd go past what you can just do with normal manipulation on an sRGB image.
On compatible screens, those emojis are appearing super bright. In fact brighter than the maximum brightness setting while the rest of the screen doesnt change.
Hurmm it does work for me in Chrome on Android.
What you will see however, is completely unknown.
chrome://flags/#force-color-profile
It's working for me in Chrome on my Pixel. I've also noticed Instagram using it recently. Very fun effect, could imagine a good dark mode prank using it...
Funnily enough, on my iPhone I'm getting the blue box question mark instead of the images on the page.
The fact that you can't turn it off system wide shows the macOS leadership is asleep at the wheel
macOS handles it about the best of the bunch.
What I hate is on Windows, you need to basically explicitly set the program, the OS, and the monitor into an "HDR mode". Then, once you're done, you need to un-set it or the colors and brightness will be screwed up.
That is tedious AF. I refuse to use it until it doesn't require constantly toggling crap on and off.
It's actually a setting in snipping tool (unbelievably)
Even nvidias was wrong for a while.
https://www.reddit.com/r/nvidia/comments/1b03yfg/rtx_hdr_pap...
I just want my hdr mapping to work
You totally can, at least on Apple's XDR displays.
Just go to System Settings -> Displays -> Preset and change it from "Apple XDR Display (P3-1600 nits)" (or whatever) to "Internet & Web (sRGB)". You lose the ability to change screen brightness (I assume because you're locked to reference brightness), but HDR is fully off.
Personally I think the biggest benefit of HDR is not even those super bright annoying colors but 10-12 bit colors and the fact that we can finally have dark content. If you look at movies from 10-20 years ago everything is so damn bright.
Of course I do agree that these things should be configurable. And on my MacBook Pro, I can set the built-in display to sRGB. Is that option not available on your particular Mac and display?
Some of it was just bad historical decisions. In particular SDR video only uses the values 16-235 instead of 0-255 because of some NTSC compatibility thing I don't quite remember. That's a huge loss!
Working fine here on my pixel 8 in Vivaldi