Well, it's pretty arbitrary. What is missing from most image (or sound) data is metadata to say what physical intensity is represented by the signal. I.e. how many nits (or decibels) should be emitted for this signal.
AFAIK, most encoding standards only define a relative interpretation of the values within one data set. And even if standards did have a way to pin absolute physical ranges, many applications and content producers would likely ignore this and do their own normalization anyway, based on preconceptions about typical consumers or consumer equipment.
To have a "correct" mixing, you would need all content to be labeled with its intended normalization, so that you can apply the right gain to place each source into the output. And of course there might be a need for user policy to adjust the mix. I think an HDR compositor ought to have gain adjustments per stream, just like the audio mixer layer.