Comment by mceachen | Hacker News Reader

Comment by mceachen | Hacker News Reader

Getting image and video orientation correct, which should be trivial, is decidedly not. In writing PhotoStructure [1], I discovered

1) Firefox respects the Orientation tag using a standard [2] CSS tag, but Chrome doesn't [3]

2) That videos sometimes use "Rotation", not "Orientation" (and Rotation is encoded in degrees, not the crazy EXIF rotation enum). Oh, and some manufacturers use "CameraOrientation" instead, just for the LOLs [4]

3) That the embedded images in JPEGs, RAW, and videos sometimes are rotated correctly, and sometimes they are not, depending on the Make and Model of the camera that produced the file. If Orientation or Rotation say anything other than "not rotated," you can't trust what's in that bag of bits.

[1] https://blog.photostructure.com/introducing-photostructure/

[2] https://www.w3.org/TR/css-images-4/#image-notation

[3] https://bugs.chromium.org/p/chromium/issues/detail?id=158753

[4] https://exiftool-vendored.js.org/interfaces/makernotestags.h... (caution, large page!)

And after all that, editing (simply rotating) the photo in different applications has different results. Some change only the Orientation tag, others change the actual data, some seem to change both so it's still incorrect when opened in other viewers; and then there's the embedded thumbnail (but that is rarely used), so the result is a mess.

I'm interested in your PhotoStructure application, just subscribed to the beta!

When you rotate a photo or video in PhotoStructure, you can have it persist rotation/orientation by updating the file directly (PhotoStructure uses exiftool under the hood), but it's not the default out of concern for unknown bugs that may invalidate the original file in some way.

By default it just writes the new orientation to an XMP or MIE sidecar. The downside of this approach is that most applications don't respect sidecars.

> That videos sometimes use "Rotation", not "Orientation" (and Rotation is encoded in degrees, not the crazy EXIF rotation enum)

Not technically degrees. MP4 encodes it as a matrix. (Now, there are matrices corresponding to degree transformations...)

I haven't seen that: can you give me an example?

(The output of `exiftool -Rotation example.mp4` would be great!)

It's been a long time, but I once wrote code to parse and rewrite ['rotate'] the matrices at a previous job.

The "Matrix Structure" is in the 'mvhd' atom.

See:

* https://developer.apple.com/library/archive/documentation/Qu...

* https://developer.apple.com/library/archive/documentation/Qu...

[Some of the best documentation for MP4 that I've seen on the web is from Apple, since it grew out of QuickTime.]

This has caused me quite a bit of trouble with my image gallery as well, I've taken to just using exiftool to remove all rotation from my photos, which breaks half of them, and then manually hard-rotating them using ImageMagick, which is technically not a lossless operation if I'm understating it correctly.

I really don't understand why it was decided that most photo viewing applications would honor EXIF rotation, but web browsers would not.

Use jpegtran, it can rotate jpg images losslessly.

https://linux.die.net/man/1/jpegtran

Make sure you rtfm carefully.

Many years ago I accidentally deleted all the metadata from many images that needed rotating.

> -copy comments

> Copy only comment markers. This setting copies comments from the source file but discards any other data that is inessential for image display.

> The default behavior is -copy comments.

Argh! Thanks! So also my "lossless" transformations .. weren't.

For those interested in how EXIF is treated in other apps, a dated but informative article from 2012: https://www.daveperrett.com/articles/2012/07/28/exif-orienta...

They should just put a gravity vector in the image.

I’m pretty sure iPhones do put it in. You should exif dump an iPhone XS image. INSANE what’s in there.

Go on...

Might as well put in acceleration and rotation while we're at it.

Pitch and yaw

We can use both sets of names, better yet, mix them!

Ah, the famous ropitchaw vector!

You're joking, but many smartphones encode gyroscope readings in the metadata.

And uptime seconds (!?)

And estimated distance to subject, AGPS information, GPS acquisition time, depth field metadata, current battery level, operating system version, ...

This would (maybe) help with a common thing my wife does when taking videos: Start recording in portrait mode, then realize you did that, and rotate the phone 90 degrees to get widescreen video (but without restarting the recording).

When you play it back on a phone (with auto-orientation mode on), starting from holding the phone in portrait mode (as you normally do):

* it starts playing back as portrait, which looks fine

* the video rotates (because the camera was physically rotated), so now you're watching a widescreen video that's 90 degrees off

* Your natural reaction is to flip the phone 90 degrees to make down "down" again, but this changes the phone into widescreen mode, and because it thinks it's playing a portrait-style video, it changes to portrait-in-widescreen mode, and now the video is again tilted 90 degrees but 1/3 the size with huge black bars on either side

If you play it back on a computer/TV, you get the same end result: a widescreen video that's rotated 90 degrees, and 1/3 the size with huge black bars on either side.

Can't help you with the video-taking technique but when playing back the videos, if you hold the phone so that the video matches the screen, then rotate it so the screen is upwards, you can then spin it so it looks the right way up as long as you keep the screen vertical enough.

In these cases I just enable rotation lock.

Haha! I guess one has to cut the video and rotate manually one part?

Won't help when filming in the ISS. Microgravity or not, I would like to view the result in the same orientation as the crewmember who filmed it.

And how would the camera know without gravity?

“You can’t trust that bag of bits.” That made me giggle.

Have you submitted a post just about PhotoStructure yet?

I've been looking for a replacement for Google Photos. It's the only thing keeping me on Google products at this point.

Shameless plug: JPEG Autorotate 3 allows you both view orientation of the raw jpeg data and preview how it looks with orientation data applied. :)

https://news.ycombinator.com/item?id=21218596&fbclid=IwAR0uH...

Wow, your description of the problem of photo organisation is exactly. I've signed up for the beta!

Reply on news.ycombinator.com