AutoFlip: an open-source framework for intelligent video reframing

> AutoFlip’s configuration graph provides settings for either best-effort or required reframing. If it becomes infeasible to cover all the required regions (for example, when they are too spread out on the frame), the pipeline will automatically switch to a less aggressive strategy by applying a letterbox effect, padding the image to fill the frame.

Looks like that was particularly demoing the "best effort" mode. If you see the demo near that paragraph, it highlights the possibility of having certain faces cut-off unless you ask for all faces to be required.

They cover this on the what's next section:

> Like any machine learning algorithm, AutoFlip can benefit from an improved ability to detect objects relevant to the intent of the video, such as speaker detection for interviews

Yeah, I noticed that. While it's possible this is the result of the machine learning algorithm implicitly picking up a male bias in the data set that causes it to rank men more highly in terms of importance, I don't know how likely that is in practice. Maybe the explanation is as simple as the man taking up more pixels, or a bias toward lighter colors (like the man's white & blue shirt) while black colors (like the woman's clothing) are more likely to be considered unimportant background. Maybe the algorithm considered both approximately equally important and the result was essentially a coin flip.

Regardless, I agree that it's not a great look when the caption on that example talks about how AutoFlip "detects the subjects of interest" and "avoids cropping off important visual content". It's definitely not sending the intended message.

Pan and scan was historically done manually. Turner Classic Movies made a great 5-minute documentary about the practice and how much of the filmmakers' intention it loses. Definitely worth a watch: https://www.youtube.com/watch?v=5m1-pP1-5K8