The Soap Opera Effect and how to make it go away | Hacker News Reader

The Soap Opera Effect and how to make it go away

hometheaterreview.com

46 pointsd99kris43 comments

It's nice to see a piece on this subject that's friendly, non-technical, and non-judgmental.

That said, there are two reasons I never watch movies with frame interpolation on: One is the old artistry argument, that the movie was shot and cut at 24 fps and anything else is a distortion of how it was intended to be seen, but the other is one I don't see much: Most interpolation algorithms are highly inconsistent.

I think I would be considerably less distracted by frame interpolation if it didn't tend to fail completely on fast motion. It's less noticeable in dialog-heavy films than in thrillers, for sure, but even a character gesticulating as they speak can trip up the most advanced 240hz interpolators, throwing a few immersion-breaking frames of judder into the otherwise silky-smooth movement.

I don't think this is really a solvable problem (outside of exotic computer vision systems, at least) – at 24 frames per second, there's often simply not enough visual information to go on. For now, only the unmatched power of your own visual cortex can fill in those gaps.

On the flip side, I quite enjoyed the aesthetic of The Hobbit in 48fps. Produced and mastered at that frame rate, it looked great.

> I think I would be considerably less distracted by frame interpolation if it didn't tend to fail completely on fast motion.

I just got a new (to me) TV and realized it was interpolating. I'm not too bothered by the soap-opera effect in general. But the first football‡ game I watched on it made me turn it off in a hurry. The ball seemed to disappear when it was passed. The motion smoothing algorithm somehow ghosted it right out of the picture, so that it appeared to leave the QB's hands and magically teleport to the receiver.

‡ Gridiron Football

For anybody else that might otherwise have to look it up: Gridiron Football is American Football.

The name comes from the layout of the pitch, which looks nothing like a gridiron (which appears to be a type of portable grill). There's probably a joke in there somewhere.

It looked like a gridiron 100 years ago. Field changes, name stays the same.

https://en.wikipedia.org/wiki/Gridiron_football#/media/File:...

I also dislike unreliable image processing algorithms. Consistent minor artifacts are less distracting than occasional major artifacts. It's not only interpolation which has this problem, but also deinterlacing. Fancy motion-aware deinterlacers fail in the same situations interpolators fail. For this reason my favorite deinterlacing filter is the Weston 3 Field Deinterlacing Filter (w3fdif in ffmpeg). It doesn't try anything too clever so it never seriously fails.

> One is the old artistry argument, that the movie was shot and cut at 24 fps

But (and correct me if I'm wrong) -- as I understand the article, the judder is an artefact of how 24fps is re-sampled to 60fps. The article didn't touch on it, but what does this look like in a cinema-presentation? The article seems to strive towards "60fps truth" (i.e. keep the judder), where "cinema truth" seems to be the relevant standard? Which display mode is more cinema-like?

The interesting question is whether the dislike for HFR cinema is:

- Purely due to habit, with people having learned to associate LFR with high quality content (film) and HFR with low quality content (video). In this case, the problem will go away as soon as people get used to HFR (or die off).

- Due to the increased realism exposing sets, costumes etc. as fake. LFR may help filmmakers by providing a distraction that masks all the little flaws. In this case, the problem will go away (but only for new movies) as soon as filmmakers learn how to make things look good enough for HFR.

- Due to some intrinsic quality of LFR that is key to the magic of cinema. In this case, the problem will never go away.

Of course, people are immediately going to seize on one of these answers based on their personal preferences, but in reality we have no concrete evidence for any of them.

I remember reading about how actors hated the switch to video and then the switch to HD, because it was far more likely to reveal their acne, other flaws etc.

In audio engineering, the Yamaha NS10 studio monitors (now discontinued) are famous for their faithful, naked reproduction of sound. As a result, more mixes sound terrible on NS10's than on other speakers. They expose the flaws. Which is why an engineer tests his mix on them. "If it sounds good on NS10's it'll sound good on anything."

The 3:2 pulldown judder that the author grew up watching is unrelated to the "look" of 24ps. The soap-opera effect is purely a frame-rate issue.

It's also unrelated to the artifacts resulting from "motion interpolation". Again, it's purely related to frame-rate. High frame-rates look real, and therefore like soap-operas. 24fps content (e.g. most films) look "unrealistic" which is a good thing. In the same way a van Gogh painting looks "unrealistic". Impressionism is actually a valuable art form, and is actually appealing to the brain. It's not something that needs to be "engineered" away as Peter Jackson thought.

Only an engineer would look at a masterpiece of impressionistic painting, and think "Hmm, if only the painter used a finer brush we'd get more detail! Say this painting could really use some up-rez'ing."

24fps video is precisely "impressionistic". It's distinctly not what our brain perceives in the real world. It's what makes movies look slightly "other-worldly". And that's a GOOD THING. In fact it's one of the reasons it's been such a hugely successful art form for 100 years. Impressionism is a fantastic form of art, whether you're talking about broad brush strokes, or low frame-rate video.

Can we all stop (looking at you Peter Jackson [1], Ang Lee [2], and James Cameron [3]) wishing van Gogh used a finer brush?

[1] http://www.thehobbit.com/hfr3d/faq.html

[2] http://variety.com/2015/film/news/ang-lee-shooting-billy-lyn...

[3] http://www.hollywoodreporter.com/behind-screen/could-avatar-...

The transition from film to digital was a bit jarring at first, as well, but after acclimating over the years, movies shot on film look a bit old-timey. Who is to say frame rate isn't a similar issue? Other-worldly 24fps could just be what we are used to and thus temporarily comforting.

I disagree. I do look at impressionistic paintings, and wish they looked good on large high-definition displays. I am not sure why you think a finer brush would destroy the impressionistic qualities of artwork. That seems like saying we've lost our appreciation for good lighting, because we use LED light bulbs instead of wax candles. Artists can easily canvas on 4k, 8k, or larger resolutions with modern tools, and achieve vistas that will outshine most previous artwork in fidelity.

I'm not trying to compare contemporary digital art to pre-modern paintings, because there would be no point. But we can compare the quality of the canvas medium, the fidelity of the work. I am certain anyone who has spent any amount of time creating a digital college, set of high quality wallpapers, etc, has noticed that pre-digital artwork is often vastly lacking in terms of resolution. This contrast is difficult to notice in a museum, where artwork is displayed alongside its peers; it is obvious when comparing it to digital styles of art.

There is nothing wrong with wishing for more detail. You can have impressionistic artwork in 16k resolution, and enjoy it too.

> as noticed that pre-digital artwork is often vastly lacking in terms of resolution

Non-digital artwork has infinite resolution (well, down to the Planck length, I suppose). The texture of the painting surface and of the brushstroke is relevant in the creation of art. In fact, you are demonstrating exactly what the previous poster said: impressionism consciously chose to omit the finer details that other schools included as a stylistic decision, but that lower level of detail in the depiction does not imply a lower level of fidelity unless your only definition of fidelity is accurate depiction of an object.

Huh? A painting has no inherent resolution that means anything in the context of digital displays. Are you talking about the resolution of photographs of paintings?

I notice the very wide depth of field of a typical video camera much more than the frame rate, which does not bother me much. Even on a static scene I can tell a TV camera from a film camera by the lack of depth in the TV picture.

You could make a similarly stupid argument against talkies or color film.

Yes, there is still room for low frame rate movies and black and white pictures, but there are plenty of special effects that audiences are missing because studios aren't willing to invest in high-frame rate pictures. And the reason they are afraid to invest is because they've been stuck on low frame rates while computer and TV have actually advanced and now they think their backwardness as a hallmark.

Imagine if TV had converted to color decades before cinema and then people claiming that colors would break the "other-worldly" look of "expressionist" cinema.

Am I in the minority in finding Judder very distracting? And film grain too for that matter? Or overly dark night scenes?

The better my monitor the more distracting those things are - I never used to notice them on CRT's. But with new high resolutions screens film grain is very noticeable which is unpleasant - I don't see film grain when I look around, there is no reason I should see it in a movie.

There is actually a lot of stuff your brain automatically filters out when you look at things.

http://www.nhs.uk/conditions/Floaters/Pages/Introduction.asp...

I find it strange insisting on "flickering" screens. No "artist" desired it (as in "let's make this movie, but it has to flicker!") it was always just the limitation of the technology. If you'd want the "whole original effect" you'd have to insist on the 35mm films and their projection in the movie theater. The only thing that is true is that with the limitations of the technology, some "corny" scenes looked more acceptable, as our brain actually got less information and in more unnatural form so then our brain turned on its "more tolerance" mode while watching, reimagining the scene better than what it saw.

Also:

http://www.projectorcentral.com/judder_24p.htm

On another side, what I fully understand is that the older frame interpolation algorithms did very poor job for moving scenes, so people turned it off because of that, but it seems that the algorithms got noticeably better.

The artist created something for the limitations of the technology they were working with. It's unfair to look at it through a different technology, where careful effort will end up looking worse than someone who threw it in. E.g. good pixel art often looks worse when upscaled than sprites where the artist didn't pay any attention to the pixels.

You know the sound of very old records, most people today can't listen to them. Now, imagine there were technology to make them sound clear, and that therefore you hear errors by musicians. It can break the illusion you had about the music of that particular record, but it actually reflects more exactly the event that was recorded.

It's the same with the films. With our new algorithms, what was once flickering "action" suddenly becomes "some people doing something that is obviously in studio, and wow I can see how sloppy their make-up is too." Which is something that we can more easily see even in the movies actually fully produced with the modern equipment and from the start with the higher frame rate: we see them more like they looked like at the set where they were filmed (not counting the CGI that now tries as hard as possible to remove or modify that).

So it is somewhat the opposite of your pixel artist example: it's that the production then didn't have to care for the details we easily see now as they were unnoticeable because of the limitations of the presentation then.

An exception to that "we don't have to worry about the details" approach: Kubrick producing 2001: A Space Odyssey. Looks great on Blue Ray with as many frames as it's possible, as it was produced to look good in high resolution. And today it would be extremely hard to organize that projected "as it was produced to be shown" (Super Panavision 70, using 70 mm prints). Other examples welcome.

This explanation is wrong. The effect is very visible also in Europe where tv is 25 fps and no 3:2 pull down is performed to 24fps material.

The soap opera look came from the video camera which operated similar way as video playback. Although the systems run 25 or 30fps they show each frame in two interlaced fields. This meant screen was updated 50 or 60 times per second, but showing either the odd or even lines of the frame at once.

Video was recorded in similar fashion so each field was captured with half a frame offset in time to give it effective frame rate of 50 or 60fps even though this meant actual frames would look funny on noninterlaced displays.

Film was transferred to video by taking both fields from one frame or with 3:2 pull down in 60hz countries. This made the effective frame rate closer to 24 fps like it was intended to be shown in cinemas.

[1] https://en.wikipedia.org/wiki/Interlaced_video

edit: added link

I used to hate it until I just left it on for a month or so on my new Samsung 4k. Eventually you get used to it and move on to more important things in your life.

You can get used to living in filth. That doesn't mean it is desirable.

Indeed, one can get so used to it that they prefer nothing else. Change isn't always bad, and maybe this will let you appreciate actors in a whole new light.

Once you are used to it you realized how much your brain was wired to see 24Hz as "movie-like". You basically just unlearn that connection and enjoy stuff no matter what the framerate is.

Hey what don't kill you, make you stronger.

Like actually, you may get fewer allergies if you grow up playing in dirt.

As if most of what's oozing out of your TV set wasn't filth. [1]

[1] https://www.youtube.com/watch?v=nRnNDkHb0MU

I have friends that always have it active on their TV, and claimed not to mind it. Having watched a few films at their house, I came not to mind it either... at least, not to any distracting extent. Still prefer having it off though.

(One funny thing about it is that it often really shows up CGI sequences, the comparative lack of noise apparently giving the interpolation more to work with than usual. The results can be either slightly comical or pleasingly eye-popping.)

I wonder when people will actually start to make films that have to be seen at high frame rate. And I wonder how long until I meet somebody who's grown up with motion interpolation and thinks things look funny without it.

>(One funny thing about it is that it often really shows up CGI sequences, the comparative lack of noise apparently giving the interpolation more to work with than usual. The results can be either slightly comical or pleasingly eye-popping.)

I've noticed this with anime in particular. Even though none of it's hand drawn these days it isn't done like 3D CGI in movies. It is, for the most part, designed to look two dimensional. The motion smoothing makes it look like you're watching a 2D diorama or pop-up book where the different cells are sliding on top of each other. I kinda like it.

This is a highly subjective topic.

Is it possible that Televisions are just doing a very bad job at interpolating? I've been using the SmoothVideoProject for over a year now and I can't watch movies/series anymore without frame interpolation. It's just to hard to follow what happens in very fast action scenes or enjoy camera pans across a beautiful, nay, any setting in 24/30fps.

Of course native 48/60fps is a lot better, but having a good interpolation is a start.

I highly recommend checking out the SmoothVideoProject at https://www.svp-team.com/wiki/Main_Page

I'm just glad to hear that evil has a name. I've bored way too many people going: "...y'know that smooth thing on big tellies..that makes all the old black and white movie actors glide out of rooms on rollers...you do know! You must know! Don't f##king tell me you can't see that... I'll be back in a minute... " and so forth

What's it called in VLC? I don't like the judder, would like to see how I get on with a bit of smoothing

I'm really glad I didn't wait to purchase my 600Hz plasma screen. Got it for an incredible deal and the image quality and color reproduction (dark scenes especially) are off the charts. These days other screens just can't compare and, as the article mentions, nobody makes plasma any more.

That 600Hz number is basically meaningless. There is no 600Hz motion interpolation being done; it's just an evil marketing trick.

That said, plasmas do have great color, but they also have motion trail problems like CRTs and severe risk of burn in.

Yep I'm well aware of burn-in issues. And the screen I have is just a dumb monitor; it has no interpolation feature so the 600Hz claim is what the display is theoretically capable of, though it's different from LCD refresh:

http://www.cnet.com/news/what-is-600hz/

Low frame rate is the reason I can't take film seriously as an art form. It seems to me utterly bizarre that people would choose a frame rate that can't show realistic action scenes. Fight scenes have annoying foley sound effects partly to compensate for the frame rate making fast movement invisible. And camera work is severely limited - fast pans are near unwatchable. I've seen 60fps Showscan film and it's such a huge improvement I can't understand how people could still argue for 24fps. It can only be "baby duck effect", where people like something only because they grew up with it. Imagine if music wasn't considered musical unless you listened to it from a wax cylinder. Technology has moved on and we can do much better.

I realize that cost was an issue, but with digital production that's no longer the case. And yes, it's true that higher frame rate exposes flaws in sets and props, but that's no excuse to stick with low frame rate. The correct solution is to improve those sets/props. 60fps should be considered the bare minimum.

Art isn't about the technology. That you can't take it seriously says nothing about the artists. Only that you don't understand it.

Deliberate limitations can facilitate art. The demoscene proves that. But what if the entire demoscene collectively refused to use modern hardware? Or insisted that only 1K demos are acceptable? Deliberate limitations are only one tool an artist can use. The film industry has collectively decided to do something equivalent to specializing in 1K demos and then denying that any other types could be any good. This does not look like a carefully considered artistic choice.

It's been tried. There's not just the "demoscene". There's Dogma.[1] There's a Fisher-Price PixelVision film-making cult. They're all dead or tiny niches. Probably the most limiting technique with a mainstream following is stop-motion animation.

[1] http://www.economist.com/node/328590

Is smearing charcoal across canvas art?

I find it bizarre that you classify the art of film by the action genre.

If you're shooting nothing but talking with static cameras then 24fps is adequate. There are some great films (eg. 12 Angry Men) that have no major frame rate problems. But this style is not a good demonstration of film, because it would work just as well as a stage play. I mentioned action genre because it's a better demonstration of the potential of film and it suffers tremendously because of the low frame rate.

you don't understand film at all

Reply on news.ycombinator.com