Mozilla Firefox Adds Support for AI-Powered Nvidia RTX Video | Hacker News Reader

Mozilla Firefox Adds Support for AI-Powered Nvidia RTX Video

blogs.nvidia.com

37 pointsmariuz9 comments

My first impression is that I'm not seeing increased resolution. It looks a lot like an overapplied sharpness filter (too much contrast around existing edges).

I gave this a quick try the other night and it looks absolutely awful with a 720p video on a 4k monitor. I want to try some more and play with it, but I think it needs a lot of work.

It basically looked like someone cranked the sharpness to 11, and then added some kind of weird blur filter at the same time. Similar to how smartphone photos look very heavily processed.

The Edge one is super gimmicky and makes the quality worse, but i think that is because it doesn't use RTX in some cases?

Does this then mean that there is now incentive for 1080p or 1440p streaming from video hosting websites to be degraded 720p streams and it's up to user hardware to do the upscaling?

Not sure if I'm for or against to be honest.

How is this any different than video services encoding their videos with HEVC/AV1 to save on bandwidth? They're lossy algorithms and use less bitrate, so it's not so different than what you're describing.

The upscaling costs are astronomical. My PC jumps from ~150 watts or so just viewing a video to ~400 with this running on my 3090.

That's ~250 watts/hr just to watch a video in slightly higher resolution.

No. It makes no sense to down sample the resolution you send. What matters with video is the Bitrate you are actually sending. What AI might to is allowing you to further decrease the Bitrate while still maintaining similar quality, but this is what every video codec does anyway.

It makes no sense to retroactively apply this and expect client up sampling to make it look good.

Unless...

If the technology became ubiquitous, in theory you as the streaming provider could save significant money on data transport fees by offloading an expected and approved AI upresolution workload to your customers.

They would pay for it with electricity and hardware expenses, which would far outstrip your transport costs, but we live in a proto-capitalist hellscape where saving $0.000001 an hour is worth slitting your grandmothers throat for, so it's not beyond the pale to assume someone has done this math.

Correct, but only if you assume that the processing is done on the server and not on the client. The grandparent comment isn’t saying that Youtube would be running AI upscalers on low-bitrate videos and then serve it to the user. They were saying that Youtube would just be ok with serving low-bitrate video content, if they know that almost all of their users rely on AI upscalers running by default in their web browser clients.

The headline literally says “Firefox on Nvidia RTX GPUs”, i.e., it is referring to the AI processing on the client. Upon reading the article, I confirmed that it was indeed the case, so the question in the grandparent comment is valid.

As for why, it is very simple - cost-cutting. Streaming high bitrate video is simply more expensive and complicated than low-bitrate video (both in terms of the actual bandwidth and the architecture/systems that handle it).

P.S. this relies on the assumption that clientside AI upscalers for video would be able to produce amazing results consistently and would be compute-efficient enough to run on an average user’s personal device. We are not there yet at the current moment, but it is imo a very real possibility that we might get there in the not-so-distant future.

Reply on news.ycombinator.com