Responsive scrollable videos without obscure video encoding requirements | Hacker News Reader

Responsive scrollable videos without obscure video encoding requirements

scrollyvideo.js.org

53 pointslovegrenoble41 comments

Looks like playback depends on "ticks" arriving from browser to the JS library, and library selects a frame according to the scroll location.

Mobile devices send this data constantly and periodically, making it smoother; yet desktop browsers do not, appearing choppy.

It works as intended on my iPhone X, yet stutters on an infinitely more powerful desktop, and only possible explanation is this.

Not to undermine your point at all, but as a tangent, you might be surprised by how powerful phone compute is getting these days.

The iPhone 15 Pro approaches the multi-threaded GeekBench score of my Ryzen 3700x (which, while not new or high-end, is still better than what 90% of desktop users have) and handily beats its single-threaded score.

I agree that the difference in smoothness is due to software design decisions and not computational power, but thought it might be novel to see how the gap between nubile and desktop has shrunk over the years!

Well, the devices I compare are a seven year old iPhone X, which has 20% of the performance of latest iPhone (in Metal API, at least), a desktop system which's equipped with a Radeon RX550 with 4GB of RAM, and an Apple MacBook Air M1.

Both of these systems are infinitely more powerful than my phone, and the latest iPhone possibly smoke that old AMD card, but not the M1, not by a large margin, at least.

Isn't the most important difference that the phone has a smaller screen?

That smaller screen on iPhone X has 2436x1125 resolution, which is only 33% less pixels than my desktop monitor (which is 2560x1440). Considering that the video plays full screen on the phone, and in a much smaller window on my desktop system, the inferior GPU can render that much more smoother than the desktop, which shows that GPU is not the problem here.

Also, grabbing the scrollbar on Firefox Linux restores the smoothness, supporting "tick" theory.

Be wary comparing benchmark scores between platforms. Different compilers for different platforms will make a massive difference to scores achieved, mostly due to the fact benchmarks typically do 'useless' work that compilers aim to detect and eliminate.

The compilers for iOS and OSX are largely the same and if you’re consistently using the same compiler family you’re probably ok even if the version isn’t completely identical (major LLVM versions aren’t frequently bringing massive changes to how the optimizer is working). Additionally the UB optimization stuff you’re referencing is all within a tight margin even across families (ie GCC and clang tend to apply very similar optimizations).

> Not to undermine your point at all, but as a tangent, you might be surprised by how powerful phone compute is getting these days.

... some phones, at least. Not everyone's on a state of the art flagship device.

Much more importantly for ARM phones video isn't decoded in the CPU at all, the SOC has video codecs built in. This allows things like all day netflix/hbo without draining the battery. AMD cpus don't have video codec coprocessors as far as I know. Not sure about GPUs though...

All of the big three GPU vendors (Intel, AMD, NVIDIA) support hardware video decoding and encoding onboard their GPUs for a very long time.

Intel and NVIDIA generally adds dedicated silicon for these tasks, and AMD prefers to use shaders for video tasks. They started doing shader based video processing in Radeon 8500 days, and they're very experienced in video processing, since they had AiO wonder cards which did also capture and even TV in some models. Trying to render analog signals nicely on a PC screen provided them considerable R&D and know how during that journey.

Correction: AMD has dedicated video encoder DSP called VCE[0].

[0]: https://en.wikipedia.org/wiki/Video_Coding_Engine

Small anecdote, I tried scrolling three different ways on a laptop. Two finger scroll, scroll bar on the right, and holding the middle red button and putting pressure on the trackpoint. the third appeared smoother than the other two. (in my mouse drivers, I tried setting the middle button set to scroll and middle click.)

It also appears all the desktop people in the thread reporting choppyness are doing so from OSX.

I both tested from Linux and macOS. Safari on macOS is same with the iPhone X Safari on smoothness. Firefox on macOS is same with Firefox on Linux, too.

FF updates when scrolling stops, or updates more frequently when you grab the scrollbar. So, it's how the library gets the scroll signal, not about video decoding performance.

Desktop is secondary in mobile first development

Most apps pretty much would benefit from being mobile first and offline first.

Yeah, this isn't an issue of compute power, the only explanation I can think of off the top of my head is the scroll ticks. There are potentially ways around this, the video could lerp its playback position instead of snapping it? Maybe that would impose too much of a delay though.

I also feel like I have seen smooth scrolling video examples on Firefox desktop, so I don't know that it's an insurmountable problem, either other solutions aren't relying on scroll ticks or they're using some method of making them come faster on desktop I would guess?

But performance/responsiveness are kind of the least of my concerns when looking at this, because sure it may actually be performant, but it doesn't feel performant if the video is constantly snapping around like this.

“Thus, to achieve maximum performance under all circumstances, it is still recommended to encode videos with keyframe = 1, if possible”

Which negates the title. Not using long-GOP encoding really defeats the purpose of using a codec like h.264/h.265. For delivering files via streaming, this is a pretty obscure thing to do. It’s no less obscure than using trick play techniques of encoding the video using long-GOP, but in multiple streams where each stream is at different rates. This is how the set top boxes would do the fast forward/reverse play.

It's basically a stream of jpegs - true!

Am I missing something? Performance on both Chrome and Firefox on OSX was pretty poor (or maybe the video just doesn't have enough frames to make it smoother.) Doesn't seem to respond to touch events, either...

Yeah, this was very jerky for me too (MacBook Pro, M1 Max) - really not a good visual experience, which is the point of these things presumably.

Weirdly it works great on my iPhone 15 pro, even with battery saver on, which it explicitly says will not work

I can’t think of a single time I have not been annoyed at scrolling video playback.

Not because of performance, though. I just think autoplay is a cheap trick to try to gain my attention, and if I’m scrolling, it’s because I didn’t want to see the damn video in the first place.

I’m always torn between wanting to kill auto play videos because they’re annoying and leaving them playing in another tab to drive up the annoying site’s bandwidth costs.

I was always stumped by this effect. Are we suppose to scroll consistently or with a big swipe? What if, I did not swipe strongly enough? Should I be ashamed I didn't make it to the end?

O the desktop it's always: should I be the phone instead?

It never felt right!

It’s the new “make it pop” demand from clients.

it is such a terrible design practice. wish it didnt catch on. but apple does it so its ok right??

Firefox has a setting to block autoplaying media, weirdly in the "Privacy" tab of Preferences. I also set "image.animation_mode" to "none" to disable animated GIFs. A couple videos manage to sneak through here and there, but that seems to catch 99% of autoplaying videos/animations.

This is more like a slideshow, than a video, on my M1 Max.

M1 here, the mp4 plays fluidly, the scrolling on the site is fluid, yet the video on scroll is missing so many frames it truly is a slideshow.

I am running on an m1 max too, works quite well in Edge and Safari. Scroll-Wheel, Dragging the scroller, all smooth.

I think this is great and can be useful for many UXs, but it needs a fallback so it isn't only reliant on scroll state.

I had the page load in the background, so the video didn't load, and I'm guessing when I brought the tab into focus a position event wasn't fired (or not how the library was expecting it), so I didn't get anything until I refreshed.

Yes, it is a bit jittery, and could use improvements, but nothing some lerping of positioning, etc probably can't solve.

In Firefox 120.01 on Mac OS the frame only receives an update once scrolling is fully stopped. I.e. if I use the scroll bar to scroll, the video is updated only on mouseup.

This page crashed my phone browser.

This might not be for every use case but definitely useful for some

Kinda choppy on Chrome/Win10, even after buffering the images. Smooth on Chrome/S21 FE, but only after it downloaded all the single frames.

Missed opportunity to name it scrollymcvideo.js

Very choppy on F-Droid Fennec and the video doesn't show on android Chromium, hopefully all users only use Chrome anyway

works well on Chrome on a Pixel 6 Pro. extremely janky and terrible in Firefox for Android on the same device. I recorded a small video to demonstrate. https://youtu.be/cK7KlcH1uPs?si=m3q7S9Ml0aS_IFoX

In response to other comments, works perfectly on Android WebView & Firefox. Actually pretty cool.

Firefox mobile? FF on my M1 Max mac this was very choppy

For me, it's pretty good in chrome on a Pixel 6. Absolutely terrible on Firefox on the same phone.

I use Vivaldi on all platforms and I would never see this on desktop because of the built in ad and blocker tracking. It did not load the video of the bridge until I specifically turned the blockers off.

I guess this must be the way that those terrible ads that appear to hide behind the text of a content farm article when you scroll, for my browser to block it outright.

That tells me this is going to be used by advertisers so I can't even get away from scrollable video ads because they're going to try to play as I scroll.

This will be great for advertisers since sites will be able to quantify how much of the advertising video was served to users based on how far down they scrolled!

This will be great for everyone's experience on the web. /s

Reply on news.ycombinator.com