Canvid.js – tiny library for playing video on canvas elements | Hacker News Reader

Canvid.js – tiny library for playing video on canvas elements

gka.github.io

66 pointscallumlocke30 comments

Same demo video as mpeg1, 243kb, decoded in JavaScript: http://phoboslab.org/files/jsmpeg/cat/

Longer demo: http://phoboslab.org/files/jsmpeg/

What are the trade offs in terms of size and processing required?

Decoding an MPEG1 video is of course much more taxing on the CPU than scrolling through a large image, but low-res videos shouldn't be a problem even on older mobile devices. jsmpeg uses WebGL to speed up the decoding process a bit. On an iPhone5S, it can decode 720p video in realtime[1].

jsmpeg only unpacks the current frame into memory instead of decoding the whole video at once, so it's memory footprint should be much lower. MPEG1 files are also much smaller than storing individual JPEG frames.

[1] http://phoboslab.org/files/jsmpeg/benchmark-720p.html

What about for a short loop? Can it cache some frames so that could be fast?

>On an iPhone5S, it can decode 720p video in realtime[1].

While killing the battery while doing it... Why we don't just expose lower level decoding apis to JS instead, I don't know...

because of how easy it would be for the already bloated ad laden mobile web to abuse those powers, and how seriously obnoxious, bandwidth eating, and battery killing the results would be.

To ...abuse the powers of having access to an API for decoding video streams?

As we've already established, this can already be done, badly, with native javascript. So adding that access would add nothing new to "abuse" or "eat bandwidth" than what we already have, and will induce even LESS bloat than the current situation.

As for merely playing movies, ads etc (without direct access to low level decoding APIs), obviously javascript can already do that with the HTML5 media api + native encoders. So nothing new here either.

So I can't tell what disastrous "results" you're referring to stemming from what I propose.

Giving JS access to low level movie decoding/encoding would just be something to use to make creating JS-based video editors and such easier and more performant.

You can do it, but as you can see, it's not very efficient and eats battery. nobody would actually put this "decode video in javascript" stuff in an ad, as it stands. Making this stuff work well is hard.

make it efficient and easy, and you change the economics of the situation to encourage abuse.

>You can do it, but as you can see, it's not very efficient and eats battery. nobody would actually put this "decode video in javascript" stuff in an ad, as it stands

No, but they can already show any arbitrary video they want by using Javascript to add any number of videos playing on the website -- that's just plain vanilla HTML5.

Adding hooks to low level decoding doesn't add anything new that they can't already do when it comes to showing ads. For ads they only care for high level decoding.

It only adds something new when it comes to creating video mangling apps in JS.

that is on desktop. It is currently not possible to autoplay video on an iphone, and that is in fact the topic of this thread, and (partially) the purpose of this library.

Beyond autoplaying, I can't imagine what you mean about adding "hooks" to low level decoding. What does that actually gain you over the current situation?

>that is on desktop. It is currently not possible to autoplay video on an iphone, and that is in fact the topic of this thread, and (partially) the purpose of this library.

Aha, I see, thought that wasn't the case.

Now, having that capability didn't turn out to be much of a problem on the desktop, did it? I mean, regardless of battery concerns etc, do we really see that many autoplayed videos?

>What does that actually gain you over the current situation?

The ability to decode individual frames, re-arrange them, add filters to them etc using accelerated native APIs -- much more than some "play()", "skip()" affair.

well actually the ability is a huge and obnoxius problem on desktop, particularly for accessibility. there are lots of autoplaying videos. i do not know what web you are on where there isn't. I'd be happy if the deaktop browsers turned off autoplaying video and audio. or at least add that as an option for me, the user who has to jump for the mute button every time some jerk developer thinks they are being clever. aside from that, mobile is a different story- I doubly don't want a loud car salesman voice telling me about his get rich quick scheme on the train ride home- eating my precious mobile bandwidth, costing me real money.

as for your special api abilities, again, you can do all those things now, even on iphone. so again, what more do you actually want?

Here's the massive jpeg sprite they use: http://gka.github.io/canvid/images/canvid-example.jpg

Jpeg is massive

The world is in trouble

Ah what we tell dem murdera

There is also cssvid (https://github.com/AndyTheGiant/cssVid). It does the same but with background positioning instead of drawing on a canvas.

Why use javascript though? Here's my crude attempt at a CSS only implementation (Using CSS3 animations).

https://output.jsbin.com/qocefivowa

Nice! Would love to see a benchmark

awesome! I would like to see a performance test with canvid, cssvid and your solution :)

You mention you're limited to short videos due to issues with the maximum size of images on ios. Why not just generate multiple images? That way you could load the video "parts" in parallel as well.

On iOS Safari you're limited to 2 parallel HTTP connections at a time so not a huge advantage to the multiple connections. Don't disagree with splitting it up though

Yep! If you have a long video, its better to split the sprite into multiple parts.

> Because you can't embed and autoplay HTML5 videos on iOS! Yeah, that sucks.

Are there other legitimate use cases than using this a fallback for iOS? Curious.

Could be interesting if you wanted to do client-side filtering/modifications, but a lot of that is probably possible with video files (and maybe WebGL) as well.

Pretty fun experiment. I wonder if just having the big sprite sheet in a div and positioning the image with -webkit-translate would perform as good.

How does this differ from Apple's launch of iPhone 5 website?

Previously: https://news.ycombinator.com/item?id=4531088

Teardown: https://docs.google.com/document/u/1/pub?id=1GWTMLjqQsQS45FW...

Apple's tech dedups 8x8 pixel blocks and creates a jpeg image that is a lookup table of 8x8 pixel blocks, using a json data object to figure out which blocks go where on each frame. this thing just stores full frames in a giant jpeg and displays them.

Nice, I did something similar few months a go. http://stanko.github.io/html-canvas-video-player/

Very cool! Good solution to how gifv's pause background music on iOS too.

if you wanted to replicate Apple's technique, there's a neat (but discontinued) app on the mac app store called phosphor that will take in movies or frames and output in the apple-like canvas/json/jpeg 8x8 pixel blob format.

https://www.divergentmedia.com/phosphor

Whether you actually gain anything from using this technique as opposed to other simpler techniques like canvid, well, let's just say YMMV

Ha, just spent some time figuring out how to do this myself -- going to take a look and compare. Sounds really neat!

Reply on news.ycombinator.com