JSMpeg – Decode It Like It's 1999
jsmpeg.com
jsmpeg.com
Particularly, I once needed to stream video from a UAV to multiple tablets and phones on a local network to allow collaborative annotations. Since it was a disaster response system there was no depending on external services, which at the time put WebRTC out of the picture (all of the easy to use implementations required internet access to use existing signaling services).
We ended up using MJPEG and then later a JavaScript implementation of a MPEG-1 decoder. This library certainly would have made my life a little easier at the time!
See: https://github.com/cjb/serverless-webrtc
That's a good rough demo of how WebRTC connections can still be established with the ask/offers being conveyed out-of-band.
To make it a little more friendly for tablets (and to accomplish before messages expire) I'd think QR codes would be a reasonable way of passing the data without depending on an external service.
There could also be some extraneous information that can be stripped to save on the amount of data you pass between peers so that the QR code isn't excessively gross (see: https://webrtchacks.com/the-minimum-viable-sdp/)
I built "drawing board" a while ago using an old school image map streaming a motion jpeg. You can interact with the video without ANY JavaScript [2]. The browser stays where you are on posting because the server returns an HTTP 204 - also highly underutilized in my opinion.
1. https://github.com/donatj/mjpeg-php/blob/master/mjpeg.php
Out of video containers, QuickTime introduced "Motion-JPEG" [1] with each frame in the container being a JPEG, and there have been formal definitions of how to communicate "JPEG-compressed video" with network protocols, like in RTP [2].
Meanwhile mixed-replace was a Netscape trick [3] to do server push with HTTP that never became a real force but was reimplemented by almost every browser trying to stay Mozilla-compatible (except IE), and happily used by webcams for dead-simple videostreaming.
[1] https://developer.apple.com/standards/qtff-2001.pdf [2] https://tools.ietf.org/html/rfc2435 [3] https://web.archive.org/web/19981203153836/fishcam.netscape....
It sounds rather interesting.
We were examining how the different individuals involved in a disaster response could best communicate their needs to the pilot of the UAV without causing significant cognitive burden on the pilot.
We explored using sketching, virtual spotlights, and audio communication. The UAV's video stream and sensor data was pushed out to client devices where users could apply annotations to the video stream that would be reflected across all devices. The pilot would see these annotations unless they toggled out of the collaborative mode.
Hardware-wise, we used a little ARM SBC (ODROID-U3) hooked into the UAV's existing piloting system to serve everything either direct to the client devices or back to servers in our mobile response lab. I believe we used Nexus 7 tablets for the UAV pilot and iPads for responders, but responders could also use their own devices if they preferred.
[1] http://digitalcommons.unl.edu/cgi/viewcontent.cgi?article=12...
I recently worked on a personal project which had to play back .webm files, and I used a similar utility:
https://github.com/brion/ogv.js/
It decodes .webm files and plays them in the web. I believe it's also used by Wikipedia to play bag Ogg files.
We had a ridiculous amount of assets(mostly animations) that had to be compressed, because one of the sales representatives noticed, that it's impossible to play any of the games if you're connected to a 2G network.
Eventually we didn't go with this solution, because it considerably reduced battery life and made the devices heat up too much.
If you're interested in this sorta thing, try taking a look at Broadway JS: https://github.com/mbebenita/Broadway
Here's a simple demo page they have setup: http://mbebenita.github.io/Broadway/foxDemo.html
Its entirely possible to use WebSockets to stream H264 to a browser and decode using broadway, and the performance is pretty good, even on mobile.
Edit: here is jsmpeg's author talking about it:
> There's been an experiment, called Broadway.js, which tries to decode H.264 in JavaScript. And there's some demos available, but I haven't been able to make this work consistently. It's very flaky. It tries to decode different stuff, and different threads And it barely works, if it works at all, so-- and you have to download, maybe, one megabyte of JavaScript for this. It's all part of EM script. [sic — emscripten?] And it's-- yeah, it's very complicated to get working, which is why the MPEG1 form of this is so nice for this, because it's so simple. And you end up with a decoder that's 30 kilobytes in size.
https://fronteers.nl/congres/2015/sessions/jsmpeg-by-dominic...
http://phoboslab.org/log/2015/07/play-gta-v-in-your-browser-...
Thor has been merged with Xiph's Dalaa into IETF's NETVC effort, and both Cisco and Xiph are backing this.
NETVC is a next generation codec designed to replace H264, H265, and all future MPEG codecs with a system that is not user- and developer-hostile wrt licensing lock-in via (possibly invalid) patents.
I also worked from just the reference book, with no prior knowledge of video coding at all, which made it quite a puzzle to get something on the screen and moving, but it was extremely satisfying when it all worked (to some extent, the thing was horribly slow, broke after one group of predicted frames, and I never implemented chroma, just luma)
[1]: https://sourceforge.net/projects/javampeg1video/PS: That video has a very late nineties, early double-ohs feel to it indeed. Good choice of video :)
They probably stop the video as a workaround.
> pauseWhenHidden – whether to pause playback when the tab is inactive. Default true. Note that browsers usually throttle JS in inactive tabs anyway.
Also, Chrome puts a little speaker icon on the tab, so users might notice.
and if you're trying to run down a user's battery I can think of better ways.
I can see it would probably be possible to get low latency, but without a fancy server stalling connections till frames become available, flushing on frame boundaries, etc, I can't see it working. Try doing such things with a CDN like cloudflare to support lots of users...
Also, for true streaming, timing should be done by the camera. Ie. If the cameras frame rate is 0.0001% slower than the 60fps advertised, the display device should slow down to match.
MPEG-DASH has no ability to do this, and would lead to a "buffering" gap for a second every few hours of playback time.
In realistic cases something like Lagarith was traditionally worthwhile simply because you couldn't read raw video from disk fast enough. I don't know whether that's still true in the age of SSDs.
The older meaning is obscure, largely redundant, and doesn't really make any sense etymologically, so it's not really surprising that the newer meaning caught on.
(And since we're being pedantic, it's got nothing to do with grammar.)
if the message is getting through, that's what matters in my book :)
That's just useless. And you know, saying "it once was A, now it's B, so it can never be A again" is exactly as "prescriptivist".
Little things add up, and before you know it people are just stringing words together as demonstrated on a million youtube videos.
> The older meaning is obscure, largely redundant
How so? How is the "new one" (which one? heh) not redundant? If you want to say something raises a question, that's an easy way to put it right there. On the other hand, I'm not even convinced that the bastardization of "begs the question" into "raises the question" wasn't simply based on not even understanding what assuming an initial point even could be, of just hearing the phrase without understanding the context and using it as another way to say something or someone raises a question. I certainly don't hear it in common usage, regardless of the phrasing used. You know, if all those other people say they just say it because "most" people do, then none of them actually do have a reason. A billion times zero is zero.
And don't even get me started on people suddenly calling low framerates "lag" :P It just destroys information, and you can call it progress because the hands on the clock moved a little, but I won't.
Except I'm not telling you what you can say, I'm telling you what other people do say, which is descriptivist. You can use meaning A if you want to, but don't expect people to understand you.
>You know, if all those other people say they just say it because "most" people do, then none of them actually do have a reason.
That's like saying no one in the US has a reason to speak English, they're doing it just because everyone else does.
Language is about shared understanding, so most people around you using a particular meaning is pretty much the only reason for someone to use it.
>you can call it progress
I don't call it progress, just change.
Begging (for) the question is a correct literal usage of those words, under both prescriptivist and descriptivist definitions of the words. You're allowed to use the same sequence of words as an idiom in other ways. I can talk about a hot potato without metaphor, and I can talk about begging a question when not describing a fallacy.
It's not the cheapest to encode, but even a baseline H.264 low cpu use profile will beat MPEG-1 or MJPEG anyday anytime on bandwidth and quality.
Love it
I´d like to know how reverse playback could be achieved. Is there a way to "undo" a P-Frame calculation, or is a second (reverse) mpeg file a possible solution?
I was pleasantly surprised with the quality of the output and speed of which the devices could decode and render the new output.
It's a nice hack to get this kind of video on the iPhone, but somehow I feel that for decoding video JS is a bit high on the stack.