Seriously.js | Hacker News Reader

Seriously.js

seriouslyjs.org

643 pointsAissen181 comments

Hi, I'm the person responsible for/guilty of this. Thanks for the comments, everyone. My sympathies for the BSOD.

Please keep the bug reports and questions coming. I'll try to fix/answer what I can.

This is awesome! I can't wait to see the mashups that spring forth from this (and hopefully make one or two myself). There will definitely be something really creative soon. Thanks for doing so much legwork on this.

Does it now or might it soon work with live video from the local camera?

Yup. You can see a live video demo with a sample of some of the effects: http://brianchirls.github.io/Seriously.js/

Although, both Firefox and Chrome have been a bit buggy for me with getUserMedia.

Try the Ascii Text effect.

Cool, thanks!

Couldn't get it to work in FF (Mac/22.0) - it asked for camera permission, camera activity light went on, but no images and constant re-prompts for permission. But, worked quite well in Chrome (Mac/27.0.1453.93).

You... you're the one! :-)

Nice work. I'm curious about where you got the video, was that something you recorded, or got from somewhere? I don't really work with video, but I'm sure I can find something useful for it in the future.

Nevermind, I found the video on youtube.

This is where I got the original, raw green screen video: http://okgo.net/2010/01/20/wtf-video-remix-project/

I kinda wish it was at a higher bit-rate. There are some jaggies at the edges of the hair and stuff that I had to cover up by tinting the color of the video.

This doesn't work for me at all on FireFox 22.0 on Windows XP. See here: http://imm.io/18sd9

Finally a demo that doesn't make me think "Wow, the web has really caught up to the 1995 desktop!"

Maybe if you're already an after-effects wiz this doesn't seem as magical, but it certainly felt like magic to me.

I've clocked many hours into After Effects, but seeing this in a browser and _interactive_ is amazing. Never would I have thought that JS and browsers would progress so much in such a short period of time.

People have been doing some pretty amazing things with WebGL lately: http://metaphysicaldeveloper.wordpress.com/2013/04/23/webgl-...

OTOH, it's mainly possible because computers are faster so Javascript isn't a slow language anymore, and the document model is just getting out of the way and letting you manipulate pixels directly.

This isn't manipulating the pixels directly. Javascript is still way too slow to do that, and I don't think that will ever change. The real work is done by the graphics card. The Javascript only uploads a frame to the graphics card, makes it run a shader and then displays the adapted frame.

The best part to me was that it was actually a pretty good music video, and the interactivity adds to it.

I thought it was just random goofing off for demo silliness. Was that a serious effort?

That was OK, Go (http://en.wikipedia.org/wiki/OK_Go) . The song is "WTF?"

They're famous for quirky video clips (original for this song is at http://youtu.be/12zJw9varYE). You should watch http://youtu.be/qybUFnY7Y8w . You'll like it. Trust me.

Ah, these are the guys who did "This Too Shall Pass"? I've been trying to work out who did this song - it's been infuriating me for ages!

Incidentally, you really aught to see how they make music with a car... http://www.youtube.com/watch?v=MejbOFk7H6c

It's OK Go's "WTF": http://www.youtube.com/watch?v=12zJw9varYE

It was filmed on a green screen for a different effect: Frame layering.

Am I easily impressed or is the instant play when you navigate back and forward impressive? Feels like switching TV channels.

You sir are thinking exactly the same thing as I am.

Really? Isn't this just a video with green screen plus shader? I don't understand whats impressive.

The technology used here to do the image processing is GLSL, in particular fragment shaders (aka pixel shaders). GLSL is a very small C-like language that's become a standard for GPUs. GLSL code gets sent (as a string) to the GPU by javascript via the WebGL API.

Seriously is a JS library for handling the boilerplate of WebGL, composing and compositing multiple shaders in a pipeline/graph, and adjusting their parameters. In addition it comes with a bunch of pre-written shaders.

Shaders themselves are a lot of fun to write, IMO. A pixel shader is just a function that computes a color given an input pixel location. (A shader can also take in additional input such as "textures" to sample from, e.g. a video frame.)

The shader is run with massive parallelization in the GPU. In theory every pixel can be processed simultaneously. This is how these effects can run in real time.

Here are some more examples of what you can do with pixel shaders (including sampling from the webcam in the browser -- should work in Chrome and FF), http://pixelshaders.com/examples/

I'm in the process of writing an interactive book about pixel shaders, http://pixelshaders.com/

https://www.shadertoy.com/browse (WARNING: it can freeze your browser)

Whose idea was it to actually run all those shaders at once? I guess someone just didn't want to store and host some thumbnails?

IN JS? This is so cool. Also: fun to compare the raw video to the original: http://www.youtube.com/watch?v=12zJw9varYE :)

How are they changing out the backgrounds in the tech demo? I was expecting the original to be a green screen background...

Click in the Raw button, you will see the green background :)

"Original" meaning the original finished released music video from 2009. The band released the raw green-screen footage later for remixing.

Respect for OK Go videos just went up a level. What an awesome idea!

Would also be interesting to see if they could recreate the effect of the original video in JS also.

Isn't the algorithm just, filter out the greenscreen, for each remaining pixel draw it to the buffer, draw the buffer to the screen?

That doesn't seem more complicated than the examples they showed.

No one said it would be MORE complicated than the examples.

"Would also be interesting" and "fun to compare" give no implication of relative difficulty. I would like to see the algorithm though so please post the code if you get it working in JS! That would be awesome!

It is actually slightly more complicated. The existing effects work in GLSL and are just transforming single frames of video. For that effect to work, you need the previous buffer states; this means that seeking will cause the image to be different, and you need to feed the previous render state back as a texture.

Just draw the frames to a 2D canvas, without clearing it, then back to the background buffer.

You still need to remove the green-screen. This either means an expensive process on the CPU side, or pushing the frame to the GPU to remove it with a shader and then using RTT and rendering that or reading the framebuffer and drawing that to a canvas. Note, it's really not difficult to do this, it's just slightly more complex than the other effects, is all.

The demo is already removing the green-screen, I don't think that's the most expensive part.

Well no, the demo is removing the green screen but only in the shaders. As I said, this really isn't a big deal, but it does add complexity.

I (very hackishly, but nonetheless) got it to sort of work:

http://imgur.com/a/PQX63#1

It's full of lots of (mostly green) noise because the demo's chroma key algorithm/parameters aren't that good, but it was fun to do!

Ah, their movements and actions in the video make a lot more sense after seeing the original.

> A real-time, node-based video effects compositor for the web built with HTML5, Javascript and WebGL

I for one am really impressed. While I'm not up to date with node.js, WebGL (or even HTML5 tbh), the boundaries that people keep pushing with web technologies is pretty wild.

EDIT: Reading more, I'm not sure if I am proud or embarrassed about my narrow focus over the last few years. I'm proud that I'm making a good living due to it, but disappointed that I haven't been able play with these things. Reading about WebGL [1], I feel like there is a new world to explore (for web programming).

[1] http://en.wikipedia.org/wiki/Webgl

Also note that "node-based" does not refer to node.js as per the FAQ: https://github.com/brianchirls/Seriously.js/wiki/Frequently-...

I completely missed that. Thank you!

"Sadly, we are unable to get Seriously.js to work on your computer. Sometimes WebGL gets a bit weird with certain graphics hardware and drivers. Please have a look here for more information."

Same here. Could it be because I am running Linux?

I got it working in Linux on Firefox. Chrome didn't work, though.

Working on Ubuntu 12.04 Firefox 21 and Chrome 27, AMD drivers.

Works for me on Ubuntu 13.04, Chrome 28(beta), Firefox 21, using Nvidia drivers.

you just need to enable it. it is disabled by default for most Linux OS / video card combos

It failed here, but without an error message. I can only see the first frame.

"Sadly, we are unable to get Seriously.js to work on your computer. Sometimes WebGL gets a bit weird with certain graphics hardware and drivers. Please have a look here for more information."

Yeah, same here, running Ubuntu on a Dell Precision laptop. Guess WebGL doesn't like my setup.

I coded WebGL a little over a year ago on a Precision "laptop" under Suse Linux. What I needed: (A) up-to-date Nvidia proprietary drivers (B) "de-blacklist myself" somewhere deep in the browser's hidden WebGL config page (I think it was about:gpu or some such back then in Chromium, but I'm not really up to date).

Yeah, I run Chrome on Ubuntu on a pretty powerful Lenovo T-something with an NVIDIA whatever optimus pain in the neck. The only way I can get it to work is to add --ignore-gpu-blacklist to the Chrome command line.

Check: /usr/share/applications/google-chrome.desktop (or something like that) You should see some lines that start with "Exec" where you can add that parameter to the end.

chrome://flags

override software rendering list

Works great for me, using Chrome on Ubuntu 12.04 on a Dell Inspiron 15R (Radeon 7700M) with the latest fglrx drivers.

Same message here (Ubuntu, Dell, Chrome). But it worked on Firefox.

I had the same message in Firefox on Windows, but it worked fine in Google Chrome.

Apparently with my Retina MacBook Pro with 10.8.4 and Safari 6.0.5, I "need to upgrade my video drivers", according to the site...

I can confirm this. On Chrome, though (same machine and same webkit, I guess) it runs smoothly.

Sadly, Safari still doesn't support WebGL by default, although you can enable the Develop menu in the prefs, and then enable it in the Develop menu.

Even if you do: it runs super slow on my Mac Pro.

Mac Pro running Mountain Lion with an ATI 5870 graphics card. Color me not terribly impressed.

Got the same thing for in Chrome. Did work for me in Firefox though.

Could someone upload a video of the result? not the finished OKGO music video please :)

Same here - FF 22.0 on Windows XP

I had the same deal with Chrome on my OSX Mountain Lion. Here's what fixed it for me:

1. Visiting chrome://flags 2. Scrolling down to Disable WebGL, hitting "Enable", and then "Relaunch now" 3. Repeating steps 1 and 2, but hitting "Disable" rather than "Enable".

Impressive! It's really amazing to see how OpenGL can enrich the web if it's used right. A few years ago I'd never have guessed shaders could be used in a browser.

I have little experience with Javascript projects, so can I ask why the name Seriously? Wouldn't a more descriptive name cover the contents? Also, why is the main source file 4K lines long? That's a little off-putting.

Good questions.

It's called Seriously for two reasons: 1) At some point, I had this idea that I would make all my Javascript libraries references to Homestar Runner. http://www.hrwiki.org/wiki/Seriously

2) "You can do that in a browser? Seriously?!"

As for the file size, yeah you're right it's too big. I'm working on a grunt.js build script, and when that's done, I'll break the main file into smaller pieces so you can pick and choose which utilities and things you need to load. Right now, it has the color lookup table and noise shaders, among other things, which are not necessary for everybody.

I'm going to think of it whenever I see something like "You can't do anything serious with JavaScript"

To be honest, the real work is done on the graphics card.

OK, but I think they should have written it in Go.

Curious, why would you think so?

The video is of the band OK Go.

You never know.

For a second I thought you were being snarky about Go. Then the penny dropped.

I see what you did there.

Note that this has been around since 2011, though with changes (still seriously cool, of course): http://web.archive.org/web/20111112185437/http://seriouslyjs...

Seriously. JS has been this amazing for a while. See also this demo from 2010: http://www.craftymind.com/2010/04/20/blowing-up-html5-video-...

Wow, I've never seen all 4 cores in my machine simultaneously spin up to 90% before. Nice trick.

You must be a better programmer than me.

If you write single threaded applications you cant go above 1 core and so will never hit this mark :)

fork disagrees.

As does pipe.

Rapidly cycling the processor affinity tries its best.

Say what?

It was weird timing because I was working on a multi-threaded application that was murdering my server (expected) when I made the comment.

Hm, that is strange. It does not even tax my CPU enough to kick it out of its lowest speed state, let alone tax any of the cores.

That's interesting. What kind of CPU?

Intel 3930k:

http://i.imgur.com/OF58RZl.png

After the peaks when it is flat at around 3% is when it is playing.

Intel announced a year or so ago they could JIT compile javascript and run it on their GPU. Maybe that's why?! If so, welcome to the bold new age of GPGPU and AMD's HSA... :)

Likewise, 5-7%, but I don't have a nice new machine like you do.

Core 2 Quad Q9950 and an nVidia Geforce 8800 GT 512MB.

That's because all the work here is being done through WebGL shaders. Your parent experienced a software fallback, I guess.

Hmmm.... My 2.5Ghz Core2 Quad is sitting at 15-20%. Compared to watching the vid on youtube at 5-10% utilization.

LOL, my old core2 duo has both cpus running over 80% on Ubuntu with a GeForce GTX260. But when I click on another tab, with the video still playing on a non-visible tab in Chrome, the cpus drop to around 20%. When I click on the video tab again, the cpu ramps up to the 80% mark again. Neat trick.

That's the nice thing about requestAnimationFrame. It's smart enough to not waste time updating if you're not looking at it.

As for 80%, umm... video driver issues? :-(

I'm running an i5 processor, and when the video was playing my cores jumped from the 5-10% range (background stuff I've got running) to the 25-40% range.

Interestingly, just switching from the demo tab to this HN tab (FF 21 on Ubuntu) drops the cores to the 10% range. They jump again when I switch back.

This is the sort of thing we're working on at Vidpresso. If this impresses you, and you want to build stuff like this... we should be friends. Contact info is in my profile.

We should be friends. Love the concept of Vidpresso; I woud have liked to try a demo but it required me filling out a form. :(

That's what I'm coding right now! :) We're reworking our marketing site.

We're already friends. Hi Randall!

Yo RH! :) So do you do rails or still just frontend stuff? And do you do like angular / other JS frameworks?

I'm a full-stack developer doing HTML/CSS/JavaScript/jQuery/PHP/WordPress/MySQL/Sysadmin stuff for pewresearch.org

Article about this on badass javascript from over a year ago http://badassjs.com/post/16583192105/seriously-js-a-realtime...

Now that's cool! Closest thing I've ever seen to fulfill the "instagram for video" prophecy

:::Takes week off to spin up an Instavideo demo:::

Isn't the green screen kind of required?

Perhaps something like this could be done: http://clippingmagic.com/

(discussion: https://news.ycombinator.com/item?id=5682831)

Something similar to that is possible for video. For example: http://www.youtube.com/watch?v=rGMqXBvYxog

I'm working on porting this over to Seriously.js so it should run in the GPU, hopefully pretty fast. The first crack at it was not so good, and now I'm experimenting with some new features that are only in Chrome nightly builds at the moment (and buggy at that). So look for this in the next 3-6 months.

the real-time-filtering would still be cool without replacing the background.

This is pretty awesome, but it seems like OK Go has cornered the market on "cool web tech" demo music videos. Doesn't anybody remember their collaboration with Google?

http://allisnotlo.st

I fully expect them to do something with Microsoft for IE11 to complete the cycle.

OK Go has cornered the market on "cool web tech" demo music videos.

Actually there have been some other very nice ones recently:

http://donottouch.org/

http://lights.elliegoulding.com/

Don't forget about Yung Jake! http://e.m-bed.de/ (seems to work best in Safari for me ymmv)

Don’t forget the Arcade Fire! http://www.chromeexperiments.com/arcadefire/

Seriously did nothing for me.

    Uncaught Not a valid HTML element: EMBED (must be img, video or canvas)

same here, I just heard some audio

    Uncaught Unknown source type seriously.js:2727

me either, except the music played, that's it.

i assume since no one else has mentioned this that it's just me: http://i.imgur.com/TdQCpsR.jpg

It sounds like you may have a browser addon that's creating an EMBED element as a fallback because the browser can't play the video itself.

If that's not the case, do you mind writing a bug issue on github with your browser details?

https://github.com/brianchirls/Seriously.js/issues

Thanks

@michaelmior you can use this to help identify your browser:

https://aboutmybrowser.com/

Running the latest Chrome on Mac OS X. (btw, realized my initial comment probably came off a little snarky. My apologies. From the sounds of the other comments, it's something incredibly cool!)

That's pretty weird. Is it possible you have a browser extension that's replacing the video tag with an EMBED?

Seems to work in Incognito. I'll let you know if I track down the extensions that's messing with things. Really awesome!

Looks like it was the HTML5 DivX Plus Web Player. Weird.

Oddly I got this in Chrome latest (Mac OS X) but it worked fine Incognito. Very impressive when it works.

A lot of extensions don't run in Incognito mode- sounds like an extension might be your problem.

Didn't really know what was I was looking at. Clicked a button. wtf-in-js-mind-blown

Great job!

I know next to nothing about graphics programming but how does this work? I don't suppose there is a Javascript loop that goes through all the pixels on each frame...

Nope. That would be too slow. I tried that a couple years ago before writing Seriously.js, and the big problem was copying around the memory. You have to first draw the video to a canvas, then you have to extract the pixel array from the canvas, loop through all the pixels and then draw the pixels back to the canvas. Now, for a high-definition video at 4 bytes per pixel (RGBA), you're looking at about 3.5MB of memory for each copy. And on top of that, it has to allocate and garbage collect that 3.5M pixel array for every frame, because it doesn't let you copy into an existing array. If you look at the first video-processing canvas demos, they tend to be pretty small, and this is why.

Instead, this uses WebGL/OpenGL. So the pixels are copied once from the video directly into the GPU memory, and they are processed in parallel there before going straight to your screen. Much, MUCH faster.

Sorry for the dumb question but how does the GPU execute Javascript? Or is it done in a declarative language specific to the GPU?

The GPU doesn't execute Javascript. It runs small programs called "shaders", written in GLSL (the OpenGL shading language), which are compiled on the fly by the graphics driver to something that can be actually executed on the GPU.

for small videos it's fast enough http://www.barbafan.de/html5video?video=tron

I'm confused, is the green screen suppose to be intentionally bad? It not only removes the background but most of the people as well.

Edit: Here is what it looks like for me: http://i.imgur.com/8wflg8Q.jpg in both Firefox 21 and Chrome 29+27

No, it is definitely not supposed to look like that. Here's what it looks like for me: https://s3.amazonaws.com/uploads.hipchat.com/23744/150541/nk...

Do you mind sharing your video card info and operating system? It will be hard to debug remotely, but I'll see what I can dig up. Please post it here or on github: https://github.com/brianchirls/Seriously.js/issues

Thanks

Working fine on 27.0.1453.93 on OSX here. Although the pictures seem to have been changed. There's only one person's face on the left on mine.

Picture? It's a video.

Is that OK GO?

It's the raw video from http://www.youtube.com/watch?v=12zJw9varYE. The real film clip features a really neat Windows 98-like green screen effect.

This is more than a year old. It was featured on Badass JS in January 2012. http://badassjs.com/post/16583192105/seriously-js-a-realtime...

FYI, I get nothing cool other than a blank area, and I see this in the console logs:

Uncaught Not a valid HTML element: EMBED (must be img, video or canvas) seriously.js:2634

Chrome on osx: https://aboutmybrowser.com/T1fotBeR

Please cite the song on the page.

Doesn't work at all for me in Firefox 21.0 on amd64 Linux. Works in Chromium 25 on the same box. This is particularly disappointing because there's a Mozilla logo on the page.

It works fine in FF 24 on a mac.

That makes me happy; it means it'll probably work on my box too if I upgrade Firefox. (I'm using Ubuntu 12.04's default apt repository.)

You know it's Serious when "TV Glitch" is available as a feature.

"TV Glitch" is the Page Curl of the 2010's

What's the awesome song?

This is "WTF" by OK Go. Back in 2010 they posted the raw greenscreen footage for people to remix.

http://okgo.net/2010/01/20/wtf-video-remix-project/

i did something like this (ok, not as smooth but it goes in the same direction) in 2010 with canvas/video/css http://www.barbafan.de/html5video?video=tron (was covered by boingboing at that time, lots of traffic, some partnership opportunities (mostly from p_rn sites), some job offers, non any interesting))

cool that somebody now packs a similar idea into a lib, will definitely check it out.

Is it just me, or is this the perfect use case for Insert Me Anywhere?

http://insertmeanywhere.biz/#/home

TypeError: video is null @ http://seriouslyjs.org/js/script.js:226

Firefox 21.0

Yeah, I got the same. Do you have Flashblock? Try whitelisting seriouslyjs.org and reloading. Fixed it for me on Fx 23.0a2.

(Note: This comment was originally complaining that the link didn't work in Firefox; edited when I found the cause.)

Not for me, but given it has a Mozilla logo and credits them for "support" I would assume it does for some.

This even works smoothly on my HTC One X (International) running Firefox Aurora with only occasional frame drop. Amazing.

Doesn't work with Ghostery enabled. Don't see why it shouldn't work with twitter/google analytics blocked...

It works for me with Ghostery enabled (and blocking both the Twitter and Google Analytics). I'd say there's something else at play.

Is there a console error?

"Uncaught Unknown source type seriously.js:2727"

Soon as I turn Ghostery off it works fine. This is in Chrome 27.0.1453.110

Works fine for me with Ghostery. For some reason, it looks like videojs is falling back to EMBED tag for some people. Will investigate.

First thing in the morning, I wake up to see this. Made my day :). Awesome job.

If only we had static types, we could do something like this with JavaScript.

Isn't this an old news? I feel like I saw this page in last year?

Anyone want to take a stab at overlaying youtube vids with this?

What is the name of this song? What a good song

Crashes my browser in seconds. Nice exploit.

'by the sea' crashed my x server!

CentOS 5.9 / Firefox 17.0.6

What am I missing? I saw a music video.

What kind of person sees buttons and doesn't click them? You monster ;)

Seriously? That's just wow.

What is the point of this?

This is incredible.

Nice song also.

Damn nice.

End of TV!

badass code. badass music.

this is AWESOME!

Whoa, this caused the second BSOD I've seen since Windows 98. When I started messing with the options, the windows just flashed wildly for a while, some popup about ATI Radeon driver errors was struggling to spring to existence, and then everything froze.

Cool stuff. Good song too.

Did you know your library name is an adjective?

Seriously? It's an adverb.

Reply on news.ycombinator.com