FLAC.js, Aurora, and the Future of Web Audio | Hacker News Reader

FLAC.js, Aurora, and the Future of Web Audio

badassjs.com

106 pointsdevongovett32 comments

This is exactly the kind of thing I would've been working on if my boss hadn't said "it's a great idea, but let's just use it if somebody else does it". I worked on converting JSMad to use entirely typed arrays last summer (fun!), so I'm especially glad to see that all fixed up in the refactoring for Aurora.

Now, what I'd really like to see is a similar project for a) container formats and b) video codecs. I've been experimenting with using JSMad to display waveforms from decoding MP3 files that are the audio tracks of videos (for assisting with subtitling), but it's kinda annoying having to download a separate file for generating waveforms rather than just extracting it directly from the same video that's playing anyway.

We do support multiple containers, and it is reasonably abstracted (but it is slightly leaky right now).

The support for multiple streams isn't there yet, it is something I would love to do, but it is reasonably complex and I have been researching about how to best support it, but I am not really sure what models works best in a browser.

If you want to add some input about it, I would love to hear it. (#ofmlabs on freenode, or send me an email)

Ps. We managed to get Typed Arrays to work with JSMad, so that stuff is in now.

For decoding the video itself, https://github.com/mbebenita/Broadway is as far as anyone's gotten so far, but it's slow (as one would expect). Still, I'd love to see something able to extract the audio from container formats.

Awesome news :) For the mp3 demo, what is the song encoded at? Also, is there a way to listen to the same tracks encoded with the various encoders to compare? I come up with my own encoder / decoder a few years ago and ran a MUSHRA test against various other popular codecs; I wish I had this tool back then!

It is 128kbit in the MP3 demo, I think I used LAME to encode it, but I am not sure, could have been the one in OS X. It plays 320kbit files fine though.

It sounds really bad in Chrome on Ubuntu 12.04. Choppy, I don't think the speed is right, etc. I didn't see a link to the original, though, so it might be just really crappy music.

Chrome and Ubuntu have been having both HTML5/Flash audio since early this year unfortunately. It's rare that I have any audio played correctly via Chrome regardless of the source, it sounds corrupt and plays at the wrong speed most of the time. Suspicions are PulseAudio. You can try killing it and reloading the page and trying again (you may also need to restart Chrome).

try adding tsched=0 to one of the lines in /etc/pulse/default.pa so it becomes

load-module module-udev-detect tsched=0

this fixed my chrome ubuntu audio problems. (source: links at bottom of http://code.google.com/p/chromium/issues/detail?id=110375 )

http://labs.official.fm/media/vibe@44100.flac, but it shouldn't sound 'choppy'.

We have had some problems on Linux where Chrome(ium) for some reason reports the wrong sample-rate, but that usually just changes the playback speed.

Does it use high amounts of CPU too?

Yep, that sounds normal. It uses about 12% CPU for Pulse and 12% for Chrome (this is a Core 2 Quad).

Could you please add an issue with some more precise information about your browser(s)/sound server at https://github.com/ofmlabs/aurora.js/issues ?

Naming it "Aurora" might be confusing, since that's what Mozilla calls the Alpha builds of Firefox. https://www.mozilla.org/en-US/firefox/channel/#aurora

Yeah, but it had the name before Mozilla called their stuff Aurora, and it stuck.

But yes, it is slightly confusing, we know; we're just out of imagination and if something as nice pops up, we might rename Aurora.js to lessen the confusion.

Oh didn't realize your project was older. :) Anyway I doubt most people would really confuse the two.

Yeah, but when developing it ends up being confusing sometimes since we're often targeting Mozilla Aurora, it is also why I prefer to call it Aurora.js.

Mozilla named their alpha release channel "Aurora" in 2011, but the name is a nod to Netscape's 1997 Aurora project, a competitor to IE4's Active Desktop:

http://news.cnet.com/Netscape-previews-Aurora/2100-1001_3-20...

I was going to suggest the name "Jazz.js" because it is a music term that contains the letters J (from JavaScript) and A (from Audio). :) But I see that there is already a project called Jazz.js (a Node.js template engine).

I was then going to suggest the name "Jade.js" (JavaScript Audio Decoders Encoders), but I see that there is already a project called Jade.js (yet another Node.js template engine).

Can you just rename it Audora.js? Quick bing search didn't turn it up. It contains "aud" from audio, plus "aurora" is kind of wussy sounding anyway with no hard consonants.

This is the kind of news I am interested in.

I tried it, but I heard only a single 1/2 second sound randomly in the track (different spot each time). I tried all three decoders mp3 and flac made that sound, alac was totally silent.

Firefox 12.0 on Linux. I have noscript and flashblock installed but I set noscript to allow all.

Also, some runs I would get a time number for the length (a negative number - I assume it's supposed to be that way?) and sometimes it would be 0:00.

Could you try and see if any of the demos at https://wiki.mozilla.org/Audio_Data_API#Working_Audio_Data_D... work?

If so, could you please add a github issue at https://github.com/ofmlabs/aurora.js/issues with some more information about your OS/Sound Server so we can replicate the issue?

(The negative number on the right is the time remaining)

Most do work, some don't. But the ones that do make my browser extremely unhappy. The whole thing becomes very unresponsive - almost unusable.

I have a 2.8Ghz I7 CPU, FF 12, on linux using native ALSA output. I'm pretty sure my webgl doesn't work though, which may have caused some of the issues.

It's hard for me to say for sure what does work since I'm not sure what some of them should look like.

Regarding your code, the time remaining number did not go down when it played - is it supposed to? (Another one of those things where I'm not sure what it is supposed to do.) Is the 0:00 at the start supposed to change? It didn't.

This is great. CPU is higher than native, but not too bad (especially considering it's an initial release).

I was getting 10% each on Chrome and a Chrome Renderer process. I paused to switch over to Cog to check its CPU usage, then switched back and it was hovering around 6%. For comparison, Cog spiked to 15% for a second then settled in at around 1%.

I'm on a 2010 Core 2 Duo MacBook Air.

The CPU usage seems to increase while the audio file is being downloaded from the server as well, which might explain why it was initial 10% and then dropped to 6%. We'll look into that more. When decoding from a local file, I've seen flac as low as 3.5%.

Yeah, after the initial spike while downloading (7%) mine settled down to about 1%: http://i.imgur.com/iFijI.png

This is on a 1st gen Core i5.

I'm impressed. The chromium pid settled around 15% on my Eeepc 900 (running Arch Linux, alsa, no pulseaudio)

Works perfectly in Chrome and Firefox here, but both browsers exhibit 30%+ CPU usage on a mid-2009 MBP. This is an outrageous figure that makes me wish NaCl gets standardized.

Interesting, it runs at about 4% to 5% CPU in Chrome here. Did you have the dev tools open? That seems to disable a lot of the JS optimizations that Chrome has.

Nope, just fired both browsers to test since I mainly use Safari those times.

I get 30% CPU usage while it's still downloading/buffering the file data, but once it's only decoding it settles down to 3-4%

This is awesome. I'd also love to see a Ogg Vorbis decoder - the Web Audio API is coming to Safari (both Mac and iOS), and it only plays the patent-encumbered AAC normally, without support for Vorbis. So that would fix Safari leaving only IE without support. Also, it might pressure browser makers to add native support for Vorbis - if their browser plays the files anyway, just slower than other browsers, why not move support in to the browser?

In Chromium on Ubuntu 11.04 with an old E4500 Core2 Duo CPU, MP3 and FLAC work flawlessly but ALAC doesn't play.

CPU usage goes from 6/8% (the base level with 3 Vim instances open, a couple of nautilus windows open, the update manager window open and two tabs in chromium) to 20/24%.

That's very cool.

Reply on news.ycombinator.com