That Whole Thing with Sound in In-Browser Emulation
ascii.textfiles.com
ascii.textfiles.com
Or, the only way apart from plugins, anyway. In comments the author writes:
> In the future, I suspect some group will do it “right” and create a wonderful plugin or default wrap-in for all browsers that will simply function as the native environment all this material needs. At that point, MESS and DOSBOX will be ported to this...
..which is essentially Flash (which has had solid dynamic audio for years, and console emulators built on it with good audio support (see:nesbox)). In a parallel universe where Flash was an open standard, it'd be exactly what the author is asking for. Except in that universe, Flash would presumably share whatever problems the author finds with web standards.
Unless we want to choose One Browser or One Plugin, I'd say we're better off with complicated, slow-moving standards (like Web Audio), even if they are developed and implemented by adversarial companies.
It was like every browser was it's own terrible brand of IE6. Each with their own weirdness that permeated every aspect of the browser.
You ended up having to write as many stylesheets as browsers you wanted to support (ok,ok, netscape, firefox and mozilla suite shared the same stylesheet).
"Cross-Browser" was as unattainable and desired as the Holy Grail, "This site works best in IE6/Firefox/Opera/Safari" was not a suggestion like "This wine pairs best with cheese" but a warning of terrible consequences like "This milk is best before curdled".
Ok, so we're still not 100% cross-browser 100% of the time, but it's a term people don't often need to think about anymore.
In the beginning, it was formatted text. That was it. You had bold, underline, italic, and maybe even justification, but images relied on separate helper programs and weren't displayed inline, but in their own windows. And that was assuming the web page and your system had at least one image format in common. (XBM? TIFF? Remember those? I'm not saying they're no longer used, but when was the last time you saw them on the Web?)
By the Here Comes Everybody stage, we'd pretty well gotten inline images and even background images sorted, to the limits of color space (256 color is... better than CGA?) and the few format problems left, but fonts certainly didn't transfer and the only music format which was even remotely feasible was MIDI, which is less an audio format and more like sheet music, in that it has to be played by software instruments on your own system. Interesting concept, but no guarantee of sound quality at the receiving end.
Skipping a bit, we now have inline images sorted out, fonts sorted out, a workable CSS standard pretty well universal, and even video mostly working, despite the political and legal complications.
Now we're annoyed that we can't use our browsers to play games that were written for computers still in use in the Web's early days. 'Progress' is getting annoyed about increasingly interesting and exciting things, it seems.
In a perfect world, a low-level web audio API would focus on providing an as direct-as-possible way to stream raw audio data (buffer queueing like in OpenAL, or a wrap-around buffer like in DirectSound), make sure that this works well with WebWorkers (i.e., worker thread should be able to queue new data when required, not when the main loop gets around to it), and move all the complicated high-level audio-graph stuff into optional Javascript libs.
But Google was able to put more engineering effort behind the Web Audio API and get a lot of web sites to use it, so that's what won. The fact that Web Audio's ScriptProcessorNodes run on the main thread instead of in workers is basically a travesty, brought about because they were tacked on as an afterthought to claim "feature parity" with the MediaStream Processing API. Until that's fixed [1], real-time audio generation in JavaScript will always be a joke that only works under ideal conditions where people open just one, well-tuned webpage in their browsers at a time and GC pauses don't exist.
(full disclosure: I work for Mozilla and am not bitter at all)
[1] https://developer.mozilla.org/en-US/docs/Web/API/Web_Audio_A... is the proposal to fix it, only three years late to the party
Transmitting Speex "frames" (where each frame is a 20ms chunk of audio) over WebSockets and decoding in JS works beautifully. However, I have a really hard time to queue up these 20ms AudioBuffer nodes perfectly, without producing any glitches or pops. I'm not sure its even possible with the current API.
What I would like to have is an AudioBuffer node where I can dynamically append chunks while its playing. Since all(?) source nodes are for one time only use, the browser could free the data that was already played again.
An AudioBuffer that allows modification of the data while it's playing would work too - you could just loop it and use it as a ring buffer. However, my understanding of the spec is, that modifying an AudioBuffer's data is explicitly prohibited once its playing.
Just curious, if your goal is realtime, why aren't you using WebRTC?
I can't use WebRTC yet because I want to have support for IE and especially Safari & Mobile Safari. I want to add audio for an iOS App[1] that's using jsmpeg[2] for video streaming.
Any proper realtime audio stack (which ogv.js is not: it targets moderate latency streaming) is going to require a jitter buffer to deal with network/processing latency variation. Writing one is a non-trivial exercise (though not impossibly so).
In any case, I'd recommend using Opus (the audio codec used by WebRTC) over Speex.
Side note: If anyone wants to take over webNES and make it awesome, free free to message me
[1] http://webn.es
[2] https://github.com/conradev/jsnes/commit/6b0ef8d5b5d0a7b17e6...
[3] https://dvcs.w3.org/hg/gamepad/raw-file/default/gamepad.html
What's wrong with the Web Audio ScriptProcessor node for example?
To really get where people want to go in terms of browser based gaming (even if the idea disgusts you) is going to hinge on 3D video, great audio composition and good controller capabilities.