Hacker News Reader

Comment by _pgon | Hacker News Reader

How to play a sine wave:

  const audioContext = new AudioContext();
  const osc = audioContext.createOscillator();
  osc.frequency.value = 440;
  osc.connect(audioContext.destination);
  osc.start();

"BufferSourceNode" is intended to play back samples like a sampler would. The method the author proposes of creating buffers one after the other is a bizarre solution.

View on HN

I picked a 440Hz sine wave because I didn't want to write a more complex demo example, knowing full well someone would nitpick this.

Please use your imagination and try to imagine one of infinitely many other streams that I could make at runtime that are not easily made with the built-in toy oscillators.

It's a higher level API and you're deliberately ignoring all of its higher level features and concentrating on the part that clearly is underdeveloped. Maybe you should use your imagination instead of putting a square peg in a round hole?

> It's a higher level API

As the title of the post asks: "I don't know who the Web Audio API is designed for".

The high-level nodes are not featureful enough for professional audio production, and too slow and underspecified for game engines like FMOD / Wwise.

The low-level bits fall somewhere between impractical, deprecated, and useless.

Who is it designed for?

I think you're missing the point entirely. It's like a modular synthesiser. It's not "serious business" but this is the browser after all.

Plug a few oscillators into each other and you have an FM synth. Feed delays into each other, etc, etc. You can do that in a few lines of code with no dependencies.

To me, that's a huge potential audience.

If you want a array of samples and depend on dozens of JS libs for functionality, well, I'm sure the AudioWorkers will catch up eventually.

The ability to build primitive synths in a few lines of code (w/o library dependencies) is fine and well, but should not have been a priority for becoming a web standard. What's desperately needed is a well-designed low-level API. That could have been done years ago, and then if there was still sufficient demand for built-in nodes, those could have been added later.

As far as potential audience, in the time I've spent lurking in the Web Audio community, it seems like developers fall into one of two camps: 1) building toy projects for their own edification/learning, and happy to have the Web Audio API 2) trying to build a serious product (DAW, game, whatever) and super frustrated with the API. It seems pretty clear to me that end-users would be much better off if camp 2 had a good low-level API to work with.. camp 1 is not making much that gets used by end-users.

> It's not "serious business" but this is the browser after all.

Modern JS performance is actually quite good, and WebAssembly is only going to make it better. I think you underestimate the potential of audio processing in the browser.

> It's like a modular synthesiser.

I own hardware modular synths, and I built a proof-of-concept modular synth environment using the Web Audio API (https://github.com/rsimmons/plinth). The API makes it hard to build even simple things like well-behaved envelope generators or pitch quantizers. So even if you viewed the API as a sort of code-level modular synth environment, it's pretty unsuited to anything beyond trivial use cases.

The browser-based experimental/modular audio stuff that has any traction (e.g. https://github.com/charlieroberts) doesn't use the built-in nodes for these reasons.

So, the answer to "Who is the Web Audio API designed for", according to you, appears to be "people who want primitive FM synthesizers".

Not game developers, not professional audio production people. It's... not an answer I was expecting, but I suppose it's valid.

Talk about damning with faint praise.

> It's not "serious business" but this is the browser after all.

This answer would have made sense to me circa 2003, but I cannot fathom it today. The web started as "let's put academic papers online". It moved to "let's put magazines online" with some modest interactivity via forms.

But we've spent the last 10 or 15 years turning the web into a "you can do anything" platform. There's been huge progress in interactivity and visuals. There's no a priori reason audio should lag so far behind.

There has been huge progress in visuals, but the web is basically still a document and content delivery system decorated with a few animation features, not a full-fat creative multimedia OS.

And that could be because there are institutional forces keeping it at a certain level of clunkiness, which is far short of the requirements of professional media creators who work with video, 3D, and sound.

I suspect the real problem is that the walled-garden corporates don't want the web to compete with their lucrative app farming operations.

Unless that changes, the creative edges of the web will remained dumbed down.

And it probably won't change. Ever.

If that's correct, the original question has a simple answer: Web Audio is designed to meet the corporate requirements of Apple and Google, not the needs of web users or web developers.

If Google really wanted to limit the power of Web Audio in order to boost Android apps, you would think they would have made a usable Android audio API as well...

I guess we're all in agreement that it's a toy.

> Please use your imagination and try to imagine one of infinitely many other streams that I could make at runtime that are not easily made with the built-in toy oscillators.

Somebody already did. Check out Fourier Theory. The oscillators (well actually just sin, the rest will give you some help as well) can be used to make any stream, technically.

Unfortunately, you need an infinite number of them to construct any signal, and I think my computer would run out of memory before that :)

Use a square wave.

Ah yes, just run a square wave at twice the range of human hearing, then use a combination of filters to extract the desired ranges. Unfortunately you'd also need an infinite arrangement of filters.

You're making music. It's not that difficult.

http://kruhft.bandcamp.com/

Here's a synth using the technique:

http://busfactor1.ca/kruhft

I propose a challenge then. I'll give you an arbitrary stream in the format of a .wav, and you send me code, in JavaScript, that uses an series of infinite OscillatorNodes that reproduces that stream completely accurately.

I won't even make you write the code that generates the stream, I'll just require that one sample.

If you can do it, I'll give you $1000 out of my own pocket.

Wagers are the mother of all innovation, so I wholeheartedly support this.

If you're not sampling at an infinite sample rate, you don't need an infinite series of nodes.

Given Nyquist, a non-infinite series of [non-trivial power of 2] nodes will do the job just fine.

Audio FFT/iFFT processing typically uses 512 or 1024 bins, although sometimes you can get away with fewer.

In practice, iFFT doesn't use discrete oscillators for resynthesis, because the whole point of iFFTs is to limit the amount of work you have to do.

But there's no reason in theory a limited number of oscillators couldn't do the job. (I've done resynthesis like this in SuperCollider when I wanted special effects that FFTs can't produce.)

I doubt WA would be fast enough on most machines, but it would probably be possible - if rather dumb - on powerful hardware.

I'm well aware of discrete fourier transforms, and that we're dealing with a ~20KHz bandlimited signal. My original "infinite series" reply was a joke poking at the ridiculous idea of using iFFT for sample playback that seemed to get interpreted as a serious response by kruhft.

I was talking about synthesis, which is what the oscillators of web audio are for. I figured that was obvious.

Thank you.

I mean, this is like saying "I can draw a lighted 3D cube with these 10 lines of OpenGL immediate mode, what do I need this 500 line Vulkan example for". And that's true!

It also misses the point, because, as with Vulkan, you just want a stable, sane, fast low-level API to access the hardware because OpenGL immediate mode doesn't get you beyond kindergarten in todays computer graphics. In audio, that is a sample-level API. Everything else should be handled by the application!

You can still make a source/sink directed graph system with components like "oscillators". In a fricking library!

> The method the author proposes of creating buffers one after the other is a bizarre solution.

I used the same solution when I tried to perform realtime audio streaming from a daemon on an embedded device to a browser (which is probably even a more realistic use-case for a browser audio API than generatic sine waves). I basically stumbled over the same issues than the author: A deprecated ScriptProcessorNode and high-level APIs which don't help me (like the oscillator one).

In the end I opted for a very similar solution as the author: Whenenver I got enough samples through websocket (I encoded them simply as raw 16bit samples there) I created a BufferSource, copied all samples in there (with conversion to floating point), and enqueued the buffer for playback at the position where the last buffer finished.

I really didn't expect that to work well due to all the overhead of creating and copying buffers and due to the uncertainity whether the browser will switch between 2 buffers without missing samples. But surprisingly it worked and did the job. I included a buffering of 200ms, which means I only started playback 200ms to be able to receive more data in the background and have a little bit more time to append further buffers. I experimented a little bit with that number but can't remember how deep the lower limit was before getting dropouts regurarly. It definitely wasn't usable for low-latency playback.

Reply on news.ycombinator.com