Comment by julesrms | Hacker News Reader

Comment by julesrms | Hacker News Reader

Parent Full thread

Not sure that being able to change the graph dynamically is a particularly good way to work with this kind of thing. In almost all synths, and even DAW engines, they only modify the graph when you modify the project, not while things are running, and that's when we expect you'd do a SOUL recompile. There's a much longer discussion to be had over this which I won't dive into now, but we have thought about many possible use-cases and how they'd be done.

Licensing: very permissive for developers; probably commerical deals for companies who want to ship SOUL-compliant hardware or drivers

JUCE dependency: no, we'll want to make this as vanilla as possible, to encourage its use in many ecosystems. There'll be JUCE integration, but also stand-alone C++ and a flat C API so it can be wrapped in other languages like python, C#, Java etc

We'll offer an embeddable JIT VM, but our end-goal with this is for there to be drivers in the OS or external hardware which does the work, and the API would just send them the code (like e.g. openGL shaders)

It would be very helpful to have some way to express graph changes due to events in the language, even if it doesn't happen in real time. Otherwise we're going to have to get really hacky with generating SOUL on the fly and recompiling it, rather than just explicitly stating that graph changes happen at the discretion of the runtime.

RE the licensing, I'm asking more about the language itself, the IR, and whether it is going to be possible to develop independent implementations.

Lastly because I always forget to ask about the IR - why not WASM?

I think we'll probably want hot-reloading of sub-graphs in a larger graph, but that'll be a project for 1-2 years down the line when we're more involved in building DAW engines using it.

Re: building a 3rd party back-end, I guess that might be technically possible but we'd rather keep control of that side of things. We've not fully decided our approach there yet.

Why not WASM? Well.. quite a few complicated reasons, involving things like the ability to generate continuations, and to get good auto-vectorisation, and being portable to weird DSP architectures. Also, the system can't just be a straight program, it had to be proprietary, at least at a high-level. And secure, so LLVM IR also wasn't quite the right shape for it. We sweated over this decision, believe me!

You mention custom DSP hardware licensing ... any reason not to target the GPU directly using CUDA or whatever?

Our original thoughts on that was "nah, that'd be silly because GPUs are the wrong shape for audio workloads"

..however, having a few conversations with more knowledgable people has changed our mind and we're definitely going to give it a try on Metal and Vulkan to see what happens. Should be interesting.

I'm curious what you mean by "wrong shape"? Because GPUs are for for very data-parallel workloads? Or because the latency is high?

Yeah, we made both of those assumptions. But it's apparently less true of the latest generation of compute engines. We probably won't bother with CUDA, but Metal is certainly a viable platform to try. It's one of those things where nobody really knows how it'll do until you try it.

Also, we do know a few people who want to use SOUL to write audio code which does need high parallelism. It's not a super-common use-case for audio, but it does exist.

I hope you'll consider writing up your results in a blog post or something! As I just mentioned elsewhere in this thread, I have experimentally found latency with Metal compute to be too high to be feasible, but I would dearly love to be proven wrong, if there's a trick I'm missing.

Yes, we'll certainly be writing it up. Similarly i'd be interested in reading about your experiences attempting the same if that's available.

I can understand the aversion to GPU (with "Graphics" being a primary point). Most ways of interfacing with the GPU use x,y,z, color, lighting wrappers around what is essentially a super powerful vpu. I also wonder how the traditional pipeline (e.g. vertex shading, fragment shading) can be repurposed for audio.

I don't have any experience with Metal or Vulcan but my intuition is that audio DSP is going to include a healthy dose of linear algebra and multi-variate calculus. That points towards some kind of fit with the GPU. Given that basic GPUs are available on practically every device (including phones) it seems like a fantastic fit. Even audio hardware developers would benefit since it would open up access to commodity priced chips (rather than custom asic/fpga/whatever).

Yep.

And even for very simple audio loads, there's often unused capacity in compute cores (even when running games) which would could do the job "for free" without bothering the CPU.

I've looked a tiny bit into this sort of thing (using Metal compute shaders), and the problem is that, even with all of Metal's optimizations around CPU/GPU synchronization, the overhead seems to be too high for real-time audio DSP in practice.

The thing I tried experimentally implementing is basically a 128x128 matrix mixer with a little bit of extra processing. On a three-year-old MacBook Pro, the GPU barely had to lift a finger to crunch the data, but the round-trip latency was still high enough that it would struggle to keep up with anything less than a buffer size of 512 or so at 48kHz (which is on the high end for live mic processing). It would be fantastic for offline processing with larger buffers, though.

I haven't tried CUDA or OpenCL, so I don't know if the situation is the same there—but of course they have the problem of vendor support as well.

Reply on news.ycombinator.com