One thing I don’t understand: if latency is important for this use case, why isn’t the CPU busy preparing the next GPU ‘job’ while a GPU ‘job’ is running?
Is that a limitation of the audio plug-in APIs?
Is that a limitation of the audio plug-in APIs?
Possibly what you describe is a bit more like double-buffering, which I also explored. The problem here is latency: any form of N-buffering introduces additional latency. This is one reason why some gamers don't like triple-buffering for graphics, because it introduces further latency between their mouse inputs and the visual change.
But furthermore, when the GPU clock rate is too low, double-buffering or pipelining don't help anyway, because fundamentally Anukari has to keep up with real time, and every block it processes is dependent on the previous one. With a fully-lowered GPU clock, the issue does actually become one of throughput and not just latency.
That’s why I asked about the plug-in APIs. They may have to be async, with functions not returning when they’re fully done processing a ‘packet’ but as soon as they can accept more data, which may be earlier.
But in general no, you can't begin processing a buffer before finishing the previous buffer because the processing is stateful and you would introduce a data race. And you can't synchronize the state with something simple like a lock, because locking the audio playback is forbidden in real time.
You can buffer ahead of time, this introduces latency. You can't do things ahead of time without introducing delay, because of causality - you can't start processing packet #2 while packet #1 is in flight because packet #2 hasn't happened yet.
To make it a bit more clear why you can't do this without more latency:
Under the hood there is an audio device that reads/writes from a buffer at a fixed interval of time, call that N (number of samples, multiply by sample rate to get in seconds). When that interval is up, the driver swaps the buffer for a new one of the same size. The OS now has exactly (N samples * sample_rate) to fill the buffer before its swapped back with the device driver.
The kernel maps or copies the buffer into virtual memory, wake the user space process, call a function to fill the buffer, and return back to kernel space to commit it back to the driver. The buffer you read/write from your process is packet #1. Packet #2 doesn't arrive until the interval ticks again and buffers are exchanged.
Now say that processing packet #1 takes longer than N samples or needs at least M samples of data to do its work and M > N. What you do is copy your N samples of packet #1 into a temporary buffer, what until M samples have been acquired to do your work, but concurrently read out of your internal buffer delayed by M - N samples. You've successfully done more work, but delayed the stream by the difference.
Or perhaps you're missing that there's an in event as part of this, like a MIDI instrument? It's an in->effect->out sequence. So minimizing latency means that the "effect" part must be as small as possible, which means it's desired for it to happen faster than "in" can feed it data