Oboe: A C++ library for low latency audio in Android

> Oboe provides the lowest possible audio latency across the widest range of Android devices

The "lowest possible" on android might not be low enough for musical instuments (<10-15ms). It's revealing that they don't reveal latency for any devices at all. (If I sound cynical, this isn't the first low-latency offering for android.) It's just "low" for android.

funfact: consoles also have terrible audio latency. Rhythm games like rockband/guitar hero don't actually need this, they need only relate it to the audio playing (and also config compensation for input latency). The latency is shocking if you use the mode where you free-"play" the instrument, and is why actual musicians often a tough time with these games. The surprising thing is that delayed sound effects from games in general aren't so jarring.

FWIW iphone audio latency also isn't good enough, despite being far better than android. That's how bad android is.

EDIT 7ms claimed in screenshot at https://github.com/google/oboe/blob/master/samples/hello-obo... - doesn't include input latency: average latency of the audio stream between data entering the stream and it being presented to the audio device

EDIT they know they need latency figures "Publish a list of Android device latency" https://github.com/google/oboe/issues/235

Does this mean we will finally get decent musicmaking apps on Android?

A decade after iOS, I guess it comes in from the "better late than never department", but I am happy. Apple seems to have lost its focus on the creative usage of their devices, and it has somewhat stagnated (compared to the initial explosion), plus iOS apps tend to be pricey.

Apple's decision to not include the headphone jack effectively puts the kibosh on their realtime audio / musicmaking apps. The latency on AirPods is atrocious. You press a key on a virtual keyboard, and count to three before you hear a sound.

Yes, this is probably solved by a dongle. Eh, I'd rather carry my Yamaha Reface DX around than think about that -- and the developers are going to have a FunTime(tm) explaining this to the users. (The sole reason musicians love iDevices so much is the JustWorks(tm) aspect - well, that just went out the window).

This Android API just might start a new wave. The video appeals to the synth nerd in me. The gear, the T-Shirt, the code samples doing the right thing.

I am looking forward to what this will bring us.

Typical Android NDK library from Google, instead of being added to the NDK, Android Studio and respective documentation, we get a github repository and "good luck" pat in the back.

And maybe if they have enough buffer on their 20% we get some of the reported issues fixed.

Like cdep, Play Services for C++, FPL libraries...

The proliferation of apps for musicians on iOS (drums, guitar amps) is explained by the iOS's longstanding API for low-latency audio. I hope we can expect the same kinds of apps, with usable latency, for Android now too.

I looked at the implementation of the openStream() function:

  AudioStream *AudioStreamBuilder::build() {
      AudioStream *stream = nullptr;
      if (mAudioApi == AudioApi::AAudio && isAAudioSupported()) {
          stream = new AudioStreamAAudio(*this);  

      // If unspecified, only use AAudio if recommended.
      } else if (mAudioApi == AudioApi::Unspecified && isAAudioRecommended()) {
          stream = new AudioStreamAAudio(*this);
      } else {
          if (getDirection() == oboe::Direction::Output) {
              stream = new AudioOutputStreamOpenSLES(*this);
          } else if (getDirection() == oboe::Direction::Input) {
              stream = new AudioInputStreamOpenSLES(*this);
          }
      }
      return stream;
  }  

  Result AudioStreamBuilder::openStream(AudioStream **streamPP) {
      if (streamPP == nullptr) {
          return Result::ErrorNull;
      }
      *streamPP = nullptr;
      AudioStream *streamP = build();
      if (streamP == nullptr) {
          return Result::ErrorNull;
      }
      Result result = streamP->open(); // TODO review API
      if (result == Result::OK) {
          *streamPP = streamP;
      }
      return result;
  }

Doesn't this leak memory if streamP->open() fails? I'm also surprised they are using new instead of unique_ptr, especially since one of their listed benefits is "Convenient C++ API (uses the C++11 standard) ".

I used Oboe (and its predecessor, Howie) to make a self-balancing robot on Android. We were able to use the headphone jack (RIP) as a low-latency output to send serial data to an Arduino[0].

Android Studio has also finally improved NDK support enough that writing large chunks of your app in C++ isn't painful anymore.

[0]: https://davidawehr.com/blog/audioserial/

Unfortunately, this is unlikely to improve latency on devices which already don't have 'good latency'. Reviewing the linked doc, this is a 'high level' API whos goal is possibly more about streamlining the API across multiple recent releases, while I'm sure, addressing latency as best it can. The problem is that this API still sits on top of other lower (and already existing api's) aaudio for current releases and opensles for older. A large part of Android audio latency is down to the HAL layer that each vendor provides. Andoid as a framework offers support for varying degrees of latency, but it's up to the vendor supplying the audio HAL to support this. This is exposed through the low_latency prop check (or 'professional' in recent releases). If the vendor HAL doesn't support 'low latency' through reduced buffer sizes and efficient (or no) locking etc, there's nothing a high level API like this can do, other than bypass the HAL and talk to ALSA directly, but this breaks the whole architecture.

It may be interesting to recall that collabora did port the linux professional JACK audio server to Android and it has far less latency than audioflinger (btw in french audioflinger phonetically mean to destroy the audio (quality), which is kinda appropriate).

Sadly no OEM ever cared to my knowledge to integrate this work in their ROMs. I also openened an issue on the android bugtracker to feature request replacing audioflinger with alsa/pulseaudio. They never answered nor closed the issue but I haven't checked since 2 years.

Edit: here is the android feature request https://issuetracker.google.com/issues/37073168 Maybe if you comment/upvote it it may gain traction.

Digression: If you are interested in audio latency, I recall that Firefox has better webaudio performance than Chrome.

Unrelated, but there's a code example on that website:

    AudioStreamBuilder builder;
    AudioStream *stream = nullptr;
    Result result = builder.openStream(&stream);

Why even bother initializing stream to nullptr if it's going to be overwritten anyways by AudioStreamBuilder::openStream?

Unfortunately this won’t help with the terrible hardware level latency on many Android devices.

Audio latency issues on PCs are largely alleviated thanks to better softwate (device drivers that shorten the path to/from the hardware - such as ASIO4ALL) combined with better hardware (soundcards interfacing over PCI-E/Firewire/Thunderbolt/USB) at affordable prices. Plenty of audio cards today will offload computation off the CPU via discrete DSPs.

On Android devices (and smartphones in general) - hardware would be the limiting factor.

Wonderful! Latency has been a massive blocker for audio on Android - this makes me want to break open the SDKs again and tinker.

So what are the latency numbers for some common devices? (apart from the 7 ms visible in the hello oboe screenshot)

Can anyone recommend the best way to do low latency audio on windows and osx?

Would this make it possible to build active noise cancelling using software?

You guys should also check out AudioKit https://github.com/AudioKit/AudioKit

The title should probably mention "for Android". It's misleading without that context.