donturner

> * Q1: I assume that, if instead of the jack, you use a usb-c audio adapter accessory mode, there should be no extra latency either, right? (I'm using late pixel phones)

That's correct, by using either the 3.5mm jack, or USB-C adapter you won't incur any additional latency introduced by DSP to improve the speaker acoustics.

However, the USB path typically does have a few ms higher latency than the 3.5mm jack path.

> * Q2: which device are you using for the USB-MIDI input?

We've used a variety of devices and found the latency differences to be negligible. At the moment I test with an old AKAI LPK25.

Below is a collaborative answer from myself and my colleague, Phil Burk who is a SWE on the audio framework team.

Thanks for your interest in the details of Android Latency measurement techniques.

Some of the measurements were collected by third parties. We cannot describe their techniques but they are probably similar to ours.

We use OboeTester to measure latency. You can find a description of how to measure Tap-to-Tone Latency and Round Trip Latency in this doc: https://github.com/google/oboe/blob/master/apps/OboeTester/d...

We do not use the Larsen Effect any more because it was too sensitive to variations in gain. We now use a random encoded bit stream that sounds like a short noise burst. We can get a better correlation peak with that signal.

> Many of the latency measurements break the latency down between > audio downlink and uplink.

It is very hard to separate the input and output latency without special hardware (like the WALT device). You can measure combined input+output latency using a loopback test. Input latency tends to be much lower than output latency. This is because, when the full duplex stream is stable, the input buffer is close to empty and the output buffer is close to full. Then, if there is a preemption, the input buffer fills up and the output buffer drains, providing glitch protection.

You can measure touch+output latency using tap-to-tone. The screen touch latency is about 15-30 msec. If you use a hardware MIDI controller (like a keyboard or drum pad) instead of tapping the touch screen then you can get a “touch” latency of about 1 msec (MIDI is a lightweight and low latency protocol). Then you can get a better estimate of just the output latency.

> The android doc discusses using a GPIO to have a zero-latency signal.

We don’t normally use that technique because it requires special hardware.

> I found oboetester, splatency, drrickorang, and google walt.

Our group supports OboeTester.

> The first interesting thing is that oboetest, drrickorang, and walt seem to need an external jack-based dongle.

The lowest latency path is usually over the headphone jack (either through 3.5mm or USB dongle). To test this path you do need a "jack-based dongle" aka loopback adapter.

The reason why wired headphones usually give you lower latency is that OEMs often introduce digital signal processing for the speaker to improve the acoustics/quality, which can introduce additional latency. Side note: this is why you will often see "best with headphones" on games and music apps.

That said, OboeTester will work over the speakers and mic in a quiet room. That is because the new random bit technique is more robust.

> I'd like to script these tests.

OboeTester can be scripted. We use it for continuous integration testing. https://github.com/google/oboe/blob/master/apps/OboeTester/d...

I'm the author. My testing methodology was to run the round-trip latency test using OboeTester: https://github.com/google/oboe/tree/master/apps/OboeTester/d....

It's a simple test: produce a tone (either on built-in speaker or over wired headphones if using a loopback dongle), measure the time it takes for that tone to reach the audio input.

For devices which I didn't have access to (our team has a limited number of test devices) I used the figures from Superpowered, but with some assumptions/rules:

#1 If AAudio was available on the device I used the measurements from that rather than OpenSL ES #2 No measurements from custom ROMs #3 I used the measurements from the latest version of Android which the OEM had released for that device. For example, if a device was originally released with Lollipop but it was possible to upgrade to Marshmallow then I used the figures for Marshmallow.

For touch screen latency measurements I used a WALT device (https://github.com/google/walt)

Damn that spellchecker! :D Thanks for spotting, I'll fix it shortly.

> the human brain is severely impacted by lag

The brain can compensate for lag/latency up to a point. Church organ players often have to deal with >100ms of delay between pressing a key and hearing the note produced through those big pipes.

If you have an app which produces a sound when you tap on the screen, and there's a delay of say 70ms between tapping and sound being produced, it's easily possible to play in time with an external clock source, like a metronome, as long as the delay is fixed.

In the 1968 paper "Response time in man-computer conversational transactions" actions which occurred within 100ms the human input were perceived as instantaneous. In my experience this holds true with musical instrument apps, although one must first learn how much latency is present in order to compensate for it when synchronising with other clock sources.