Sorry, I was reading this late at night and must have misread the microcontroller. Then, the limitation lies in the codec and it would not be a minor modification or a matter of trading bit rate for latency.
I need to record ultrasound for work onboard construction vessels, currently we use either specialized equipment for PAM, which is limited in some aspects, or a USB sound card with a SBC and a hacky setup for sending PCM over TCP.