Wow, thanks for pointing that out. I didn't know that was a thing. It's not even encrypted with a device key or something on my phone - I can listen to a recording my own voice streaming straight from myactivity.google.com to my desktop web browser.

I guess the voice scrambling thing must have been specific to their human-supervised learning program. I expect the unaltered voice data is treated with some special care on Google's backend... but I'm still creeped out.