We have much more compact and widespread generally available formats out there with libraries for many languages. Want unbounded JSON but more compact? msgpack or bson. Want stuff more efficiently packed based on the message structure? Use protobuf.
Yes, there is a little more effort needed for the engineers there. But ya know, if the inputs and outputs of a thing are actually DOCUMENTED and the schemas are available, it's not some massive reverse engineering feat :P
(Okay, maybe you're stuck doing something in a niche environment where handy protocol/format libraries aren't available to you; MATLAB for Microcontrollers or something. But if you're there, you're probably having fun dealing with all the nuances of implementing an efficient and safe recursive unicode text serialiser/serialiser for a format line JSON anyway :P)
Also if you're going the fairly standard route of "web API over HTTP", the protocols give us way more options readily available for much more efficient streaming of binary data.
It's not "wasting time" to teach devs that there are better ways of doing stuff. base64 encoding mp3s into JSON strings strikes me as "junior dev given 2 weeks to quickly implement something without somebody there to review and suggest alternative ways of doing stuff".