The problem with that is that MIDI is a very strong standard, as bad as the format itself might be. Every single synth or keyboard out there exports to MIDI, and most of the time ONLY to MIDI.

Also, the MIDI format is not slow to generate or to read, it is actually designed for speed (and may other perks) and any implementation of it would be very likely faster than any other format you could come up with. Of course, you would still have to deal with its complications and the many particularities and edge-cases it has.