Great project! I used MNIST because it is easy to work with as a dataset. Audio classification would be quite interesting as a follow up, but I assume one would need some kind of transform to deal with the data in an easier way.
Thanks! Yeah transforming into a time-frequency is the standard method. Short Time Fourier Transform (STFT) using FFT is the most common, though one can use FIR/IIR filterbanks also. It is however quite challenging to do in just a few kB of RAM. It looks doable with 4 kB in total, miiight be possible with 2 kB.
Maybe something simpler, like a haar wavelet, would also work? Or DFT using Görtzel?