FFTs can be useful when constructing CNN pipelines in unsupervised domain adaptation too. Fourier domain adaptation (FDA) uses the calculated amplitude of a target set to modify the source (train) set. Cool and quick to implement.
I think the fourier transform might be a really approachable version because it is already in so much signal processing. Sometimes its periodicity can work against you.
From what I can tell you, what you need are:
1. invertible, identifiable basis
2. Something like Parseval's Identity
3. Able to "smooth" out things like the heaviside function or dirac deltas
4. Workable on graphs as well
5. decays at infinity
6. eigenfunctions are all orthonormal
Turns out, making something like that gets you something similar to the fourier transform, and the math is probably the simplest since you are working with easy exponentials that work nicely with convolution.
People have done similar work with Chebyshev polynomials, and there are actually a lot of applications in ML/AI using Chebyshev polynomials in graphical neural networks and triangulation.
edit:
The fourier transform also comes about pretty nicely in quantum mechanics because of poisson bracket stuff and its nice derivative properties for position-momentum space. I am don't think other transforms/function basis come out so naturally.
Nobel disease, but even without the Nobel-price?