An Interactive Introduction to Fourier Transforms
jezzamon.com
jezzamon.com
I believe I know now: it's because of the convolution theorem. When performing a Fourier transform, we represent the signal in a new basis, where each component will independently get transformed by linear shift invariant systems.
Basically the Fourier basis diagonalizes the convolution operator. And the even deeper reason for that is that a complex exponential function can be shifted by simply multiplying it with a constant.
In simpler words, it comes down to the barber pole illusion. If you rotate a spring-shaped 3d curve, it looks as if it was traveling upwards. And the 2d projection of the spring are sines and cosines.
And it turns out that linear, shift-invariant systems are really common (or are at least a good approximation of many common natural phenomena), so it's very helpful to break up a signal into pieces that each get independently transformed, without any interaction effects.
Actually I thought it is the other way round: A frequency is defined as a sine basically, it has an * amplitude * frequency * phase shift
The phase shift is why we need cosines and sines in the Fourier transform as cosine is just a shifted sine.
The frequency is the constant in the argument of the sine. And here is where the tail might chase the dog: it's the definition, not an observation I'd think.
> When performing a Fourier transform, we represent the signal in a new basis, where each component
... each component is a sine, that is described by the 3 constants above.
> In simpler words, it comes down to the barber pole illusion. If you rotate a spring-shaped 3d curve, it looks as if it was traveling upwards. And the 2d projection of the spring are Sines and cosine.
Exactly, a complex exponential function is just that spring. And if the absolute value of the argument is not exactly one, the spring spirals outwards or inwards.
But this is also true of any periodic wave. The beginner question is, why don't we use triangle waves of square waves? Those also have amplitude, frequency and phase. Frequency just means the reciprocal of the period.
To which my answer is that the magical property of complex exponential functions is that they can be shifted by constant (pointwise) multiplication. Which is a really non-obvious fact at first but is crucial in the machinery.
The complex exponentials constitute an orthogonal basis which diagonalizes the convolution.
(A deeper thing going on is that a periodic function can be thought of as a function whose domain is a circle. Circles have obvious rotational symmetry, and when you have symmetry you can use representation theory to decompose things into an (orthogonal) basis. In this case, rotations commute with each other so by some theory the decomposition is going to be entirely through eigenvectors of rotation, which happen to precisely be the exponential functions e^(n theta i) for n an integer. This decomposition is also an isomorphism that carries convolutions to point-wise products in both directions. Also: if you make it so the circle is the complex unit circle, a Fourier transform is the idea that you can create a Laurent polynomial that extends the function to the complex plane minus the origin.)
Minor point: to me derivatives are just one specific linear time invariant operator, a kind of convolution (with a generalized function) so I think LTI is the thing we really care about.
The Laplace transform seems to be just using the fact that <f,g> = integrate(f(x) g(x), x from 0 to infinity) is an inner product for the space of square integrable functions (probably better would be <f,g> = integrate(f(x) conj(g(x)), x from 0 to infinity) as a Hermitian product). The various e^(ax) functions are linearly independent, so the functions g |-> <e^(ax), g> are linearly independent functionals. If the exponential functions are actually enough, then this means you can study a function by studying the vector consisting of its value through all the functionals, which is the Laplace transform.
The Laplace transform has a pretty bad inverse formula, partly because the exponential functions are not orthogonal with respect to the inner product.
You can think in terms of complex exponentials, and the use of complex numbers makes even more sense when you know the differential equations can be solved by polynomial methods, which naturally leads to complex numbers.
However you can also answer "why sine waves not triangle/square waves" with a geometrical answer. (This is how I learned it at school, before I knew about complex numbers or differential equations.)
A property of sine waves is that their sums and products are also sine waves or simple combinations of a small number of sine waves.
The sine wave shape persists. This neat property is unique to sine waves, and you can think of it as a type of symmetry.
For example, in the simplest cases: adding two sine waves with the same frequency and different phase produces another sine wave with that same frequency. Multiplying two sine waves with different frequencies produces the same as a sum of two sine waves, having the sum-of-frequencies and difference-of-frequencies.
Doing so with any other wave shape results in a different wave shape than you started with.
A lot of audio and radio signal processing depends on this property, and the way sum-of-frequencies and difference-of-frequencies works for every sine wave component at the same time. Our whole concept of a radio "band" of signals that you can tune into comes from it.
The math is also simpler with sines and cosines which makes a difference for both practical implication and learning.
Why we break arbitrary periodic functions into a sum of sines and cosines (or circular motion of different integer frequencies) is because uniform circular motion is very well understood and studied, so we have many tools for working with it. It’s very easy and convenient to isolate particular terms, and each term has a pretty simple shape, and is infinitely differentiable. These bases are conveniently orthogonal relative to a uniform weight.
As has been pointed out elsewhere in this thread, they have nice mathematical properties. But another important thing is that they typically work "well enough" for applications. Consider audio. Tones clearly have frequency, but they also have a position in time. Doing a sine-cosine decomposition of a whole song doesn't really make sense, since it has no way of saying that a tone on the piano is played at a given time.
So you would think that it would make sense to break the signal down into stuff with frequency and time. Some kind of wavelet probably. Maybe something that very accurately models what a human hears.
The thing is that chopping the audio stream up into windows, and decomposing those windows into sines and cosines, while a bit ad hoc, just works well enough.
Interestingly wavelet based compression went nowhere because although they have nice mathematical properties, when applied in a lossy compression scheme, they did not fit well with how humans perceive detail/quality, both in terms of psychoaccoustics and psychovisuals, i.e. PSNR vs subjective quality diverged more than with other systems. Not surprisingly none of the state of the art lossy compression algorithms use wavelets.
Or to put it another way, if you imagine being faced with diff eqs and thinking "Hmm, is there a way I can turn these into problems I already know how to solve... like, say, polynomials?", then the idea that there even exists an invertible transformation that lets you do such a thing is itself quite fascinating in its own right—regardless of any elegance the transformation itself might have from a theoretical mathematical standpoint. And having such a motivation and application in mind (and knowing the physical nature of the problem, where we know we'll end up with waves) helps ground the idea in something very concrete and avoids making it look like it's just a random transformation we're studying because we're bored.
It's about FFT but it goes through the (less known) polynomial multiplication foundation which seems more practical and related to the convolutional applications.
[0]: https://doi.org/10.1093/comjnl/9.4.404
[1]: https://en.wikipedia.org/wiki/Chebyshev_nodes
[2]: https://en.wikipedia.org/wiki/Discrete_cosine_transform#DCT-...
We use sines and cosines, but not only sines and cosines. There are many other interesting sets of functions. For example, polynomials, or derivatives of the gaussian functions (called Hermite functions).
Bonus point: sines and cosines are polynomials evaluated on the complex unit circle.
> Basically the Fourier basis diagonalizes the convolution operator.
It also diagonalizes the second derivative, which is the linear operator that governs many physical processes, like wave propagation and heat diffusion.
Could you explain further?
https://jackschaedler.github.io/circles-sines-signals/index....
In a nutshell Fourier transform, the derivative operator, exponential function and CAUSALITY are related at a very deep level. And that's why it's such an important mathematical tool. Because it is related to causality.
Thanks.
https://www.youtube.com/watch?v=GbqA9Xn_iM0&list=PLPH7f_7Zlz...
Notably, it treats QM with much more rigor than is usual in introductory courses, so having a nontrivial mathematical background is not an obstacle.
https://www.discovermagazine.com/mind/the-brain-ringing-in-t...
Yes, yes, yes, yes! Anybody teaching really has to understand that if you need a formula to explain a concept, you haven't understood the concept yourself OR you simply don't know how to explain it yet.
Sure, there are exceptions (there always are), but in my experience, lecturers, teachers and academics reach for formulas to "explain" things too quickly. Formulas are not explanations. They serve as PROOF that what you explained actually works.
https://www.youtube.com/watch?v=yyox358zIRw
You can use FFT principles to isolate and remove repeated patterns in images (such as the patterns created by those textured papers of old photographs).
https://www.3d4x.ch/Swift's-Reality/FFT-Photoshop-plugin-by-...
I've personally used this in a project and the results are impressive.
I enjoyed looking at the square sound, as it now makes sense why a square sound is a lot fuller. It simply has a lot more waves for your ears.
Also the short JPG intro was really good!