Comment by wnewman | Hacker News Reader

Comment by wnewman | Hacker News Reader

The author wrote "This formula, as anyone can see, makes no sense at all. I decided that Fourier must have been speaking to aliens, because if you gave me all the time and paper in the world, I would not have been able to come up with that." That sounds like a predictable symptom of trying to understand Fourier analysis while avoiding linear algebra. And that seems like unnecessary masochism, because basic linear algebra is very useful and pretty easy. And once you have it, (elementary) Fourier analysis becomes trivial to understand as a change of basis by recognizing the supposed "no sense at all" formula as a perfectly sensible change of basis to a basis of sinusoidal functions.

Then you just need to understand that the sines and cosines are a complete basis. So think about the sines and cosines for a while until you can say "yeah, they're orthogonal, and I can believe they're a complete basis for the kind of functions under consideration." Then to promote this from "I can believe" to "obviously," for the discrete FT (the orthogonality and) counting/dimensionality arguments suffice, and for the continuous FT you can look at Gaussians, say "obviously Gaussians are a complete basis for the kinds of functions under consideration" and then do the easy integrals to show that any Gaussian can be expressed as a linear combination of sines and cosines.

(This assumes you're interested in transforming reasonably smooth things like wavefunctions in chemistry, as opposed to trying to see how far you can push Fourier analysis into the netherworld of bizarre jagged twisted functions shown to exist by invoking the Axiom of Choice. If you want to do that, feel free to take a course from Terence Tao studying theorems whose prerequisites involve concepts like "countable.")

Obviously the author was exaggerating for effect and humor. Rather than characterizing his description as "avoiding" linear algebra, I say the author admirably confronted his confusion head on and developed his own intuition. Even more admirable is that he shared it.

Good points.

To add to this: if you want an intermediate path that'll give a bit more insight than just basic linear algebra, reading a bit about Hilbert spaces might be worthwhile too. Depending which space(s) you choose to work in, it doesn't necessarily have to bog you down in coping with pathological cases. You probably will need to grok things like countability, limits and converge though.

The benefit: you get to pin down exactly what it means to talk about things like 'basis' and in particular 'orthogonal basis' in a meaningful way when it comes to infinite-dimensional spaces of functions like the ones the Fourier transform works on. Turns out you need some extra tools to do this; intuitions based only on experience with finite-dimensional linear algebra will likely have some handwavy gaps. (Not that that necessarily matters :)

About the article though: I really liked it. It wouldn't teach me from scratch how the transform works, but every extra bit of intuition one can get from looking at a concept from a different point of view helps, and this adds some extra physical intuition in a really nice way.

Linear algebra alone isn't enough, you also need complex geometry for Euler's formula - I can't just look at exp(0+xi) and go "hey, that's cos x+sin x i". And even then, you need to go a little way into linear algebra to encounter basis functions. Finally, you pretty much need someone to assert orthogonality to even consider it.

And if you aren't working with wavefunctions regularly, like physicists and engineers might be, that gets obtuse fast (I'm referring to the use of the FT in econometrics and image processing). Then it's not at all clear that you should even want a sinusoidal basis with integer frequencies.

To illustrate your point, here is a java applet plus source code visually explaining not only the discrete cosine transform but also principal components analysis and Haar wavelets as all being the same exact operation with a different orthogonal basis:

http://www.cc.gatech.edu/~phlosoft/transforms/

I agree, this is the best way of looking at the issue. However, this approach requires one to learn a more advanced material first. It’s probably inevitable, but it may be a good idea to start with something more elementary. I’d suggest to start with a loose analogy with Taylor series (calc2).

Reply on news.ycombinator.com