Hacker News Reader

Comment by nonrandomstring | Hacker News Reader

+1 for the recommendation for "The Scientist and Engineer's Guide to Digital Signal Processing" by Steven W. Smith. I spoke to Steven many years ago about his putting all of the book PDFs online for free and since then have recommended at least 5 university libraries to buy it because the students could get free copies of a great book that can be referenced in the library.

FWIW the The Scientist and Engineer's Guide doesn't actually cover a lot on sound. It starts in a particular DSP way with frequency domain definitions and convolution - and I actually think Steven's background is in medial imaging, though I could be mistaken.

View on HN

Wow. Such a profound insight could not get published for 15 years because others couldn’t grasp it. Including Lagrange!

It’s an uncomfortable reminder of how essential reputation and credibility are in the machinery of science.

”The paper contained the controversial claim that any continuous periodic signal could be represented as the sum of properly chosen sinusoidal waves. Among the reviewers were two of history's most famous mathematicians, Joseph Louis Lagrange (1736-1813), and Pierre Simon de Laplace (1749-1827). While Laplace and the other reviewers voted to publish the paper, Lagrange adamantly protested. For nearly 50 years, Lagrange had insisted that such an approach could not be used to represent signals with corners, i.e., discontinuous slopes, such as in square waves. The Institut de France bowed to the prestige of Lagrange, and rejected Fourier's work. It was only after Lagrange died that the paper was finally published, some 15 years later.“

from chapter 8 http://www.dspguide.com/ch8/1.htm

> Lagrange had insisted that such an approach could not be used to represent signals with corners, i.e., discontinuous slopes, such as in square waves.

Isn't that true though? You can only approximate it, right?

In practice, yes you can only approximate it, because in theory it requires an infinite series of sinusoidal waves. But since maths is all about the theory, Laplace's objections were silly. (though I admit I don't know what the state of acceptance of infinite series was at that time)

Also, it's fine to cut off the infinite series of sinusoidal waves because all physical systems cut off around a given frequency, for example human hearing around ~20kHz.

Also, in practice, there's no such thing as a truly sharp corner, which makes sense intuitively based on the math, I think. But real systems have inertia, friction, stray resistance/capacitance/inductance, etc. so you can't have a truly perfect square wave in a physical or electrical system in the real world.

* Lagrange's objections

whoops! thanks. Too late to edit.

Not quite, that's not the problem. We're in the realm of maths here and therefore of pure abstraction, so that's not the point. Adding terms to the Fourier series of a square wave you have an overshoot that does not converge to zero as the number of terms goes to infinity. This is called the Gibbs phenomenon https://en.wikipedia.org/wiki/Gibbs_phenomenon

In the same sense that a circle does not exist, because you can only approximate it with an arbitrary number of points/lines. But an infinite number of points the same distance from the origin is a good definition of a circle. In a similar way, an infinite set of sine waves approximating a square wave is a square wave, mathematically speaking.

Sort of. The error at a square wave's "cliff" does shrink in width as you add higher and higher frequencies to the approximation, but the height of the error does not diminish, so you have to be explicit about what you mean by "converge". If you considered MAX(ABS(approximation(x) - square(x))) as your error metric it would not go to zero even in the limit; for most intents and purposes we don't care about that, though, and in any case the limiting behavior is much nicer for "normal" functions that don't have discontinuities like a square wave.

Corners in signals are like limits in calculus or irrational numbers in arithmetic. Once you accept them as an abstraction that you can't touch them directly all the anxiety dissipates.

Perfectly square waves also can't physically exist, as long as there is a speed limit on the universe.

I have this conviction that I ought to be able to code up crude audio DSP processors before I understand the totality of the theory.

For example, every DFT/FFT explanation seems to start with complex numbers. I wish there was a resource that was programming focused, starting with "Step 1: Process this artificially created periodic signal by multiplying it by sine waves of frequencies from from 1 to N. There are our bins! Step 2: OK, for real world signals we need phase, so now let's talk about complex numbers."

I've been studying math for a while trying to build up the prerequisites for writing audio DSP code. I have this sneaking suspicion that at the end what I want to achieve won't be as hard as DSP resources imply. At least there will be parts that I could have done with hardly any theory at all. But because of the way these resources are written, I have to consume massive amounts of theory first.

(For background, I've worked as a mastering engineer and have done a fair amount of audio production, so I know intimately what the tools ought to do.)

I've been writing notebooks about doing DSP and audio processing in JavaScript. It might be a good place to crib off of:

https://observablehq.com/collection/@skybrian/digital-signal...

Wow, thank you very much! Straight to the point and very helpful, especially the web audio stuff.

> "Step 1: Process this artificially created periodic signal by multiplying it by sine waves of frequencies from from 1 to N. There are our bins! Step 2: OK, for real world signals we need phase, so now let's talk about complex numbers."

Maybe the latter part could be prepped by resynthesising the time-domain signal by summing the sines, and seeing that it doesn't match the original. And it can't, not least because all of the sines start at 0. But if you have cosines as well, it can. Then refer to the geometrical relationship between sine/cosine and phase.

A preliminary to the earlier part might be to multiply a single long sinusoid by another one, and see what happens when their frequencies do or don't match. (But there is a whole well here about what it means for frequencies to "match", which in the discrete world has to do with how long the relevant part of the signal is.)

I posted this as a submission a while ago but you might find it helpful. While learning about signal processing my self I built a collection of educational tools for visualizing some concepts. [1]

Especially the fourier cube [2], the complex exponential [3], the digital filter designer [4] and the signal generator [5] might be helpful.

Additionally the matrix multiplier [6] has an option for complex numbers that highlights the perspective that complex numbers can be seen as just a subset of 2x2 matrices.

[1]: https://tools.laszlokorte.de/ [2]: https://static.laszlokorte.de/fourier/ [3]: https://static.laszlokorte.de/complex-exponential/ [4]: https://static.laszlokorte.de/signal-transform/ [5]: https://static.laszlokorte.de/signal-generator/ [6]: https://static.laszlokorte.de/matrix-multiplication/

Your intuition is right, in that most of the heavy math is simply unnecessary or irrelevant at the practical level. Complex number notation doesn't fall into that category, though. The notion of frequency is intimately tied to the notion of vector rotation, and just as you can't represent Cartesian translation without negative numbers, you can't represent Cartesian rotation without complex ones.

The Smith book is definitely for you; also check out Rick Lyons's books. Bo Pirkle and Julius O. Smith are good for audio-specific theory and applications.

Saying that you can't represent cartesian rotation without complex numbers seems a bit much? You do need two numbers. But you can teach a trigonometry class without complex numbers, using either cartesian coordinates or polar notation. And matrices do work.

I once found one article that started with the concept of "probing" with a test signal, and then you need to probe with both sine and cosine to reconstruct the phase. Complex numbers seem to be a neat mathematical notation for this sin/cos pair.

The book is amazing: http://www.dspguide.com/ (link for the lazy)

pdfs are the chapters are available freely on that website, but if anyone wants to spare themselves the effort of combining them all into one for easier perusal, feel free to email me (in bio) and I can send the combined pdf. I'm currently around page 200, and it's been great as an introductory resource as someone with no other background in DSP.

Here [1] I found a link [2] to download all chapters in one .zip file.

[1] https://www.analog.com/en/education/education-library/scient... [2] https://www.analog.com/media/en/technical-documentation/dsp-...

I'm actually reading through it right now after trying out a few other DSP resources. It is by far the clearest and most approachable, without sacrificing much depth at all. It's a great book.

It's the best. I learned DSP from that ~20 years ago and still use those mental models whenever I need to think through a problem. A good book like that is a friend for life.

It's one of my favorite technical books. So simply explained, so full of engineering wisdom.

Reply on news.ycombinator.com