Hacker News Reader

Comment by lbourdages | Hacker News Reader

I've been trying to find the source for a while, but I recall reading about a fundamental property of analog computers that makes them inherently unstable (as in, noise will always win on the long run), unlike digital computers. No matter how good the design is, little bits of noise add up in complex analog computers and the output is inexact.

I guess my Google-fu is good enough, because I've been unable to find where I read about it.

View on HN

Multiplying two numbers in an analog circuit requires taking the logarithms and add those, then converting back, or some similar trick.

In practice this is done using non-linear effects of transistors[1], however the exact details of those effects are individual to each transistor and is also temperature dependent.

Since the multiplication circuit relies on different transistors behaving identically, compensation circuitry and trimming is required, which will never be perfect.

[1]: https://www.analog.com/media/en/training-seminars/tutorials/...

Is it true, though?

Look deep enough, and every modern digital circuit is emulated by an analog circuit, just because the components are analog in their nature, and the 0s and 1s are an interpretation of analog data. That includes digital computers.

Does this make digital computers inherently unstable? Clearly, simple enough computations both of digital and analog kind are good enough to be useful. So there must be a breaking point further away, but on what axis?

A transmission line accumulates noise at it increases in length. Eventually the signal to noise ratio is too high. In old school analogue comms, the solution is to decode the signal to the digital domain before it reaches that point, which allows us to drop the noise. Then the data is reencoded back to a clean analogue signal again.

In digital logic, this process happens at every gate. Hence the reliability of digital logic.

Analogue logic doesn't do this. So analogue logic is only useful if the noise introduced at each step is lower than the error from your source data was already (at whatever point in the computation you have reached). If there is a way round this, I don't know it.

There is a way around this: discretize your values. That's how analog calculations can be a basis for digital calculations.

That's why I think the original comment was about a specific, limited meaning of "analog computation" that does not allow for emulating anything digital. But I struggle to come up with one that doesn't throw the baby of being universal out with the bath water of emulating digital computations.

There's a term for analog systems using discretized values: digital.

Exactly! And under that lens all fundamentals applying to analog also apply to digital: that they are inherently unstable and that noise will always win in the long run.

Which sounds at the very least imprecise to me, considering that I'm writing it on a digital (and therefore analog) computer.

In fact, Von Neumann proved mathematically that it's possible to build a reliable system out of unreliable components. But, modern digital logic uses a simpler mechanism: the thing you are missing is that noise does not propagate or accumulate in discretize (digital) systems, as long as it remains below a threshold, and circuits are designed so that it is far below the threshold.

Are you thinking about the property of analog data that it can't be stored, reprocessed, or copied without the noise increasing?

That doesn't make the computers unstable. It just makes the data less fit for long-term storage. And even then, people manage.

No, it wasn't about storage. It was really during calculations, you can't prevent the noise from affecting the results, from a fundamental level.

For any complicated calculation, you need to store and propagate multiple intermediate results.

This is what goes wrong.

error(x + y) > error(x) + error(y)

Floating point calculation is "noisy" and yet there are models that use smaller and smaller floating point numbers without losing much accuracy.

Oh, ok. You can't.

I'm not sure you can find a citation though, it's like searching for a source that the sky is blue.

Keep in mind that digital calculations have noise too. The digitization noise behaves in a completely different way, but any single computer has a finite precision whatever the technology behind it. Infinite precision doesn't exist on the real world.

Could this be the reason why human (and things with brains) need to sleep from time to time? Sleep would reset the accumulating noice from the brain.

It's not really clear what you mean by the brain accumulating noise. An analog computer can suffer from compounding imprecision, because error bars carry forward. We are analog computers in a vague sense, but as humans we aren't imprecisely crunching numbers all day such that our results are out of whack by nightfall... You could say sleep is like a reset for the brain, but that doesn't say much. Sleep is complex and relates to most everything else about an organism in complex ways.

Nah, humans just blank-out for a couple of seconds mid-task instead.

Reply on news.ycombinator.com