What Is Analog Computing?
quantamagazine.org
quantamagazine.org
The article seems to put a bit of a silly spin on an interesting subject by wanting to present analog computers as a potential alternative to digital ones, when in fact that is only rarely going to be possible.
Digital computers support programming - solving of problems by describing the process by which they are to be solved, whereas analog computers instead support solving of (limited types) of problem by building an analog of the problem (which you could think of as "describing" the problem, rather than how to solve it), either using analog electronics (a patch panel analog computer), or by building a physical analog of it such as the Antikythera mechanism which models the motions of the planets.
The article oddly omits any mention of quantum computers which (unless I'm totally misguided!) are essentially a type of analog computer - one that is no replacement for digital computers and general problem solving since it can't be programmed, but rather only works when one can configure it to be an analog of the problem to be solved. As with any other type of analog computer, the problem is then solved by letting the system run and the dynamics play out.
This connection opens doors for research in both directions, how to design quantum algorithms and how to built physical computing systems that make use of quantum mechanics.
Analog quantum computers (like annealers) are simpler to make, and they have a lot more play w.r.t. their building blocks when trying to make interesting effects... but ultimately they are limited by noise. Digital quantum computers restrict themselves to finite gate sets, often requiring expensive decompositions to do basic operations (e.g. [1]), but those gates are compatible with error correction so noise can be suppressed arbitrarily (e.g. [2]). It's very similar to how an analog classical computer can do addition with two resistors and a junction, but the accuracy is limited by the precision of the resistance. Whereas a digital classical computer will decompose the addition problem into bits and gain more precision by adding bits so that increasing precision becomes about increasing quantity instead of quality.
For example, a chemistry simulation can be done in first quantization; where the state is a list of superposed 2s-complement integers indicating the positions of the electrons (as opposed to a more direct one-qubit=one-position mapping). And the list is initialized in a way that satisfies the Pauli exclusion principle by using a sorting network [1]. This is presumably not at all how Nature does it.
Another example is factoring. Shor's factoring algorithm is not at all like behaving analogous to a physical system. It's about modular exponentiation and Fourier transforms; math things not physics things.
Yet another example is Hamming weight phasing. If you need to rotate many qubits by a common angle around the Z axis, you can achieve that effect more cheaply by temporarily computing their Hamming weight (under superposition) and then rotating the first qubit of the Hamming weight register by the angle, the second by twice the angle, the third by four times the angle, etc. And actually that various-rotation-angles operation can then also be replaced by an addition into a special reusable phase gradient state; this achieves the desired effect by phase kickback [2]. Adding the Hamming weight of their spins into a helper state is probably not how Nature goes about precessing electrons in a uniform magnetic field.
[1]https://brainscales.kip.uni-heidelberg.de/
edit: newer link: https://open-neuromorphic.org/neuromorphic-computing/hardwar...
I work with QDI systems, and I've long suspected that it would be possible to use those same design principles to make analog circuits robust to timing variation. QDI design is about sequencing discrete events using digital operators - AND and OR. I wonder if it is possible to do the same with continuous "events" using the equivalent analog operators, mix and sum.
[2] https://open-neuromorphic.org/neuromorphic-computing/hardwar...
Just read about it and there are familiar names on the author list. I really wish this type of technology gained more traction but I am afraid it will not receive the deserved focus considering the direction of current research in AI.
> I wonder if it is possible to do the same with continuous "events" using the equivalent analog operators, mix and sum.
Don't know much about QDI but sounds promising.
Worth noting the exception here which is current loops.
This is a rare misfire from Quanta. No, there is no practical way to model anything non-trivial - especially not ML - with analog hardware of any kind.
Analog hardware just isn't practical for models of equivalent complexity. And even if it was practical, it wouldn't be any more energy efficient.
Whether it's wheels and pulleys or electric currents in capacitors and resistors, analog hardware has to do real work moving energy and mass around.
Modern digital models do an insane amount of work. But each step takes an almost infinitesimal amount of energy, and the amount of energy used for each operation has been decreasing steadily over time.
Are you aware that multiple companies (IBM, Intel, others) have prototype neuromorphic chips that use analog units to process incoming signals and apply the activation function?
IBM's NorthPole chip has been provided to the DoD for testing, and IBM's published results look pretty promising (faster and less energy for NN workloads compared to Nvidia GPU).
Intel's Loihi 2 chip has been provided to Sandia National laboratories for testing with presumably similar performance benefits as IBM's.
There are many other's with neuromorphic chips in process.
My opinion is that AI workloads will shift to neuromorphic chips as fast as the technology can mature. The only question is which company to buy stock in, not sure who will win.
EDIT: The chips I listed above are NOT analog, they are digital but with alternate architecture to reduce memory access. I've read about IBM's test chips that were analog and assumed these "neuromorphic" chips were analog.
The key thing is that analog components are influenced much more by environmental fluctuations (think temperature, pressure, and manufacturing), which impacts the "compute" of an analog network. Their novelty was that the chip can be "trimmed" to offset these impacts using floating gate MOSFETs, the same that are used in flash memory, as an analog offset. It works surprisingly well, and I suspect if they can capture the low power market we'll see a revitalization of analog compute in the embedded space. It would be really exciting to see this enter the high bandwidth control system world!
"The advantages of digital computing are real, but so are the drawbacks. Perhaps, by reaching back to computing’s past, researchers will be able to steer a sustainable path toward our computational future.
Correction: August 2, 2024: The $100 billion data center would require 5 gigawatts of power, not 5 megawatts."
So it's the end of the article indeed.
Thank you for the link! :)
https://museum.syssrc.com/artifact/exhibits/251/
I’ll note that there’s been plenty of work on general-purpose, analog computing. Analog’s competitiveness also lives on in mixed-signal ASIC’s. The problem with new products, though, is analog requires fully-custom, circuit design. Outside some niche uses, you can’t just synthesize them from higher-level specs like you can digital designs with standard cells.
That puts analog at a severe disadvantage on NRE cost. Especially as both design rules and mask costs increase with every node shrink.
This is why now a days bleeding edge process nodes might be lacking in IO (especially fast serial IO) when they're first released.
Additionally, analog just hasn't scaled for quite a while now. Even traditionally analog circuits like PLL's and IO are getting more and more digital as the digital scaling is making more digital architectures competitive in space and power when they weren't before.
"But it’s not obvious why a system that operates using discrete chunks of information would be good at modeling our continuous, analog world."
OK, fine. But, if my "discrete" modelling is using 64 bits of precision, or, in decimal, about 19 digits of precision, and your analog system on a good day has about 3, how could your analog system even "tell" mine is discrete? It can't "see" that finely in the first place.
Oh, you don't like how it's excessively precise? It is off-the-shelf numerical tech to include precision in calculations. It isn't something every programmer knows, but it's been in books for years.
Oh, by the way, it is also off-the-shelf numerical tech to estimate errors using digital computers. That's going to be a hard trick to match in your analog system. Also, digital systems offer many tradeoffs with regard to error levels and computation time. Another thing your analog system is going to have a hard time with.
"But floating point numbers have lots of problems with instability and such." Most of those problems will manifest in your analog computers, too. Many of those aren't so much about floating point per se as just doing inadvisable things with numbers in general, e.g., if you set up an analog computer to subtract two quantities that come out "close to zero" and then divide by that quantity, your analog computer is going to behave in much the same way that a digital computer would. Ye Olde Classic "add a whole bunch of small numbers together only for them all to disappear because of the large number" is also the sort of problem that you can only get because digital is good enough to reveal it in the first place; accurate adding together millions of very tiny numbers is something your analog computer almost certainly just plain can't do at all.
A carefully crafted analog system might be able to outpace a digital system, but it's not going to be the norm. Normally the digital system will outpace the analog system by absurd amounts.
Now, are there niche cases where analog may be interesting? Yes, absolutely. Neural nets is an interesting possibility. Obviously a big possibility. There's a handful of others. But it's a niche within a niche. There's no problem with that. There's plenty of interesting niches within niches. You can make careers in a niche within a niche. But I don't know what it is about this one that convinces so many people that there's some sort of revolution budding and we're on the cusp of an amazing new era of analog computing that will do so many amazing things for everyone... as opposed to a niche within a niche. We had that revolution. It was digital computing. It is the single largest revolution in the history of man as measured by factors of magnitude of improvement from start to now, and the ditigal revolution is still not done. Analog is not going to overtake that. It doesn't have the headroom. It doesn't have the orders of magnitude improvement possible. That's why it was left behind in the first place. The delta between my kid 3D printing a clock mechanism and the finest Swiss watchmaker who ever lived is still a small single digit number of orders of magnitude in terms of their mechanical precision, and in the digital world we still expect that degree of progress about every 5-10 years or so... and that's slower than it was for many decades.
Noise is always a problem. Just trying to measure a small signal is a problem since your instruments (which are not perfect) can influence the circuit. Component tolerances are also a problem - if you're trying to do something super precise, you'll have to spend extra money on better components, which is why measurement instruments such as oscilloscopes are so expensive.
In some ways the hardware fights against you in analog design. Digital just brushes that away by clamping all possible values to just two.
Actually, a digital potentiometer is likely used today to tune extremely tight tolerances. So a bit of digital technique but overall it's analog (since digital pots are just turning analog resistors on/off with switches)
You buy 5% accurate components everywhere then just trim the errors down to 0.01%. Not easy per se, but 99.99% accuracy is doable at the student level with the right designs.
Professional level precision analog designs are the 7+ digit multimeters (99.99999% accuracy). Yes, built out of just 1% accurate parts.
If you need matching pairs, they make specially matched resistor chips or whatever.
--------
The concepts aren't hard btw though maybe slightly out of date. The book Art of Electronics analyzes a 7+ digit accurate multimeter design from Agilent or something.
Instead of using an analog pot to tune, today you'd use a digital pot. Otherwise, I'm pretty sure the overall analysis and technique holds in today's electronics market.
VLSI does have techniques though. A clocked capacitor can sorta-kinda act as a trimpot if you calibrate the clock for example. And capacitors and clocks are more amicable to VLSI.
I think it's because they look so fucking cool.
I don't really know much about analog computers, but I assume they're going to need at least an op amp per analog adder and those aren't going to be small.
For example: 24-bit ADCs may exist but with nonlinearity errors, offset errors and noise problems.
You will likely get better results out of Instrumentation OpAmps (specifically designed OpAmps for (A-B) * C type operations). Less noise, more accuracy and more linearity across wider voltages and currents.
------
And if my Analog design is already better than the best possible A-to-D converters out there, then no digital technique can ever possibly catch up. Just converting the analog signal to the digital world already introduced significant errors before the computer even booted up... Let alone the final D-to-A conversion back. (Most things today need a voltage or current applied to operate, possibly in proportion to the computed values).
I expect radio (WiFi, cellphones etc. etc.) to remain analog for the near future. SDR techniques are crazy good today but still expensive.
I'm comfortable calling it that. I'm happy to take an expansive definition.
When one end of the output is analog, it makes perfect sense that you may have a stack of components that are operating in that regime. Similarly I'm sure top-end robot arms have a lot of components designed to do things that you could model in the control software as some operation, but it's easier in every way to add things like real physical dampening and such. That sure isn't going anywhere, but neither is that about to start displacing the digital portions; the symbiosis is already near optimal probably.
So they don't have to do everything at once.
(the joke is that analog computers don't have clocks... Do they?)
Laws of Physics?
Does the animal brain have a clock?
[1] ChatGPT says otherwise — says, yes, they did have a clock of sorts. I'm wondering if something like a voltage sweep (triangle or sine wave?) would be a kind of "clock input" for an analog computer though.
Not a digital clock, no.
But your modem controller (2.4GHz, 5GHz or whatever) has a frequency that the radio analog chips are adding, multiplying, exponenting and decimating to pull the data out of the radio waves.
And computation to pull data out of (or push data into) a 2.4GHz channel is best done with analog today. Sure, SDR exists but its far more expensive than some PLLs and other analog techniques.
And yet not even one paragraph in this article?