Analog Computers
blog.degruyter.com
blog.degruyter.com
https://en.wikipedia.org/wiki/Analog_computer
Takeaway: Analog computers are limited in precision and by "analog noise"; the precision of the components used determine the precision of the output. Usually no more than 3 or 4 decimal places are possible, at least with the tech that was used in their heydey. I would say that is still close to the case even today. Of course, one could do things like cryogenic cooling or such, but it becomes a cost factor at that point.
Something else that wasn't mentioned:
The ADALINE/MADALINE memister technology is an analog component, and is a hardware component equivalent to a McCulloch–Pitts neuron model (perceptron).
The memister is NOT to be confused with the memristor, which is a different technology; the memister was a 3-terminal device (memory transistor):
https://en.wikipedia.org/wiki/ADALINE
...whereas a memristor is a two terminal device ("memory resistor"):
("Somewhere" in the Concorde megathread is a description of its analogue computers: http://www.pprune.org/tech-log/423988-concorde-question.html )
- limited dynamic range of perhaps 30dB (1000)
- it is easy to saturate a signal (there is no overflow bit)
- oscillations are easy to induce, but once again hard to detect, especially in a circuit in the middle of a calculation chain
- noise gets amplified across the system
Would the surface tension across a film that accurately finds the shortest path between n points be considered a "computation"?
One thing fascinating by "analog computers" is the way they seem to be practically instantaneous regardless of n. That is perhaps part of the efficiency reflected in the article.
If this were true, it would have profound implications. It's probably not true.
There are also conceptual issues that the airflow across a wing is ideally under stable low angles of attack, constant, but during "fun times" which is precisely when you'd want a model instead of a test pilot, the air flow will vary over time (to the general detriment of flying ability...) so what it means to instantly solve a wing is unclear in itself.
By that do you mean accurate to 1 part in 100 (3dp) or 1000 (4dp) or what? Since the scale of a representation is arbitrary, I'm not sure what dp means here.
This could be expressed, at the end, simply in decibels, though. Signal-to-noise, as in classic analog systems.
The typical problems with analog computers are many... precision of components (e.g. gain or attenuation) is limited to ~0.1% for resistors and ~1% for capacitors (inductors aren't typically used). You can try to tune things (ratiometrically) to get higher accuracy, but at the cost of increased noise and temperature sensitivity. The more complex the system, the more things can go wrong... so you end up needing simple systems or simple tools (digital).
The typical problem is that if you build a filter (e.g. a transfer function with a summer or differencer) then you will tend to clip the dynamic range or either with a maximum voltage (integrators) or a minimum noise level (differentiators) pretty quickly. You can play some games with log converters, but accuracy really still matters and drift or gain error with time is rarely an option.
The best way to use analog computers is with negative feedback to null the input. They do that amazingly well... so you can build a temperature controller, missile tracker, or actuator that only minimizes an error so that high gain corrects for any inaccuracy or offset.
If you make a graph of PITA vs bit resolution, we're all pretty comfortable emulating digital computers on analog real world circuits using binary ones and zeros. Surely the gain is very little and the PITA increases very much by implementing digital computers on trinary + - 0 analog computers. Some think the graph is U shaped and at some resolution level, the PITA of analog high resolution falls beneath performance so it makes sense. Many like me think that graph never U shapes such that anything is "better" at emulating digital computers than using analog physical computers based on binary 0/1. AFAIK no one has built a modern floating point accelerator using opamps and A/D and D/A converters, so I find it unlikely its useful.
A two transistor NAND gate is after all just a analog computer using simple binary signals. All computers are analog its just the popular digital ones are only defined and well behaved when using binary analog signals.
There is some audiophile effect going on. Surely a mp3 codec running on a vacuum tube opamp would sound more mellow and all that.
http://www.cisl.columbia.edu/grads/gcowan/vlsianalog.pdf
http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.325...
Now, I won't argue with cheaper to develop since analog is manual with a lot of issues to contend with. I'm just wondering if there are more applications that can get huge speedups at lower power or cost than digital. I know the ASIC makers in power-sensitive spaces are already moving parts of their chips to analog for power reduction. That's what mixed-signal people tell me anyway: the specifics are often secret. So, I have to dig into CompSci looking for what they've tried.
Couple of points from your lower link:
- return of "wafer-scale"! Nice.
- " the average power consumption is expected to stay below 1 kW for a single wafer"; not bad but you're still going to need to cool that
- actually a hybrid system: long range comms is digital and multiplexed to save wiring, converted to analogue at the synapse
- "All analog parameters are stored in non-volatile single-poly floating-gate analog storage cells developed for the FACETS project" => basically analogue Flash? A development of MLC I suppose
On reading the whole thing, it seems the magic is actually in choosing which bits to make digital. The "long range" neural events are sent as differential 6-bit bursts, multiplexed, which they claim saves significant power.
This is one of the smartest things i've read on HN. I guess you are correct. Although who knows, perhaps a differential equation solver could be faster using D/A -> analog computer -> A/D?
Don't forget the temperature compensation! Then there's irreducible noise like Johnson noise. As you say, the best use is in (properly stabilised) feedback systems which seek to minimise a difference.
Slide rules absolutely give you decimal places. A decent slide rule might give you three decimal places of accuracy. A really good one might give you six decimal places, or 0.0001%, or 60dB. You could more precisely quantify their accuracy than just a rough order of magnitude, so perhaps the accuracy would be 55dB or 62dB, but "decimal places" gives you a sufficiently good idea of the accuracy for most purposes.
To bring it back to the digital comparison, a really great slide rule that's accurate to six decimal places is equivalent to a digital computer with 20 bits of output. If you put in a ton of work building an incredibly precise slide rule you might be able to add another order of magnitude and get seven decimal places. On the digital side, you'd only need to add 3 or 4 more bits to match that improvement.
So you could have a high gain but "low precision" (in the sense of deviating from an ideal, not in the sense of not being noisy) component, and through feedback you can make a low gain, high precision (having desired properties, not low noise) component.
I don't know the math of such things but did take a stab at it. My idea was doing something similar as we do for high or unlimited precision on digital computers. They usually emulate the higher precision using a series of lower-precision, primitive operations. My thought was that you could probably implement higher precision in analog if you could do a similar emulator with operations acting within the precision common in analog components. All I could guess at, though, since I'm in over my head here.
One other thing I always note is the brain seems to be mostly analog. Look what all it can do which includes memory and high-precision math. So, there's almost certainly some tricks we can use to do something similar with analog. Maybe an analog/digital hybrid. The wafer-scale project on NN's shows the potential esp if it was made 3D w/ a cooling system:
http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.325...
Funny enough, that is the difference between accuracy vs. precision, which I thought was irrelevant here. Thanks for pointing that out. :)
http://www.americanradiohistory.com/Archive-Elementary-Elect...
There's not a ton of content there yet, but please feel free to add anything you come across. I'm a fan of the idea and suspect, like the author of this piece, that there is "something there".
He developed Pertecs, which is a rudimentary analog computer paradigm written in C.
http://tcode.auckland.ac.nz/~mark/Signal%20Processing%3A%20P...
He got me to write some code to compile schematic diagrams into the XML config files. He's also done something similar now to compile from LaTeX, and he ported the controller from a Mac Mini to a Raspberry Pi.
Pertecs is being used to control an artificial lung, which is used for research into obstructive sleep apnoea (snoring).
The problem is, he's retiring, and I'm probably the only other person in the world who knows how to use his program. I would move back there, but the immigration policy got more difficult (minimum salary of $75k), so I'm seriously wondering whether I should stay in Taiwan longer and try to naturalise here.
That and I doubt you would find many people working at F&P Healthcare that earn less than $75k.
[1] https://www.immigration.govt.nz/about-us/media-centre/news-n... [2] https://www.immigration.govt.nz/employ-migrants/hire-a-candi...
$75k is 4x my current salary in Taiwan. Yes, I know the economy is totally different in NZ, but I don't have high hopes that changing country will suddenly make me become rich. My boss here pays me the minimum that the government allows for a Masters graduate on a foreigner work visa.
The other consideration is my girlfriend. She applied for Working Holiday, but wasn't one of the 600 lucky ones. We were in an internet café with the fastest connection in Kaohsiung, but the site just wouldn't load in time. She's 30, so she can't try again next year. If we wanted to get a partnership visa, we would have to live together and share a bank account for 1 year. Getting married doesn't even help, just living arrangements.
We're getting kind of sidetracked from the original topic of analog computers, but if it's something you want to talk about more, then just search for my name on Facebook and send me a message. It would be nice to personify the immigration forms.
https://en.wikipedia.org/wiki/MONIAC https://en.wikipedia.org/wiki/Hydraulic_macroeconomics
https://pdfs.semanticscholar.org/7244/1c8377b1dfde1909d21463...
[0] http://i.telegraph.co.uk/multimedia/archive/03593/emerson6_3...
And don’t forget, analogue random is the shit. In your face entropy!
Apparently something similar to FPGA exists for analog signals [1], I wonder how popular/practical it is.
[1] https://books.google.com.br/books?id=qjnnBwAAQBAJ&pg=PA93&lp...
Fun fact, with analog computing, one could imagine achieving Real Computation (https://en.wikipedia.org/wiki/Real_computation) which is above and beyond Turing. completeness.
Note the fun sentence "If real computation were physically realizable, one could use it to solve NP-complete problems, and even #P-complete problems, in polynomial time. ".
We are on a wrong evolutionary branch of computing. Bits are lame-o-rama, whereas differentiable signals are pure unadulterated flavortown.
Whether or not there exist physically realizable computations that are not computable by a turing machine is an open question, but most physicists and computational complexity theorists seem to believe there does not exist such a class.
what does it mean?
I'm familiar with the Turing thesis but he's wrong.
I know Scott Aaronson and all, but I won't believe it until someone tries to build one and fails.
Could you at least explain in what way you think he is wrong. Surely you must guess how participants on a programming forum will react to a statement like that.
Look into the work of Lenora Blum. She wrote a book "Complexity and Real computation".
You're going to have to back up a statement like that with a whole lot of supporting evidence if you want to be taken seriously.
This is the most important paragraph in the entire article.
Analog Computers can be made very small. It'd take an ASIC, but the 741 OpAmp was less than 100 transistors. A more modern OpAmp might be under 1000 transistors... although noise issues would be abound.
Bernd Ulmann has developed a methodology that performs non-trivial computations (such as: http://analogparadigm.com/downloads/alpaca_4.pdf), but its still hand-programmed by connecting wires together.
If it were digitally programmed with a digital crossbar switch (consisting of CMOS Analog Gates instead), then it'd be controllable by a real computer.
I think what Ulmann is arguing here... is to use analog computers as a "differential equation accelerator". Perform a lot of computations in the digital world, but if you need to simulate a differential equation, then simulate it on an analog circuit instead.
And there are a large number of interesting mathematical problems that are described as differential equations.
-----------------
The main issues, as far as I can see, would be the multiplier, logarithm, and exponential functions. IIRC, these are created using a bipolar transistor... and modern manufacturing doesn't really mix BJT with MOSFET.
I mean, IGBT transistors exist, but modern computers are basically MOSFET all the way down. MOSFETs would be able to make a lot of things though: digital potentiometers / variable resistors... the crossbar switch, capacitors, resistors, and OpAmps.
And all of those can simulate addition, subtraction, derivatives and integrals. More than enough to build a "differential equation accelerator" that the author proposes.
This is one of the worst "brain" analogies I've read lately.
That's why digital computers have so far been so much more successful than analog computers. While analog computers may have the potential to be "better", they'd require massive global hardware and software changes to become the next big thing. People are lazy, and there's no way they'd want to port everything over to a completely different paradigm just for better energy efficiency and speed.
Huh? So we now have computers more powerful than the human brain? I thought that was still some decades off. And how would one even measure such a thing? In the apples-to-apples comparison, a stupid human trick floating-point calculation savant might manage 1 flop/s.
I find it amusing that there is much hype about computer systems beating humans in very specialised areas, such as go and chess.
But the missing piece here is that the human is still doing this while continuously processing all the sensory input that is occurring to that human, dealing with so much more than what the computer system is dealing with. The computer system is dealing with one and only one subject matter at a speed many magnitudes faster and is only just getting ahead.
+1
Kasparov didn't simulate 200 million moves per second to make his move.
The problem is that it takes far more than raw computing power to make AI. We have sufficient computing power but we don't know how to use it, not even close.
As for how it's measured, it's basically a matter of guesstimating the computing power of a single neuron based on its inputs, outputs, and the computation it appears to do to map between them, and then multiplying by the number of neurons in the brain. This is horribly imprecise so estimates vary a lot (describing it as "38" gives the estimate way too much credit, should probably say 10 or 100 instead) but they're probably in the very rough ballpark.
I think the meme of the Universal Machine is causing some sort of phase transition in humanity as it propagates.
This is exactly why the digital computer has won over the analog computer.
Essentially, with an analogue computer you have a rack full of analogue building blocks and you build an electronic system equivalent to your real-world system from them. Then you can apply inputs and observe outputs. Often, the inputs were connected to sensors in a device, and the outputs were connected to actuators or recorders.
When analogue computers were already in wide use, there were maybe three digital computers on the whole planet. Later still, in the 60s to perhaps the early 80s "analogue computers" could (to varying degrees) perform some simulations orders of magnitude faster than contemporary digital computers. Only when digital computers became fast, cheap and easy enough to do these they became a replacement, however, moving from an analogue computer to a digital program could be quite difficult, since the two operate in vastly different ways.
Large systems often used digital computers since the ~70s for e.g. recording and analyzing outputs: a company in my home town developed test rigs for performance and crash testing of cars (and also did the testing to some extent); they still had a massive hybrid computer in the 80s (multiple analogue racks plus I think two DG Nova systems).
Adrian Thompson at Sussex University used a genetic algorithm to auto-design FPGA circuits in the early 90s. Since no one told the GA that FPGAs were supposed to be logic circuits, it happily used the FPGA as an analog machine.
Even an Intel i7 chip is an analog machine that approximately implements the i7 computer design. They throw away the ones that don't approximate it up to tolerance.
The difference isn't that FPGAs don't operate on analog voltages deep down (who said they don't?). The difference is in the set of tools and tolerances they give you, and in that sense FPGAs are only an analog coprocessor in the sense that a car can, technically, be used as a sailboat.
Why do you say "so-called"?
can be built by using FPGAs.
Also, FPAA's (Field Programmable Analog Array)[1]
[1]: https://en.wikipedia.org/wiki/Field-programmable_analog_arra...
It seemed like an exciting idea, but it never took off perhaps because the kind of accuracy that makes it worthwhile was not achievable?