Hacker News Reader

Comment by 3pt14159 | Hacker News Reader

I know a military contractor working on some pretty cool stuff, especially back in the day. On one of their projects they had to solve some really gnarly partial differential equation and weren't really getting anywhere. Since what they actually wanted were values (not the algebraic solution) what they ended up doing is building the equivalent circuit and measured what the voltage and current was in the circuit to get the values that they needed for some completely different domain.

I always thought that that way of thinking was pretty neat. Can the same abstraction have a different physical manifestation that's cheap to create? If you kinda squint a bit, it's what we do with computers.

View on HN

I've been doing some restoration on an analog computer. While it seems bizarre now to solve a problem with analog circuits instead of a digital computer, analog computers were a reasonable approach until about the 1970s.

The basic idea of an analog computer is you use op amps to sum voltages. Integration is easy, by adding a capacitor. By hooking up op amps, capacitors, and resistors (usually with a plugboard), you can quickly and easily solve differential equations. (Surprisingly, multiplication by a non-constant value is hard with an analog computer.)

The nice thing about analog computers is they gave the answer almost instantaneously. You could turn the dials and experiment with different parameters interactively. A digital computer in 1970, on the other hand, might take minutes to solve the same equation. This also made analog computers useful for real-time applications such as flight simulators.

A big problem with analog computers is that the accuracy is limited by your components. If you want 0.1% accuracy, you need expensive 0.1% resistors and capacitors. With a digital computer, you can get as many bits of accuracy as you want using cheap components. Basically, Moore's law killed off analog computers.

Just fire up any older electronics textbook, and they'll probably have examples of integrator and summing circuits solving differential equations. At least that's what we had in our control (engineering) books.

In my dad's Calculus II equivalent class in Morocco, one of the question in their exam was to built the circuit to solve a given differential equation. Pretty neat!

That being said, that exam was substantially shorter and easier than the Cal II exam I had to take!

> multiplication by a non-constant value is hard with an analog computer

Researching this topic also qualifies you to write technobabble for sci-fi movies. Like "operational trans impedance amplifier."

If you haven't seen it yet, the retroencabultaor video [1] is a masterclass on how to do it.

[1] https://youtu.be/RXJKdh1KZ0w

The original too!

https://youtu.be/Ac7G7xOG2Ag

I actually use a transimpedance amplifier for my work, and it’s even operational most of the time! But one does wonder if techno-babble really does sound like real techno jargon to none experts.

There is practically no scenario where a digital computer couldn't do better if what you wanted was a numerical result and not a symbolic one.

It's simply criminal how underutilised the technique of Automatic Differentiation is! It can solve incredibly complex differentials to machine precision, good numerical stability, and minimal tuning or hand-holding.

Lots of these "difficult" problems aren't, it's just that people aren't aware of the solution because it's not commonly taught in a University setting. Automatic Differentiation is one of those techniques that's "boring" and "numerical", so there are no courses that teach it.

Doesn't literally every optimisation and ML class teaches it nowadays?

Actually, no! You would think so, but again, it's a boring numerical technique so it is all too often sidelined in favour of the more "flashy" symbolic techniques.

An awful lot of tertiary education has been caught up in the publish-or-perish mentality. Unfortunately for AD, it is not "impressive". Differentiating hideously difficult expressions symbolically is impressive. Publishing a paper packed full of enormous, scary looking expressions is impressive. Merely saying that "we used AD and it worked just fine" is unimpressive.

For decades now we've been incentivizing intellectual wankery over practical utility.

Here's an example from my field: "The derivatives of the finite-build coils are computed -- with almost no additional effort from the authors -- using automatic differentiation." Well, what kind of scientist are you, if you are too lazy to take derivatives? /sarcasm

http://meetings.aps.org/Meeting/DPP20/Session/BP14.32

The parent comment is talking about solving differential equations. That's a very different thing from computing differentials. Automatic differentiation doesn't help with differential equations at all.

Yeah, this was a while back. The guy used to work for the South African military during apartheid and later worked on systems in Canada and Asia. I'm sure you're right that digital computers could do the job today, but back then the software and hardware wasn't close to being up to the task.

Isn't that the basis of analog computing? https://en.wikipedia.org/wiki/Analog_computer

Engineers used to use the shape of a soap bubble to calculate the torsional stiffness or perhaps stress distribution of a non-circular shaft.

http://www.newsteelconstruction.com/wp/wp-content/uploads/Te...

Soap film is also used to solve the steiner problem: https://www.youtube.com/watch?v=dAyDi1aa40E

Isn't that just analog computing? Or is it something more fancy?

This is analog computing, and it is really cool. Yet, I guess if it corresponds to a circuit it should be more an ODE than a PDE (unless your "circuit" involves large pieces of surface of non-uniform resistivity).

It was a true PDE, not an ODE. The circuit was non-trivial, but though I knew PDEs at the time he told the story, I'd never used them on electronics (only heat / pressure flow, and other physical engineering stuff) so I couldn't really understand what he was saying.

You can definitely do this! Check out Karplus's "Analog Simulation" for examples with conductive fluids and resistance networks: https://archive.org/details/analogsimulation0000karp

Yes. And a numerical calculus software would probably get there faster and with less work.

and higher precision.

Numerical solution of PDE is not really a solved problem. If your PDE involves shockwaves or turbulence (which happen for really simple PDEs) then the methods to compute solutions get incredibly fancy. There's no "black box" solution for numerical PDE, as there is for ordinary equations.

If you can solve with an analog computer, without problems with metastability, oscillations or excessive noise, then you can solve it with numeric calculus too. Those computers have much more strict limitations than digital math.

I should call the guy up sometime and get the details about what they were working. It wasn't a run of the mill analog computer, but who knows how novel it really was.

Your description covers pretty much every quantum computer ever conceived. How certain are you?

Not sure you can call any of the actual quantum computers we've made "analog", but if you're talking about the quantum annealment machines, they are generally more limited and take longer than an equivalent supercomputing cluster.

Hum... Quantum computers tend to be digital, not analog. The word "q-bit" is there for a reason, just because the bits are entangled it doesn't make them analog.

Besides, the set of problems you can solve is different from the set of problems you can solve quickly. When discussing analog and digital computers, the speed difference isn't all that relevant, but you seem to be confusing them here. Quantum computers do not solve any problem you can't solve on a classical computer, they just solve some faster.

Well yeah, that's exactly my point. Quantum computers are digital, which is not what the above comment claimed, except for quantum annealers which are closer to analog computers.

What I'm saying is that even quantum annealers aren't really faster at any practical problem than a comparable supercalculator as of yet.

Oh, I was intending to reply to the parent of your comment.

Can you provide a simple example of this?

The classic example are Navier-Stokes equations, a simplified model of fluid mechanics. Mathematicians do not yet know how to prove that they have a solution, but they can implement discrete versions of these equations that behave somewhat realistically. Yet, it is extremely tricky to get it right; for example, to observe turbulence at the same speed that appears on a real fluid.

A much simpler example where naive numerical methods do not work, is any first-order equation with converging characteristics. For example the inviscid burgers equation u_t + u u_x = 0. Even if you start your evolution with a smooth profile, it will form discontinuities. When you try to discretize it, computing derivatives by finite differences does not work at all, and the simplest upwind schemes have a lot of diffusion (which does not correspond to the model).

> Mathematicians do not yet know how to prove that they have a solution

Probably just a typo, but it's not hard to find particular solutions. The question is more whether solutions exist for all reasonable initial conditions. Here's a famous particular solution:

https://en.wikipedia.org/wiki/Taylor%E2%80%93Green_vortex

I've used a variant of this one to test CFD software.

The Millenium prize problem has a succinct statement of what's under question:

> In three space dimensions and time, given an initial velocity field, there exists a vector velocity and a scalar pressure field, which are both smooth and globally defined, that solve the Navier–Stokes equations.

https://en.wikipedia.org/wiki/Navier%E2%80%93Stokes_existenc...

Keep in mind that this is for incompressible Navier-Stokes with a constant viscosity. For inviscid flows you could get discontinuities as you've indicated with the inviscid Burgers equations, and I know that existence and uniqueness problem for the incompressible Navier-Stokes equations has been proved for certain uncommon viscosity laws (which should be reasonably realistic as a constant viscosity is not quite right).

(And of course there are other complications not considered in this problem like boundary conditions.)

> Probably just a typo, but it's not hard to find particular solutions.

Indeed, a fluid eternally at rest is surely a solution :)

It depends on what "back in the day" means. Modern computers are absurdly capable compared to what might have been available to these engineers "back in the day".

It is. I hope it comes back in the photonic form.

I knew a guy who built specific surfaces with plaster and threw marbles at them to find some special solutions to nonlinear mechanics problems. He thought of himself as an experimental mathematician.

Also see Optical Fourier co-processors for a similar example.

Banks do that all the time with monte carlo simulation. Can't get a closed form solution? Simulate every possible outcome and aggregate them.

This is neither how a Monte-Carlo process works (it's sampling the possible outcomes randomly, not simulating all outcomes), nor what is described here (building a physical system where an observable value is described by the differential equation one is trying to solve).

In practice the number of simulated scenarii is quite large which is what I meant by "every". Not rigorous though, I concur.

Monte Carlo methods are used to solve partial differential equations, I found both approaches similar in the sense that they look at how the system actually/could behave, instead of looking at the equations solely.

This is a pedantic response. There isn't an important difference between building a physical system and building a system in code whose generating process is that which you're trying to model. And everyone knew he didn't mean every outcome literally.

You’re talking more about goal-seeking functions right? That’s the one I always see in a finance context.

Worked in the securities division of a leading investment bank for 8 years including on the equities exotics and hybrids trading desk. And I can confirm that we used to use MC whenever we had something we couldn't price using a closed form method. That said, it's a lot slower and more compute intensive so when you could use a grid pricer or pde solver or whatever people would use that instead.

You're not really simulating every outcome, just a "large enough" representative sample.

As an aside, my dad was a chemical engineer and Newton-Raphson was his equivalent everything-is-a-nail numerical method. We used to use brent root and monte carlo simulation in the same way.

Numerical methods are awesome in general.

There is actually even a connection between PDEs and SDEs (stochastic differential equations), namely the Feynman-Kac formula which is used a lot in financial mathematics, but it generally allows you to switch between the expectation operator and solving a PDE.

Is this something an FPGA or something similar could do? Would be interesting to know of any home-brew attempts at this out there.

I wonder how difficult it would be to re-purpose a modular analog audio synthesizer for this. Since they're direct descendants of analog computers in the first place.

There are ASICs out there that simulate analog synths, such Universal Audio. Brings to mind some interesting pure software approaches if that were the case.

Do you mean FPAA? Totally a thing.

https://en.m.wikipedia.org/wiki/Field-programmable_analog_ar...

FPAAs don't really exist as products you can buy, as far as I know (unlike FPGAs, which are a very mature technology to the point that I suspect there's nontrivial (say, >5%) odds the average HN reader owns something with a FPGA in it somewhere.))

I didn't know about this. It's curious that the article mentions that recently (2016) improved FPAAs were demonstrated. Perhaps an opportunity where there wasn't one before.

Reply on news.ycombinator.com