I have a question for you, lots of people have been claiming memristors are right around the corner for almost half a century at this point, what do you think about the viability of the memristor in the future?
Edit: Appears some HP associated labs had success with just such a chip! Surprised it hasn't been more success in that field.
https://www.technologyreview.com/s/537211/a-better-way-to-bu...
The alternative would be to use the annoying/tedious analog hardware to significantly (possibly) reduce the amount of time/energy costs to train a neural net. Then have special equipment to measure the learned weights and convert them to 16-bit weights that'd be easier to deploy.
Facebook/Google are showing that custom chips for deep learning make economic sense. That market feature could tip the scales for specialized neuormorphic training chips which deal with analog circuitry but offset those costs & complexities by driving efficiency and/or speed while still deploying via traditional digital chips.
Of course, this path relies on the assumption that an analog circuit could be much faster or energy efficient. It's not too far fetched though based on the current numbers and time for training vs biological system energy costs.
[edit: grammar]
However, comparing the energy needed to modify the neural net data in-situ via analog signals vs shuffling that same data in digital form back-and-forth repeatedly to simulate the analog process seems to provide a viable use case. A quick mental check seems to lean toward back-propagation being "cheaper" in the analog processes as the data doesn't need to be moved while the calculation is performed as part of the same signal propagation via the properties of the analog circuit.
In other words its cheaper to move the computation "units" to the data than to move the data to the computation engine for this particular case. Performing the training in digital form requires repeatedly shuffling all the weights for _each_ training iteration. That process is expensive. Luckily the inherent nature of the back-propagation algorithm adapts to the "sloppiness" of analog circuitry. Transferring the final weights to digital form could require final but light post-processing training to remove particulars of the underlying analog circuits.
But replicating and distributing the final trained model would be more efficient since it only requires a single shuffling of the weight data to apply the neural net and get an answer. Applying the trained model via standard digital means should be cheaper/easier for all the reasons you mentioned previously.
[edit: grammar & clarity]
From the same wikipedia article:
> There is no such thing as a standard memristor. Instead, each device implements a particular function, wherein the integral of voltage determines the integral of current, and vice versa. A linear time-invariant memristor, with a constant value for M, is simply a conventional resistor.[1] Manufactured devices are never purely memristors (ideal memristor), but also exhibit some capacitance and resistance.
Just reading the 'Background' and following 'Memristor definition and criticism' sections gives a super clear and lucid answer that even a goober like me can mostly degest.
In the engineering world, if it talks like a memristor, and walks like a memristor if you squint a bit, we are damn well going to use it like a memristor. Hence we are happily building and testing nonvolatile storage (MRAM, ReRAM, PCRAM) and other applications - I'm gonna start research on applications as ML accelerators in May :-)