They also have a lot of disadvantages, the main one being that no one has any idea of how to use them properly yet.
https://arxiv.org/pdf/1611.02344.pdf
With an explicit (potentially arbitrarily large) memory component, they can also capture long-term dependencies that don't have to be maintained in, say, the convolution parameters between presentations of data. This is one thing we want to pursue with the Faiss library at FAIR.
Regardless of how something is implemented, it's the what that is important. What's the actual algorithm or algorithms that the brain uses? It's likely easier to explore this question in floating-point math than putting together a lot of spiking neuron like things and trying to cargo cult your way to the answer.
You're thinking about this as if it's purely an algorithm problem and that computer architecture should always be designed to do the algorithms bidding.
Neuromorphic is flipping the problem around and creates efficient architecture where algorithms do the architectures bidding.
The former is much better in the cloud with massive armies of general purpose computers while the latter is much better on the edge for anything implantable (very little heat can be generated or else you cook) or needs super low power (less than 1W).
Having worked on both sides of the spectrum I think there's room for both and the relative research funding seems reasonable; much more invested in straightforward neural network research with GPUs etc versus the neuromorphic field where it's mostly a handful of Caltech folks left at places like Stanford, UCSD, GT, John's Hopkins, or UF.
SNNs can work in real time.
We can build analog level based networks and I don't see any reason why they would have to be slower than spike based ones. And as far as the noise, it's also unclear to me why a spiking network would be more noise resistant or stable. Do tell.
The power efficiency argument from biology translates well to silicon when you want to build large structures. Here you cannot have 32-bit floats being passed around to thousands of target synapses, so you must make do with less.
And also, your assertion that you "can't have 32-bit floats" and therefore we should go analog (which is what this project wants) is a rather poor understanding of the challenges facing computer architecture in the modern day. The problem is data movement, not the computation. For example, Lawrence Livermore National Laboratories estimates that the cost of moving a 64-bit word 1mm ON CHIP will be approximately equal to the cost of a 64-bit FLOP on 10nm.
This is why I said it is a problem passing them around.
>the fact of the matter is that they're still far behind RNNs/LSTMs.
Technologies that are far more developed purely based on the number of man-years poured into them.
Man-years were poured into LSTMs because they were promising and effective, not the other way around.
I'm not sure what you tried to say about 32 bit floats: are you saying it's easier or more efficient to represent a 32 bit float using time difference between spikes than using a voltage level? it might be true, but I don't see how.
Why? We are not trying to transmit spikes, we are trying to transmit information. It's not at all clear to me that encoding a signal as a series of spikes is somehow more efficient than placing a certain voltage level on a wire. More specifically, given a particular noise floor, and a desired data rate, you either use binary encoding (spikes), but have to use higher frequency, or you use lower frequency but pack more bits into each voltage level. This would probably be affected a lot by the required frequency range.
One example comes to mind: T1 transmission lines (NRZ encoding), and DSL phone lines (QAM). I don't know which one is more power efficient, and not sure if that was even a concern there, but it would be interesting to know how they compare in terms of power efficiency, and how that changes when we increase data rates or noise levels.
Analog voltages will be subject to voltage drops as they traverse the chip
The spikes will be subject to exactly the same voltage drops, what's your point?
and require a massive number of wires because you cannot multiplex the voltages without extremely expensive (area wise) analog multiplexers
I'm not sure what you mean here. What do you want to multiplex, and why spikes are better for that?
the brain may use square waves because it is part of a strangely evolved biology which has required things like fear and adrenaline, and uses long, salty, high impedance connections.
the brain's architecture may not be the best for all jobs