The power efficiency argument from biology translates well to silicon when you want to build large structures. Here you cannot have 32-bit floats being passed around to thousands of target synapses, so you must make do with less.
The power efficiency argument from biology translates well to silicon when you want to build large structures. Here you cannot have 32-bit floats being passed around to thousands of target synapses, so you must make do with less.
And also, your assertion that you "can't have 32-bit floats" and therefore we should go analog (which is what this project wants) is a rather poor understanding of the challenges facing computer architecture in the modern day. The problem is data movement, not the computation. For example, Lawrence Livermore National Laboratories estimates that the cost of moving a 64-bit word 1mm ON CHIP will be approximately equal to the cost of a 64-bit FLOP on 10nm.
This is why I said it is a problem passing them around.
>the fact of the matter is that they're still far behind RNNs/LSTMs.
Technologies that are far more developed purely based on the number of man-years poured into them.
Man-years were poured into LSTMs because they were promising and effective, not the other way around.
I'm not sure what you tried to say about 32 bit floats: are you saying it's easier or more efficient to represent a 32 bit float using time difference between spikes than using a voltage level? it might be true, but I don't see how.
Why? We are not trying to transmit spikes, we are trying to transmit information. It's not at all clear to me that encoding a signal as a series of spikes is somehow more efficient than placing a certain voltage level on a wire. More specifically, given a particular noise floor, and a desired data rate, you either use binary encoding (spikes), but have to use higher frequency, or you use lower frequency but pack more bits into each voltage level. This would probably be affected a lot by the required frequency range.
One example comes to mind: T1 transmission lines (NRZ encoding), and DSL phone lines (QAM). I don't know which one is more power efficient, and not sure if that was even a concern there, but it would be interesting to know how they compare in terms of power efficiency, and how that changes when we increase data rates or noise levels.
Analog voltages will be subject to voltage drops as they traverse the chip
The spikes will be subject to exactly the same voltage drops, what's your point?
and require a massive number of wires because you cannot multiplex the voltages without extremely expensive (area wise) analog multiplexers
I'm not sure what you mean here. What do you want to multiplex, and why spikes are better for that?
the brain may use square waves because it is part of a strangely evolved biology which has required things like fear and adrenaline, and uses long, salty, high impedance connections.
the brain's architecture may not be the best for all jobs