Hacker News Reader

Comment by orbifold | Hacker News Reader

One of the authors here, happy to answer any questions you might have.

Could you comment on this general SNN critique? https://news.ycombinator.com/item?id=27366059

As the author of that "general SNN critique" I'd like to add that this was not a general critique, but a specific reply to the question why I think low-energy deep learning is a misguided promise for SNNs.

Personally I think SNNs are a very exciting research field, both from a neuroscience as from a computer science angle. The work we are discussing here is deeply impressing for its rigour, and it addresses an important problem in spiking network research.

Whether spiking networks will provide lower-energy deep learning is a totally different question.

I actually think I largely agree with the points that you made in the comment thread above. The name of the game can't be to just map a deep neural network to a SNN, I find it far more interesting to identify more natural mechanisms by which SNNs are able to perform information processing on sparse asynchronous event-based data. Also thank you for the nice words :).

Maybe "critique" was the wrong choice of word. Your point is now much more clear to me, thank you for your insights.

Absolutely right

I'm kind of an amateur, but incredibly curious, willing to learn and do hard work.

I have many ideas and questions regarding your paper:

- How do you adjust weights between different spikes?

- Do you use or implement a kind of wavelet for wave-propagation, in example for spike interferences?

- What neuromorphic hardware can I buy to run your code/ the SNN?

- Weights are adjusted at the end of the backward in time (adjoint) integration, according to the weight gradients accumulated at pre-spike times.

- We only consider one kind of model system in this paper but this method would work for any kind of hybrid dynamical system, so also other physical substrates (a lot of exciting work to do there).

- We used to sell a neuromorphic hardware system Spikey for ~3000 Euro (basically at cost), we've recently completed a similar project, we also provide access to remote users via the ebrains collaboratory (https://ebrains.eu/service/collaboratory/). There are a number of commercial offers in the works (SynSense, Inatera). You can also buy SpiNNaker boards or access them via ebrains. Loihi and TrueNorth either don't sell or are pretty expensive, but they have "research agreements" in place.

> What neuromorphic hardware can I buy to run your code/ the SNN?

Current neuromorphic hardware is not easily accesible, but you can simulate spiking neural networks. Check out, e.g. https://brian2.readthedocs.io/en/stable/ or Nengo.ai

In table 3 you compare your result with other publications. Reference 44 has a larger accuracy than you with fewer hidden neurons. What is the difference between your method and theirs?

Also, will you release your method as code?

That number is what they reported in their publication, but it turns out they actually used both recurrent neurons (compared to feedforward as they state) and 512 instead of 100 neurons (see here https://zenkelab.org/publications/errata_zenke_vogels_2021/). We will adjust those numbers in the final publication.

My aim is to release the method as part of Norse https://github.com/norse/norse. There is some subtlety involved in implementing it for a given integration scheme, though. The event based simulator underlying the paper will also be released in due time.

Other author here, as was pointed out this was a typo in the cited publication. We will definitely be releasing the source code in due time.

Reply on news.ycombinator.com