jegp

It's still possible to train a network that's aware of the physics and then transfer that to physical devices. One approach to this from the neuromorphic community (that's been working on this for a long time) is called the Neuromorphic Intermediate Representation (NIR) and already lets you transfer models to several hardware platforms [1]. This is pretty cool because we can use the same model across systems, similar to a digital instruction set. Ofc, this doesn't fix the problem of sensitivity. But biology fixed that with plasticity, so we can probably learn to circumvent that.

[1]: https://github.com/neuromorphs/nir (disclaimer: I'm one of the authors)

Thank you for your seriously thoughtful insights. You're saying we should have reproducibility, communication, and exploration. I fully agree with that, but I would like to push back on your point that novelty is destructive. Can't redundancy be similarly destructive? Given finite resources, shouldn't the authors have some responsibility to check their findings against previous insights? Otherwise, in the worst case, Goodhart can forget about his law because we'll never agree on any metrics in the first place.

To clarify, this is orthogonal to the point about the paper. I'm still not convinced it's entirely different, but I agree that the value of the paper doesn't purely depend on that. It's a nice exposition irregardless.

Is this novel? Spatial codes have been known for long and neuron (populations) are known to code for time [1]. That reference also states that "Neurons that respond selectively to the temporal and spatio-temporal structure of sensory stimuli have also been identified in the auditory system of birds and mammals". Doesn't that pre-empt the conclusion that "Thus, the brain can represent time and space as overlapping but dissociable dimensions".

[1]: https://royalsocietypublishing.org/doi/10.1098/rstb.2012.046...

A friend of mine built this course on real-time coding for GPUs with Rust, and I'm seriously impressed. The scope is great, the content, and questions are spot-on, and it's crammed with humorous comments and memes. If you're into fast programs on modern hardware, this is a great start!

Spiking neural networks and neuromorphic hardware platforms that emulate neural dynamics are slowly gaining momentum and entering main-stream usage. Despite a well-established mathematical foundation for neural dynamics, the implementation details vary greatly across different platforms. Correspondingly, there are a plethora of software and hardware implementations with their own unique technology stacks. Consequently, neuromorphic systems typically diverge from the expected computational model, which challenges the reproducibility and reliability across platforms. Additionally, most neuromorphic hardware is limited by its access via a single software frameworks with a limited set of training procedures. Here, we establish a common reference-frame for computations in neuromorphic systems, dubbed the Neuromorphic Intermediate Representation (NIR). NIR defines a set of computational primitives as idealized continuous-time hybrid systems that can be composed into graphs and mapped to and from various neuromorphic technology stacks. By abstracting away assumptions around discretization and hardware constraints, NIR faithfully captures the fundamental computation, while simultaneously exposing the exact differences between the evaluated implementation and the idealized mathematical formalism. We reproduce three NIR graphs across 7 neuromorphic simulators and 4 hardware platforms, demonstrating support for an unprecedented number of neuromorphic systems. With NIR, we decouple the evolution of neuromorphic hardware and software, ultimately increasing the interoperability between platforms and improving accessibility to neuromorphic technologies. We believe that NIR is an important step towards the continued study of brain-inspired hardware and bottom-up approaches aimed at an improved understanding of the computational underpinnings of nervous systems. NIR is available at https://github.com/neuromorphs/nir

If the goal is to share the visual contents of the pad, why not just stream the graphics over SSH?

I recommend checking out reStream: https://github.com/rien/reStream/ Using nixos, sharing your screen is as simple as nix run nixpkgs#restream

If we think about spikes as discretized transfer functions, I would say it's totally viable to "hack" them to represent numerical approximations. In fact, a recent paper demonstrates that exact mappings between ANNs and SNNs exist: https://arxiv.org/abs/2212.12522 There are some pitfalls here, though, and I'm biased against these kinds of methods because they don't use the temporal traces of the neuron integration.

Regarding RWKV, someone actually trained a "SpikeGPT": https://arxiv.org/abs/2302.13939 That's a neat insight, which will be great for porting these models onto energy-efficient devices. But the learning problem is still the most interesting open question to me. If we crack that, we can scale down GPT-like models by several orders of magnitude since we can "re-learn" subproblems instead of "hardcode" a silly number of permutations, like the present models do. Neuromorphic hardware (brains included) lend themselves incredibly well to learning. We just don't know how to exploit that yet.

Regarding the optical layers, are you referring to optical chips like this one https://www.nature.com/articles/s41467-020-20719-7 ? That would be an example of using optics to implement your stateful transfer functions (https://en.wikipedia.org/wiki/Optical_neural_network), but there are several of other incredibly promising technologies such as memristors (https://en.wikipedia.org/wiki/Memristor), quantum materials (https://arxiv.org/abs/2204.01832) and even biologically based chips (https://en.wikipedia.org/wiki/Wetware_computer). My take on this is that these technologies exploit different principles of physics to "compute" in some way. But I like to think that our computational theories and principles are independent of the implementation substrates.

There's still a long way to go, but practically speaking, I'm convinced this kind of hardware will have profound consequences for the way that we compute today. We're talking at least 3 orders of magnitude in compute. Imagine ChatGPT running 1000 times as fast. It's ridiculous.

I'm a PhD student working with neuromorphic computing. I like to think about SNNs as RNNs with discretized outputs. The neurons themselves may have some complicated nonlinear dynamic (currents integrating into the membrane voltage somehow etc.) but they are essentially just stateful transfer functions. The notion of spikes is a crippling simplification, but it's power efficient and you can argue for numerical stability in the limit. So I tend to consider spikes as an annoying engineering constraint in some neuromorphic systems. Brains function perfectly well without them, although in smaller scales (C. elegans).

The true genius of neuromorphics in my view, is that you can build analog components that performs neutron integration for free. Imagine a small circuit that "acts" like the stateful transfer function, with physical counterparts to the state variables (membrane voltage, synaptic current, etc.). In such a circuit you don't need transistors to inefficiently approximate your function. Physics is doing the computation for you! This gives you a ludicrous advantage over current neural net accelerators. Specifically 3-5 orders of magnitude in energy and time, as demonstrated in the BranScaleS system https://www.humanbrainproject.eu/en/science-development/focu...

Unfortunately, that doesn't solve the problem of learning. Just because you can build efficient neuromorphic systems doesn't mean that we know how to train them. Briefly put, the problem is that a physical system has physical constraints. You can't just read the global state in NWN and use gradient descent as we would in deep learning. Rather, we have to somehow use local signals to approximate local behaviour that's helpful on a global scale. That's why they use Hebbian learning in the paper (what fires together, wires together), but it's tricky to get right and I haven't personally seen examples that scale to systems/problems of "interesting" sizes. This is basically the frontier of the field: we need local, but generalizable, learning rules that are stable across time and compose freely into higher-order systems.

Regarding educational material, I'm afraid I haven't seen great entries for learning about SNNs in full generality. I co-author a simulator (https://github.com/norse/norse/) based on PyTorch with a few notebook tutorials (https://github.com/norse/notebooks) that may be helpful.

I'm actually working on some open resources/course material for neuromorphic computing. So if you have any wishes/ideas, please do reach out. Like, what would a newcomer be looking for specifically?

Wonderful to see this getting attention! I met with Minh since we were both using Futhark for our thesis. It's blazing fast and quite fun to work with.

If anyone's interested, my thesis contains some more examples on deep learning with Futhark: https://github.com/Jegp/thesis/

This. And it's completely reproducible. IMO this is the best bet for stability + long-term survivability.

Exactly. And it gets even better if you pair it with a personal voice assistant, like mycroft.ai

"Hey Mycroft, play songs about better software integrations"

What is lambda-calculus and why is it important for the future of computing and natural science?

From personal experience, using NixOS saves me a lot of time compared to Debian based distros. The initial setup is painful, but after that you are in complete control. Don't need to update? Don't! I had desktop sessions running flawlessly for months without any interruptions. Never seen that before on other distros, really.

A workshop and playground for biologically realistic simulations of neurons for control problems. Live and in-the-browser, thanks to the great Mybinder [1] project.

The workshop serves to illustrate the potential in neuromorphic hardware and event-based sensing for robotic controls.

Based on the spiking neural network simulator, Norse [2].

[1]: https://mybinder.org/ [2]: https://github.com/norse/norse

Co-author here. Happy to hear your comments and answer any questions you might have!

\tau is a common symbol for time constants which, briefly put, determines how fast something decays over time. Syn for synaptic current and mem for membrane voltage. We've written some documentation around our neuron equations in Python that explains this: https://norse.github.io/norse/auto_api/norse.torch.functiona...

See also our tutorial on neuron parameter optimization to understand how it's useful for machine learning: https://github.com/norse/notebooks#level-intermediate

There's also a great book on the topic by Gerstner available online: https://neuronaldynamics.epfl.ch/

Disclaimer: I'm a co-author of the library Norse

Regarding the target audience, it's actually not entirely clear to me. This lies in the intersection between computational neuroscience and deep learning, which isn't a huge set of people. So, I think you're question is highly relevant and we (as researchers) have a lot of work in front of us to explain why this is interesting and important.

Let me just mention Gopass (https://gopass.pw) which is a brilliant terminal-based `pass`-compatible password manager (https://www.passwordstore.org/). Gopass is open-source, free, and based on open standards. Meaning, you can use your keystore practically anywhere. For instance on Android with the Password Store app (https://play.google.com/store/apps/details?id=dev.msfjarvis....)

Great research! I guess this solves backpropagating on spiking dynamics. Ground-breaking and with a lot of potential for generalisation.

One question; do you know whether this is implemented somewhere we can try it out? And do you see this happening in a specific neuromorphic platform anytime soon?