Hacker News Reader

Comment by buboard | Hacker News Reader

brain floating point ... cool name

i guess brain's synaptic precision could go way lower, as low as 26 distinct synapse weights: https://elifesciences.org/articles/10778

> A particularly interesting research direction puts these three trends together, with a system running on large-scale ML accelerator hardware, with a goal of being able to train a model that can perform thousands or millions of tasks in a single model. Such a model might be made up of many different components of different structures,

yup, he is building a brain

View on HN

> many different components of different structures

> yup, he is building a brain

The limbic system of the brain is made of many different structures. It handles much of the ancient fixed function instinctive operation of the brain -- heavily involved in sleep, reflexes, appetite, and motivation, for example.

However, our general intelligence and learning capability is mostly due to our neocortex, which has a highly regular structure. The neocortex also subsumes the roles of much of the limbic system as development progresses, overriding the specific structures.

This suggests to me that intelligence / learning doesn't benefit from specific structures, but from general structures, capable of encoding behaviour as data (by long-term potentiation of synaptic weights in the brain's case).

> i guess brain's synaptic precision could go way lower, as low as 26 distinct synapse weights: https://elifesciences.org/articles/10778

Thanks for the link. Artificial neural networks all the way down to binary weights [1] although this approach doesn't seem like the most efficient one. It's interesting how we're still seeing a ton of variability in ML architectures: it suggests we haven't stumbled on the right area yet. It reminds me how early aviation had a huge diversity of aircraft plans, but now, after a lot of optimization, we've settled on that one standard airliner shape everyone uses everywhere.

[1] https://arxiv.org/abs/1602.02830

A formal theory of deep learning is proving to be much more elusive that avionics. Interesting times though

> yup, he is building a brain

Plus applying ML to improving the underlying hardware and software. On the other hand it's not recursive yet and moore's law ending throws a wrench into exponential self-improvement, but it's still a little concerning.

Reply on news.ycombinator.com