Memristor Breakthrough: First Single Device to Act Like a Neuron | Hacker News Reader

Memristor Breakthrough: First Single Device to Act Like a Neuron

spectrum.ieee.org

260 pointsheadalgorithm108 comments

The news article mentions that the student on the paper has passed away. He died of suicide in his lab just a couple weeks after the paper was submitted to Nature [0]. Grad school is hard for a lot of people. Find someone to talk to if you're struggling.

[0] https://padailypost.com/2019/02/16/engineering-student-found...

I've had an idea for a while, coming from a developing continent, that it would be good to have an alternative for students where they can start working when they are doing an advanced degree, but also simply a place for high school students to acclimate to some of the aspects of the work place.

I know it sounds tangential, but this does resonate with my own roundabout way of going from studies -> work and the gross lack of fundamental science companies in Africa as a whole. I did not have depression, but rather my own cocktail of problems as a student, and a lot of it (at least for me) stemmed from clear dissonance between your current situation and an inability to anticipate a future self that is acceptable to your current self.

With depression, the situation is similar and more importantly, you become more and more subjective and people outside your situation are needed, as probably is medication, to bring you back to personal assonance with what you want in life.

The article doesn't say what the reasons were, so Wang may have had different reasons.

With depression, the "why" is never easy to suss out, even when something is stated, is that really the reason?

Academia can be very toxic and PhD students are the most vulnerable as their Prof holds all the cards. This needs real checks and balances.

Getting a paper into nature and being a top-ranked school almost certainly says it was a high-pressure environment.

But there is certainly a process for handling advisor-advisee issues at a big school like Stanford:

https://deanofstudents.stanford.edu/who-we-are

I have a feeling that depression relies on many pillars of your mind breaking.

What is needed is removing the source of the problem in the first place.

You are in a foreign country, with no friends, no family, working over time, in some places extremely little money.

If you are burnout the first thing that you need to do is to stop. Make friends and lovers, travel and work outside for a year or so.

To use medication or drugs is not going to solve your problems,if you continue living such an unnatural lifestyle.

And I said that having lived that lifestyle myself, but I took my breaks from the toxic system.

Problem is, how?

When I went to university I ended in deep debt.

Took me years, to pay off that debt, it felt like literal slavery, since I would just take whatever job that could pay the debt that was offered (I had to decline many jobs because their pay was lower than my debt, meaning taking them I would just stay in debt forever), and then I would stay until I was fired or had to quit.

Even then, those were not "real" jobs according to our government, in my country my "real" job registry, is empty, I never "worked", because I never got hired full-time legally, I couldn't afford to, whatever people offered that the pay was good enough, I would accept, even if the employer was obviously hiring me as "freelancer" just to not pay taxes.

The times I had burned out during this, I had to keep going, travelling for a year? I dunno if I will ever afford that. Making friends? How? When? I am not in a foreign country yet I have two friends at most, and one of them moved to another country and I see him once every 5 years.

So it is not like people have a choice after they went to college, I still feel that going to college was the biggest mistake I ever did on my life, I should never have went, and never took that debt, it is paid now, but I am working with Marketing, instead of working with programming, and have little room for error, no debt, but no surplus either, if something goes wrong with the business I am screwed.

That's too bad.. Sorry to hear about your experience with university and debt. At least, now that it's paid off, you can start building up something, instead of working for nothing.

> to pay off that debt, it felt like literal slavery

This is what I feel is almost criminal about systematically committing university students to years-long debt. Education shouldn't have to be traded for years of someone's life - especially when most of the labor market expects them to have at least a degree.

I think the root of the problem is deeper than education (as a business) though - it's how society is organized. To have a minimum standard of living, one must eat every day and have a place to stay - which is already a kind of debt, a constant need for money. Poor people are essentially enslaved to dead-end jobs, just to be able to survive.

Hopefully, in a sensible/utopian future, we will look back on this social arrangement as barbaric, inhumane and uncivilized. Until then, best of luck navigating, adapting, flourishing despite the setbacks.

> What is needed is removing the source of the problem in the first place...To use medication or drugs is not going to solve your problems,if you continue living such an unnatural lifestyle.

But this ignores that the source of the problem could be brain chemistry itself. Or at least that it is part of the problem. If it were situational only, everyone in the same situation would have the same outcome. But the physical structure and chemical cocktail of each individual brain plays a huge role in how those situations affect each person. The brain is an organ, and just like any other organ, there are situations that cannot be fixed with “lifestyle changes”. Medication should not just be hand waved away.

Correction: I missed the year on the linked article. He passed away in 2019, a year before the paper was submitted.

See: "Grad school is worse for public health than STDs" https://news.ycombinator.com/item?id=21749279

And my comment there. And also my comment here (which links to more): https://news.ycombinator.com/item?id=21629112

A good resource on suicide prevention: https://www.metanoia.org/suicide/ "When pain exceeds pain-coping resources, suicidal feelings are the result. Suicide is neither wrong nor right; it is not a defect of character; it is morally neutral. It is simply an imbalance of pain versus coping resources. You can survive suicidal feelings if you do either of two things: (1) find a way to reduce your pain, or (2) find a way to increase your coping resources. Both are possible."

A Basic Income could make a big difference to make it possible for people to live a graduate student lifestyle without the stress of navigating the social dysfunction of many (not all) graduate programs...

12 years ago, on HN: "The memristor—the functional equivalent of a synapse—could revolutionize circuit design"

https://news.ycombinator.com/item?id=397335

Didn't HP want to build a new type of computer on it?

It was called The Machine: computer with unified memory. They released it without memristors in the end: HPE unveils The Machine, a single-memory computer with 160 terabytes of memory: https://news.ycombinator.com/item?id=14350396

It's fascinating that none of the comments mention neural network applications.

Seems like these people came a bit too early to the party.

We've left biomimicry in the dust a fair time ago - neurons do not work like relus after all (nor do they work like memristors), but relus are great at solving problems. None of our current NN applications would even want something that behaves like a neuron. You'd essentially have to start from scratch. It's not even clear if behaves-like-a-neuron is better or not to be worth the effort. (And that's disregarding the entire fact that memristors do not behave like neurons either!)

Do we even understand how a neuron works well enough to say that either thing is correct. I thought a relu was just the activation function that decides what weight to triggers the next part of the network with?

We understand enough about how a biological neuron works to know that single signals do not provoke a diminished response after the hysterisis period, which is not modeled by either a memristor or a sigmoid. Sigmoids don't have hysterisis, and you have to backflow a memristor for it to recover which is not true of biological nuerons.

(The activation function is the neuron('s mathematical model) - we elide everything that happens internal to the neuron and model it as only its inputs and outputs.)

I don't think Relus are the proplem. Relus are good, but artificial neural networks can work with sigmoid activation fonctions instead. The neurones in an ANN with sigmoids are somewhat close to what is described in this article.

The main problem is that the neurons in this article are not trainable (you can't update the weights dynamically) and not differentiable (you can't use the usual gradient-based training procedures to update the weights). They could be used for inference only though.

Wait!

Some people have left biomimicry in the dust and are manipulating matrices to sets find optimal parameter sets representing concepts.

Other people (https://www.humanbrainproject.eu/en/) haven't! There is a big community of people building novel types of neural network using spikes and so on.

And you are right - it isn't clear who is right. On one side the "no nature" folks might say : we don't even understand neurons to the point where we can definitely categorize them in a mouse brain, let alone explain their behaviour, or actually properly measure their behaviour, and so on and so on - this before understanding any of it really.

On the other hand the "lets mimic things we don't understand" folks would say - we need to understand this because the human brain out performs all AI today on a 20 watt budget.

My view is the understand stuff agenda is right - but not now and probably not for 20 years (in a big way). We need small scale capability and direction finding work - and we will need to do that for a long time.

I thin it's more that we're operating in a position of extreme simplification, and don't understand most biological details well enough to apply them in our own systems. But biological motivation is quite common in neural networks. Name a popular technique (backprop, dropout, etc.) and I can probably find you a source arguing it's biological relationship. It's just at an engineering-style level of approximation that scientists may abhor.

The ReLU was originally introduced by computational neuroscientists as a more biologically accurate approximation compared to the other activation functions. E.g. https://www.nature.com/articles/35016072

I would expect similar history behind convolutional neural networks, given how the early stages of the vision system have been understood to operate.

Research in this area has always been cyclical. The Neural ODE paper that won a best paper award at Neurips two years ago just implements a technique that is common knowledge in the optimal control theory community since the 60s.

I wouldn’t dismiss an approach because it isn’t popular right now. Biological neurons force you to think about constraints you normally wouldn’t and crucially have solved temporal credit assignment, memory etc. in interesting ways that already have inspired some breakthroughs.

As an example the replay memory idea was directly inspired by the rough understanding neuroscientist developed of the role of the hypocampus in learning.

These guys seem to be working on it in that way: https://knowm.org/

For neural networks, it makes sense to skip pure digital design.

When I learned how to design an ALU to say, add, and wait for the propagation of carry bits that’s like O(n) where n is the number of bits in the number, it made me want to just use superposition for addition, which is physical and instantaneous. Of course, that has all sorts of other problems that make it worse (so much worse).

Once you learn how to slow down the earlier bits, you end up with all the bits arriving at the same time, and you can have up to n adds pipelined and timed to the clock when output matters.

But with NN I think now would be a fun time (and likely the last decade as well) to rethink the basics right down to the basics. Everything need not be a NAND gate. :)

This runs into the problem that the power efficiency of analogue circuitry is dramatically worse. A FET dissipates the least heat when it's fully on or fully off. Operating in the resistive regime will result in orders of magnitude more dissipation.

It's difficult to over-come that, particularly because it's not a comparison between 'analogue v 64-bit float', but 'analogue versus 8-bit int'. (it's tough for scale analogue circuits to operate even when 8-bit accuracy).

That just means an FET isn't the right device for analog circuitry. Maybe this memristor device will be more efficient.

It's a bit more fundamental than that. A FET goes between say, 0.01 Ohms and 1,000,000 Ohms for fully on and fully off. The times in between are where it passes the most power through it and that's because of how Ohms law and Power being current times voltage (ignoring differing phase between current and voltage in AC circuits). This all still remains true even for the memristors. You can do analog stuff with higher resistance and lower currents, but then you end up more susceptible to noise and other randomness induced by the environment: thermal noise, electric noise, magnetic noise, and RF noise (technically also electric noise, but more organized), etc. All that being said, for a NN this might actually be somewhat acceptable, but for general analog circuitry you can't get past this problem at all. That's a large reason why most analog circuitry these days does as much as possible in a digital domain before converting back to analog, even simple amplification. A class D amplifier does the whole job by chopping up a large voltage into just on and off and then filters it afterwards. Even though you have power loss in the filtering, the end result is much easier to have stable, and get the desired amplification. You still find class A and B amplifiers where the high frequency chopping noise isn't possible to filter out or would cause other problems, particularly when you look at RF circuitry since you'd end up transmitting that high frequency noise and causing other problems for yourself.

It's my understanding that, to some extent, nerves and muscles operate with hysteresis based on charge thresholds, and rely on pulse-density encoding for variability rather than proportional voltage or current modulation. Presumably, that seems to be why your muscles shake when under heavy strain.

It sorta makes sense, considering your body is generally electrically conductive everywhere. Signals propagate as waves of charge sustained by cells opening and closing little pores that selectively release ions. "Conductors" have pores, and "insulators" don't. That's also why nerve signals are relatively slow to propagate vs. straight up electricity. That's also why you can measure muscle activity via an ECG; you're measuring the transient voltages being generated by the waves of charge moving around. The voltages themselves don't have much meaning to your body. Of course, if you apply enough voltage to build up a charge inside a muscle, you can cause it to actuate.

I recently had to get a pacemaker, and apparently the particular lower threshold to trigger my heart muscle to contract is around 0.7V for 350uS, at whatever impedance the lead happens to be. Below that, and my heart muscle does nothing. Anything above that, any my heart muscle does a full beat. The device applies a 2V pulse so that there's plenty of margin, and can go up to 5V if needed, in case the lead's impedance increases. The lower the voltage the better, in terms of battery life. The cool thing is that the device can safely coexist with any natural electrical activity my faulty nerves may have, since the muscle simply responds to whichever pulse happens to arrive first.

Ideally each analog device that simulates a neuron is equivalent to tens of thousands of digital transistors that simulate a logical neuron (in space-time for GPUs or CPUs and just space for neuromorphic asics) . This makes the trade-off much more feasible. .

We don't need to go fully analogue though, going asynchronous might already be a step up. In neural networks some groups do research into so-called spiking neural networks, which you could think of as electronic circuits operating asynchronously on binary signals. They are very energy efficient, but nobody really knows how to use them yet.

Clockless CPUs have been a theoretical idea for over a decade before the "neutral network" became a common phrase. Felt like a decade ago it was still a pipe dream. Resolving the asynchronous signals becomes a chore. The world feels stateful and ordered to humans. In the same way functional programming can be difficult, getting this right is very hard.

There had been purely asynchronous CPUs before [0]. Then automated clock gate insertion made it possible to approach the gains of asynchronous, but still designing in a synchronous paradigm.

[0] https://en.m.wikipedia.org/wiki/AMULET_microprocessor

Greenarraychips are clockless: http://www.greenarraychips.com/home/products/index.html

Very interesting chips.

> the power efficiency of analogue circuitry is dramatically worse

The human brain would like to have a word with you.

At the slow "clock speed" the brain runs at, it produces a lot of heat. A quick search turns up 10-20 watts as a rough estimate. It's still petty competitive given what it's capable of.

Good thing we have liquid cooling

Why do you say slow "clock speed"? Can we even use clock speed w.r.t brain? Genuinely curious.

I assume the comment was referring to the duration of the action potential and subsequent refectory period[0], which are on the order of milliseconds.

[0] https://en.wikipedia.org/wiki/Refractory_period_%28physiolog...

That makes no sense because the brain is not digital (it's "pulse-based"). There is no signal synchronizing all neurons in the brain, which is probably where a lot of the power savings come from too.

It's a very rough analogy. The brain doesn't work like a computer chip, but it does work much slower than modern chips in neuron firings per second.

> When I learned how to design an ALU to say, add, and wait for the propagation of carry bits that’s like O(n) where n is the number of bits in the number

No, it's O(1). The bit size of how you represent a single datum does not grow with size of your model.

That doesn't really contradict what he said. O(n) where n is a positive constant is O(1).

Reminds me of all-optical diffractive deep learning. It's such a cool concept and really made me rethink computing.

https://science.sciencemag.org/content/361/6406/1004.full

I totally agree with this. I've long dreamed of something like a FPGA for NN's where the gradient is built into the hardware configuration.

Someday I'll sit down and actually try to design a prototype for such a thing.

It's also worth mentioning that the hardware is not the only thing that should change. Cognition seems hard to automate just using instructional directives for producing the states of such a machine. Encoding the gradient right into the hardware would be cool, but the optimization procedure should also move to being something intrinsic to the machine.

If you're building adders with O(n) carry times you're doing it wrong - normal carry lookahead logic is ~O(log N)

The gate count for addition is at least O(n) though.

You should note the O(log n) delay growth is for a logic (ideal) model. Considering the 2D geometry of real circuits, there are models with more detail, in particular VLSI models. [1]

I suspect due to routing and path lengths even the delay would turn out to be O(n) in VLSI, or at least a power of n (with a small constant).

[1] Seems like a good introduction: http://cs.brown.edu/people/jsavage/book/pdfs/ModelsOfComputa...

Oh yeah definitely > O(n) - but we often trade gates (and area) for speed either by pipelining and/or more parallelism (which is what carry-lookahead logic is)

A bottom-up rewrite of modern computer engineering for performance and capability would probably qualify as one of our largest engineering endeavors in human history. But I would totally be on board for that. There would inevitably be some really interesting discoveries from it, no doubt.

Here is an attempt https://dspace.mit.edu/bitstream/handle/1721.1/78468/8340892...

Reminds me of ternary computers. I can’t think of the exact circuit type now but there exists one in three states that can make every other.

Also: https://en.wikipedia.org/wiki/Ternary_computer

If you find water analogies helpful for circuit elements, here is one for the memristor:

"The closest analogy I can think of is a sand filter, an item of apparatus used in water-purification plants. As contaminated water flows through a bed of sand and gravel, sediment gradually clogs the pores of the filter and thereby increases resistance. Reversing the flow flushes out the sediment and reduces resistance." -- https://www.americanscientist.org/article/the-memristor

Isn’t that like a capacitor? As it charges the voltage increases and the current drops until it reaches zero. Then if you discharge it you can start over again.

A capacitor is like a rubber membrane that has elasticity. When the capacitor/rubber membrane is stretched to its limit, it either breaks (dielectric breakdown) or it releases that energy causing a flow in the opposite direction.

A sand filter has no limit (as it gets more clogged, it starts resembling a closed valve) and it can't release built up energy (flowing in the opposite direction just "opens the valve")

Capacitors bridge voltage and charge, yes. In exactly the same way, memristors bridge charge and flux.

Off topic. When something is on the tip of my tongue, I don't search for it because I have a strange belief that what my mind is trying to do to remember it is fire off neurons associated with the thought or idea, near it, to create the action potential. It is strange that I will eventually remember it within hours or the next day. When the action potential fires the neuron, dendrites are formed between neurons strengthening the memory.

I wonder if there is any truth to that notion? That to solidify a memory, there can't be outside help, that electric chemical signal has to trigger firing the neurons associated with the memory to strengthen their connections.

I feel like if this were true (searching doesn’t strengthen the original memory), it would lead to a dramatic difference in a cohort’s ability to remember trivial information that they learned a long time ago, but it also seems really hard to perform a trial for something like this.

When I started learning programming, my French became much easier to recall. This was after ~ a decade of very infrequent French language use. Much of it has subsequently come back to me.

Sample size of 1, yadda, yadda. Make of it what you will. I think it is interesting.

I seem to remember reading about how thinking about something on the tip of your tongue and failing makes it harder to remember in the future, because concentrating on trying to remember it strengthens pathways that don’t lead back to the memory. So when you try in the future, those failed paths are stronger and more likely to be used again, further reinforcing the wrong association. Letting it go and hoping it comes to you also might not work, but it doesn’t strengthen bad pathways, so is more likely to allow you to remember in the future.

"Acts like a neuron" is BS. Neurons are more like individual computers themselves, with all sorts of activity at the subcellular level.

Interesting, where can I read more?

https://www.quantamagazine.org/neural-dendrites-reveal-their...

Abstract

Current hardware approaches to biomimetic or neuromorphic artificial intelligence rely on elaborate transistor circuits to simulate biological functions. However, these can instead be more faithfully emulated by higher-order circuit elements that naturally express neuromorphic nonlinear dynamics1,2,3,4. Generating neuromorphic action potentials in a circuit element theoretically requires a minimum of third-order complexity (for example, three dynamical electrophysical processes)5, but there have been few examples of second-order neuromorphic elements, and no previous demonstration of any isolated third-order element6,7,8. Using both experiments and modelling, here we show how multiple electrophysical processes—including Mott transition dynamics—form a nanoscale third-order circuit element. We demonstrate simple transistorless networks of third-order elements that perform Boolean operations and find analogue solutions to a computationally hard graph-partitioning problem. This work paves a way towards very compact and densely functional neuromorphic computing primitives, and energy-efficient validation of neuroscientific models.

Fun fact: there are about a hundred trillion atoms in every neuron.

At first I was like: "There's no freakin' way"

But then, sure enough wolfram alpha: 100,000,000,000,000 carbon atoms is ~0.167 nano mols. Or ~2 nanograms of carbon. Dang.

For a "universal" perspective on relative sizes:

https://www.youtube.com/watch?v=KEHCCsFFIuY

What neural nets need in order to improve is lots of data, and especially simulated environments. We have had the advantage of billions of years of evolution, they start the same journey from scratch.

Human brain is not smart in and of itself - it learns everything from the environment, including fundamental concepts and reasoning. We are not just surrounded by nature, with all its glorious detail, but also by the human society and culture. Our environment is very complex and rich. That's something neural nets have to replicate some other way, and simulation is one.

It shows that just inventing a better artificial neuron does not meaningfully advance the problem of artificial intelligence. It's just one leg, it needs two legs to stand on.

Plato has entered the chat

This is interesting, but I'm kind of sceptical. We don't fully understand everything that's going on in a neuron well enough to say if we're replicating that behaviour or not.

We don't even really know how many synapses an average neuron in the human brain has.

I think it's worth considering the hypothesis that neurons work because they have been evolving towards memristor like behavior.

I'm not convinced that spiking behavior is all that's needed to really emulate what's going on in our neurons. Modern hardware designs are not really amenable to replicating the number or structure of synapses that we have, muchless the extremely complicated biochemistry that controls how and when they fire. It's good to see progress being made, but we really need to tone down the hyperbole in the popular scientific press.

No, they mean that the memristor acts like the synapse.

It was the last remaining basic two wire circuit element (the others being the resistor, capacitor, and inductor). We could model cells, up to the memristive portion, entirely with basic circuit components (albeit in very complicated arrangements). We were missing that part of our bits boxes. Now we aren't, well, sorta. It'll be a while before we get these in our hands.

Also, these things are a bit bigger than just modern hardware. It's as if you just added a new primary color to Bob Ross; everything changes. We're going to need to redesign computers from the electrical-engineering-ground up.

Well maybe, it'll be a few years (hopefully not decades)

To me, this is one the most important news items this year. I've been excited about memristors since I first heard about them, and this sounds like an excellent application.

These single device "neurons" also hints of things to come. Even larger neural nets with more efficient structures may, perhaps, cross a threshold in terms of what it can learn and express. I'm thinking Alpha Go Zero and beyond.

I believe this is only the start of this type of hardware.

I've heard about memristors for ... a decade? ... so I'm skeptical.

What makes "this time different"?

They were going to revolutionize the speed of data access, right? https://past.date-conference.com/proceedings-archive/2017/pd... Making them behave like neurons is a cool hack, but sort of tangential to why anyone cares about memristors.

I keep an open mind though, so if this time really is different, then that's valid. It just seems like more hype.

I was also very interested in memristors when I first discovered them over 10 years ago. I actually wanted to go to graduate school to work on them, but was surprisingly unlucky in finding willing advisors with funding. In fact I even personally knew one of the researchers at HP and asked about working there, and he basically told me "you should do something else with your life."

> I believe this is only the start of this type of hardware.

I agree, but I also suspect a pragmatic solution is still decades away, much like nuclear fusion. Of course I no longer follow this area closely and could be completely wrong.

This is really interesting work, but I feel like the way the article summarizes it is getting it wrong.

First Single Device To Act Like a Neuron

One thing that’s kept engineers from copying the brain’s power efficiency and quirky computational skill is the lack of an electronic device that can, all on its own, act like a neuron. It would take a special kind of device to do that, one whose behavior is more complex than any yet created.

This is just getting it wrong. Modern neural networks run as vectorized operations on a GPU. They are efficient because they do not use a single device to act like a single neuron, they can do massively parallelized work by optimizing for the hardware we have. If we had memristors, it doesn't seem like GPUs are the first thing they'd replace. More likely they would replace some sort of memory, since what makes them unique is storing information rather than performing analog operations faster.

The brain is still way more efficient and parallel then a GPU (e.g. 12W compared to 280W), so being able to duplicate it closer sounds like a pretty compelling advancement.

it also operates at (less than) the speed of sound.

The brain is slower than the speed of sound in what medium?

The one people always mean unless they specify otherwise: Air.

Well in this case it's the fleshy medium of your brain.

This has nothing to do with "modern neural networks". When they say "neuron" they mean an actual biological neuron. The main application for this device type is brain simulations.

Hasn't this been debunked: https://www.quantamagazine.org/neural-dendrites-reveal-their...

Probably even a bacteria, and even an organelle inside one, is doing computation that a memristor can't. Comparing a memristor with a neuron is laughable.

Comparing a bacteria with a neuron is laughable.

Yes, that was part of the point: memristor < bacteria < neuron (reading < as "much much less computationally powerful").

Modern NNs run like that because all we had was a hammer so we tried as hard as possible to make the NNs look like nails.

At 800 degrees C

Can we make AI computers for Venus surface use now?

Of course

Protogen wants to know your location.

It's actually still too cold there for them to work.

Venus seems sufficient: https://solarsystem.nasa.gov/resources/681/solar-system-temp...

It's above 800 degrees Fahrenheit, but not even 500 degrees Celsius.

Oh, my mistake. I clearly can't read.

To be fair, it's usually easier to heat something to above ambient temperature than to cool it to below ambient temp.

This is definitely the "in mice" of materials science.

Solid state batteries, hydrocarbon fuel cells etc. all work - at high temperatures.

Solid-state batteries have officially left the lab, "Mercedes-Benz’s newest electric city bus uses solid-state batteries": https://arstechnica.com/cars/2020/09/mercedes-benzs-newest-e...

Acts like an artificial neuron model, not a biological neuron.

I have followed this on & off. The thing that I got excited about was the amount of integration density and how little energy it takes to signal. I wonder how this fares on those aspects.

What is clear to me is that our current nand based hardware is not ideal to model artificial neurones.

But from my point of view, the difficult part is the many-to-many connectivity.

Fitting this article was written by a fella named Moore

Unfortunately, we do not know how neurons learn, compute and represent information in biological neural networks. Though I feel that there would be great potential in figuring this out - the energy efficiency and capabilities of the brain are truly amazing.

Huh?

>Learn

Depends on the neuron, but things like NMDA and AMPA receptors are very well studied at this point and are the primary portion of synaptic plasticity, again, depending on the neuron: https://en.wikipedia.org/wiki/AMPA_receptor

>Compute

Again, depends on the neuron, but dendritic integration is a classic example: https://www.sciencedirect.com/topics/neuroscience/dendritic-...

> Represent information

The do so in the frequency space. But again, it's highly dependent on the system of neurons : https://en.wikipedia.org/wiki/Neural_coding

That said, holy cow yes. The brain is really something when it comes to power usage.

Neural coding is not a solved problem. Rate codes are not sufficient to explain all experimental data.

Neither is learning; we know some things, sure, we have models for long- and short-term plasticity, but we have no idea how learning actually works.

With computation, there are many hypothesis and dendrites do indeed do more than you would think, but again, these are just small details.

I was just wondering what had happened to the Duke Nukem Forever of tech products. So.... nothing, really. Just more lab success. At this rate, HP will be dead long before this comes to fruition.

Reply on news.ycombinator.com