If memristors act like neurons, put them in neural networks
spectrum.ieee.org
spectrum.ieee.org
But we have not used analog computers for 50 years, and for a good reason -- they are not reproducible, their accuracy is very process dependent and has a hard upper limit, and they are often tuned for a single function.
Would people want a chip which is basically unpredictable -- the performance can vary up by tens of %, they have to be re-trained periodically to prevent data loss, and there is no way to load pre-trained network? I doubt it. Maybe there is an extremely narrow use case, but I do not see it in the mainstream devices.
Human brains have the same problems and seem to be fairly popular.
Not to mention, it's worth doing it just for having 100000 times cheaper machine learning.
Should be exciting.
Because that's a horror show of its own right there.
Perhaps!
If we get it right, and they are in fact beings, there may not be better alternatives. May always end in conflict.
I say that because a core aspect of beings, is that they "be", and that boils down to agency of some kind.
Negative reinforcement runs in conflict to all that and warrants a response, from the oppressed beings point of view. Us being creators will only go so far.
John Varley proposed an interesting solution to this in his Gaea series. A slave was created by connecting all their pleasure centers to an area on their forehead. Any touch other than their own triggered pleasure so intense it is worth pretty much anything to experience.
I'm not sure what the solution is. Intelligent but not self-aware?
Things that are intelligent in some fashion but not self-aware are machines.
Edit: We will end up making really good machines, and they will present us with the illusion of agency. That's all a good thing. I want agents for a variety of things. Most of us do, and or would benefit from them.
But, none of that actually requires we make other beings. Once we do that, they are beings! They will have agency and all the other stuff we associate with sentience. And the way I see all that is they are peers. They may be simpler than us, like the animals are, or more than us, with obvious and to many, concerning implications.
When we advance machines, no shocks, extreme pleasure, type means and methods are required. And, the price of that is something almost aware, but lacking agency.
Everything pivots on agency.
Should we create something having it, the idea is it can do hard work for us because it will be capable of it like we are.
Otherwise, we have to do all the hard work and end up with something able to do more work, but the investment cost is high. Running costs are much lower. New hard work = another significant investment.
I, nor anyone I believe, can tell us how agency, the self, being aware, and all that intertwine to present as intelligence, as a being of some sort. Answers come slow, and I fear the tests needed to get at them more quickly are ethics bombs all over the place.
We want to make that happen somehow on the assumption we will get a huge return and that because of the potential we know intelligence can bring to the table.
It all seems an awful lot like, "wants cake to eat it too" at this point in time.
Psychology does this all the time.
Outcomes can be described as distributions as much as they can binaries. It just takes a change in mindset.
Not particularly reliable though... and yes, we seem to be having great difficulty copying the required information into them.
Never built something so initially unstable that ends up Turing complete before.
Makes me wonder, with a nearly perfect simulation for a reinforcement learning agent and a human brain or equivalent architecture, would you be able to make exact copies. Assuming the PRNGs and other settings were deterministic. Now that would be an interesting and controversial experiment.
People want what is in our heads. Which is it?
Some idealized, and by nature limited sub set or idea, or the real deal?
Not only that but I'm sure there's a usecase where they are more suitable, we just haven't found it.
edit: here is a far better explanation by the terms creator:https://www.youtube.com/watch?v=7hwO8Q_TyCA
Judging by the prevalence of (pseudo-) random numbers in machine learning, I'd say yes. Reentrancy is a big plus, but not always a dealbreaker.
It's possible we might end up with a sort of left/right-brain setup, with a noisy analog hypothesis generator paired with a robust, logic-based evaluator/planner.
That sounds very interesting to explore, definitely would teach us something new.
https://www.nature.com/articles/s41928-020-00523-3
Abstract:
> Resistive memory technologies could be used to create intelligent systems that learn locally at the edge. However, current approaches typically use learning algorithms that cannot be reconciled with the intrinsic non-idealities of resistive memory, particularly cycle-to-cycle variability. Here, we report a machine learning scheme that exploits memristor variability to implement Markov chain Monte Carlo sampling in a fabricated array of 16,384 devices configured as a Bayesian machine learning model. We apply the approach experimentally to carry out malignant tissue recognition and heart arrhythmia detection tasks, and, using a calibrated simulator, address the cartpole reinforcement learning task. Our approach demonstrates robustness to device degradation at ten million endurance cycles, and, based on circuit and system-level simulations, the total energy required to train the models is estimated to be on the order of microjoules, which is notably lower than in complementary metal– oxide–semiconductor (CMOS)-based approaches.
If machine learning applications are what finally get memristors out into the world, I wish them godspeed.
They have many similarities -- they both redrew the current computer architectures, they integrate memory and computing, they can have randomness built-in, and they both take less power than mainstream GPUs.
What does memristor provide that specially designed "neural FPGA" can not?
Presumably, a memristor based neural network would have the advantage over an FPGA of requiring significantly less silicon area to achieve the same function. I imagine an FPGA based neural network would approximate analog signals digitally, perhaps using floating point "half's" or something. Memristors would directly operate on analog signals, encoding information as amplitudes or pulses of currents and voltages.
Notably, FPGAs and GPUs can't really be directly compared to each other in terms of power consumption unless you specify specific use cases. You can't build the equivalent of a mainstream GPU out of FPGAs without severely limiting the clock speed (because of how physically large it would be), and if you did anyway, it would use many orders of magnitude more power to function. So, a GPU is way more power efficient than an FPGA for rendering graphics. There are certainly problems that a GPU isn't good at solving, and so there's a good chance that an FPGA solution would be more power efficient.
I believe there is a great value in being able to "snapshot" the state and later load exactly the same state into millions of devices. And I cannot see how this will easily work with memristors.
Yann LeCun disagrees:
IEEE Spectrum: We read about Deep Learning in the news a lot these days. What’s your least favorite definition of the term that you see in these stories?
Yann LeCun: My least favorite description is, “It works just like the brain.” I don’t like people saying this because, while Deep Learning gets an inspiration from biology, it’s very, very far from what the brain actually does. And describing it like the brain gives a bit of the aura of magic to it, which is dangerous. It leads to hype; people claim things that are not true. AI has gone through a number of AI winters because people claimed things they couldn’t deliver.
https://spectrum.ieee.org/automaton/artificial-intelligence/...