Numenta releases brain-derived learning algorithm package NuPIC
numenta.org
numenta.org
It looks like this new version is a completely different implementation [1] but, in my experience, the idea is sound and the approach is very promising.
I'd love to revisit the project at some point. The fact that this is now open source makes that much more appealing.
[1] http://numenta.org/faq.html#whats_the_difference_between_thi...
I feel like they're making up their own terminology in a bubble instead of integrating with the rest of mainline research?
CLA - Closest 'mainline' term would be a recurrent neural network, but the neurons and network organization are very different.
OPF - This is more of a runtime environment or a set of tools to work with.
Encoders - These are actually a step below what you think of as an autoencoder. They translate many data types into useable inputs for the network. Then the spatial pooler takes over and tries to find efficient representations.
While it's possible to describe Numenta's work with existing frameworks, and unifying nomenclature is always useful, I hope you'll take the time to learn the system so that we can apply the correct shared terminology!
Full Disclosure: I used to work for Numenta.
https://www.groksolutions.com/technology.html#cla-whitepaper
There were some earlier ones that used to be on the Numenta website (before the name-change), but I don't see those posted now.
With the exception of being able to handle on-line learning, I have a slight preference for their previous, more abstract approach. Their current approach seems to model the details of the neural connections at a lower level than what seems necessary to me. This is not, by the way, any kind of valid scientific opinion, just the views of a widely-read amateur.
Then Chapter 6 of "On Intelligence" From there you should be and to tackle his whitepapers which can be found here: http://blog.mohammadzadeh.info/index.php/hierarchical-tempor...
If you are interested in generalized strong AI, this is some of the best work out there.
Just describe the basic mathematical formulations and define what is different between you and the other. I wish they would do this more and write less verbose papers.
While I was in grad school, I spent several weekends tinkering with my own implementation of a very simple HTM to process shapes in Go (the game). It never did much in the way of playing Go, and I stopped messing with it after something else more practical got in the way, but I'd love to dust off that code and now seems like a great time to get back into it.
The narration on the audiobook is very high quality, and, for me at least, the ideas were quite accessible through audio. I highly recommend it if you're the type of person who might purchase such a book in paper and never get around to reading it (but you have time to kill in the car).
On Intelligence is a prescription for an intelligent system which Numenta's software is intended to fill. So On Intelligence would tell you the motivation for the system but not how the system works in any detail.
Ever since I first read the book "On Intelligence" I thought Jeff Hawkings was on the right track with Strong AI. The insights I got from reading the book I thought were invaluable. I don't know how close they are to Strong AI or if they are closer then anybody else but I would expect that if they were making any real head way into the field that Google would already have made an offer to buy them.
At least that is what I would do if I wanted to be the first to control the technology. The fact that nobody seems interested in acquiring them sends the signal that they probably are no better than anybody else building AI tools.
>>"The fact that nobody seems interested in acquiring them sends the signal that they probably are no better than anybody else building AI tools"
Seriously? The bar to being successful is an acquisition offer from Google? I'd understand if money was the driving factor. But Jeff Hawkings has already made a bunch of that from his previous ventures and to me at least, it seems that he's genuinely passionate about building something extraordinary. So, I'm still rooting for Numeta.
In the SV bubble the measure of success may be getting "acquired", but I hope Numeta is more about actually creating some groundbreaking advancements in AI & not just getting acquired.
The thing is, Since On Intelligence came out I haven't really seen any real products from them. Sure, they created some tools that they expect other developers to build projects on top of them but that is it. Honestly, with all his talks claiming how his algorithms are much better than the current state of the art I would expect really good image recognition, speech recognition or something like IBM's Watson to come out of their labs. But everything I've seen from them seems average. Not that much better than the state of the art.
If they had anything groundbreaking and I were google, I'd want to acquire that technology to further the goal of Strong AI. Google is interested in producing AI and will acquire anybody that can help it with that goal. Google is not interested in Numenta's technology, and I'm sure they've checked their technology, which makes me think that Numenta doesn't really have much to offer.
Whether Jeff is interested in selling or not is irrelevant.
They absolutely have a commercial venture going: https://www.groksolutions.com/
Context: I'm not familiar with ML though I've been reading some papers recently (I'd like to learn ML; I wanted to have some ML baddies in a game and let them play each other, etc).
I imagine the ML component being a black box with inputs and outputs, but it's probably more complicated than that. The FAQ says "The algorithm lends itself well to high-speed temporal data" which makes me think of high frequency trading or touch-screen input smoothing or something like that, but I don't see examples for those kind of things in their github tree (but maybe I'm just so unfamiliar with ML that I don't know the names of the concepts?).
I recall George was the technical lead at Numenta and has since left.
Hawkins' model centers on having an overall model of the brain based on prediction. I remember watching a Hawkins video where he makes the reasonable point that neuroscience has failed to create overarching visions and it seems plausible that such a vision (or several competing vision) could be useful. But I also think that prediction in-and-of itself probably isn't sufficient.
FWIW, When I earlier scanned their website, Numenta was trying to sell its software for big bucks and not giving any detailed documentation for their algorithm. Now they seem to be giving everything away for free. That might be an indication that this is the end of the road. But it's hard to know, of course.
George Dileep left Numenta and co-founded Vicarious Systems[1].
http://web.archive.org/web/20110412231749/http://www.dileepg...
I'm sure the code is great for machine learning purposes, but I doubt his theory has much bearing on how brains work. The brain is one of the most heterogenous and complex structures in the solar system.
The thing is, the hypothesis that there is a more-or-less basic neocortical algorithm wouldn't be disproved by modest amounts of variation in the neocortex.
And position that the neocortex is rather uniform is fairly standard neurophysiology as far as I can tell.
http://www.pnas.org/content/early/2013/01/02/1221398110.abst...
Per the wiki, the latter is the successor of former, but now it seems the may have resurrected the old brand.
It reads like he's gone the long way around to learn the same things ML already knows while resisting anything that ML doesn't already know.
Things seem to get quite after 2011...about the same time they started going to Grok. Maybe they found something.
Their latest whitepaper is available here:
https://www.groksolutions.com/technology.html#cla-whitepaper
Some of the earlier Numenta documents are archived here:
http://blog.mohammadzadeh.info/index.php/hierarchical-tempor...
Machine Learning: A Probabilistic Perspective (Adaptive Computation and Machine Learning series) by Kevin Murphy
They don't seem to reference much outside work as far as my cursory check revealed.
1. "On Intelligence", Jeff Hawkins, 2004
2. "An Organizing Principle for Cerebral Function", Vernon Mountcastle, 1978
1) Modelling the higher level phenomena is really, really hard to do at anything other than a basic level. We could just build formal logic systems, but we don't want to have to put all the information in there ourselves, so they've got to learn it somehow.
2) You've got to get the concepts into the AI to start with, we're still trying to work out how to tell the difference between objects in images.
3) It is being done, it just doesn't sound as sexy and so you don't hear as much in mainstream news.
4) It may have fewer short term benefits.
5) Biology has a working example we can try and steal.
Check out "On Intelligence". It talks about their reasons for following biological constraints, in opposition to your point of view, common among computer scientists.