Hacker News Reader

>The big difference this time is that the ceiling with machine learning is high enough for large-scale profitable applications. That wasn't the case with the previous rounds.

My company (fortune 500) depends on expert systems, some even date back to the 80's AI boom. Doing what we do, at the scale we do it, would simply not be possible without them.

Lots of companies make a lot of money with expert systems, but they have no incentive to fund AI research since expert systems have hit a clear capability ceiling.

> Lots of companies make a lot of money with expert systems, but they have no incentive to fund AI research since expert systems have hit a clear capability ceiling.

I'm not sure I follow the reasoning here. Is the suggestion that AI cannot do better than expert systems?

I think the point is that just because companies are making money off of AI advances, that doesn't necessarily mean we won't hit another AI winter once we hit a ceiling with current techniques.

Okay but what's the relevance of expert systems to this argument?

This isn't the first round of AI that's been useful for "large-scale profitable applications"

Expert systems are one of the, supposedly not profitable, previous AI booms the preceded an AI winter.

My point is, they were and are profitable, but clearly that didn't keep winter at bay.

What's a good resource for learning about expert systems? I'm curious about this "dead end" but most of what I've found online has been about the ai winter and not the technical details of expert systems.

The Wikipedia article isn't a bad start. https://en.wikipedia.org/wiki/Expert_system

I have previously built an expert system - a system of codified expert human knowledge built into a system of executable, interrelated rules. Rule engines are the execution engines of expert knowledge in an expert system. The RETE algorithm is another good place to start.

https://en.m.wikipedia.org/wiki/Business_rules_engine https://en.m.wikipedia.org/wiki/Inference_engine https://en.m.wikipedia.org/wiki/Rete_algorithm

When I was first learning, I downloaded and played with CLIPS, a pretty cool old expert system tool built by NASA. https://en.m.wikipedia.org/wiki/CLIPS

:) The incentive all good corporate robots need, to fund research is generally completed research funded by the public. We know this. Don't worry yourself too much. We'll call you when we need a freeloader.

It's no my corporation. It's just the one devouring my soul.

>How long until the next big idea? The last "AI winter", after expert systems, was 15 years.

Oddly enough neural nets languished in the frozen tundra for 50+ years if you take it back to perceptrons (mid 60's)[1]. Misnky and Papert's take down of perceptrons set the field back years. If you want to be more strict then it's been 31 years since Rummelhart and McClelland published their book[2], based on their earlier work that Hinton co-authored. They laid out multilayer NN with back-propagation. I got hold of the book in the UK in '87 and used it to code a multi-layer back-propagation network. It was a machine that would read out aloud given some text, even for words it had not seen before (good luck with through, though, and trough.) The problems were data and processing speed. I had to hand code features and wait for overnight training runs on VAXen and later a Sun-3 clocked at 16.67MHz[3]. It took big data an brute force to get the glimpses or magic that we see now. The fundamentals have been around for a long time.

It may be the case that one of the many approaches that fell by the wayside will make a comeback a al NN. NN added more layers and tweaked backprop, plus data and speed. It may be the expert systems will rejuvenate as something "new" with machines writing their own rules and somehow getting over showstoppers like the unmanageability of a large rule bases. It's probably going to be some combination of things that we already know about, along with yet more data, and processing power that will get us to the next level. The odds are that whatever that beast is will be self-training and regulating in an analogous way to back-prop but with logic thrown in. Good luck to us all for inventing such things, and for trying to understand what they are really doing.

[1]https://en.wikipedia.org/wiki/Perceptron [2]https://mitpress.mit.edu/books/parallel-distributed-processi... [3]https://en.wikipedia.org/wiki/Sun-3

Each was overhyped at the time, and each hit its ceiling.

That each was over-hyped I can buy. I'm less convinced of the latter part of that statement. Take ANN's. You would have thought that they had hit their ceiling at one time as well, but eventually the confluence of more compute power, more data, and better algorithms resulted in the recent resurgence. I don't see any reason to think that some of the other existing AI techniques might not also experience an explosion in utility, if combined with massive increases in compute power, more data, and better algorithms as well.

I think that machine learning, particularly deep learning, will increasingly be used in combination with some of the older AI techniques to produce hybrid systems, which will solve previously intractable problems. AlphaGo is a great example; of course it used deep learning, but it also used classical search, as well as reinforcement learning.

The point is, progress isn't linear but exponential: each new technique multiplies our capabilities.

I think that machine learning, particularly deep learning, will increasingly be used in combination with some of the older AI techniques to produce hybrid systems, which will solve previously intractable problems.

This is what I believe as well. I've been doing some work lately on some older approaches to automated abductive inference, with an eye towards doing some of this kind of hybridization. No results to report yet, but I do think there's some meat on this bone.

I feel that Numenta will break the ceiling that we're coming to now. I have quite a few upcoming personal projects that I plan to use there neural network(HTM) structure to enhance, the key being enhance.

The mindset that I feel is lacking in the ML/AI community is that we should be creating systems that can be augmented and improved using AI, the product itself should stand alone without using any AI at all. This is similar to how JIT compilation can improve the machine code as the program runs. Part of how our own brains work is that the cerebral cortex doesn't really have any direct control over the body at all, it can only watch and identify patterns and project intentions to the the rest of the brain/body. This is a passive structure initially, it observes both the environment and how the lower structure interact with that environment. Only after it's observed and learned the "commands" can it begin to influence how the body moves around and interacts with the world. You could look at the CLI terminal as an example, on it's own I can interact and use it perfectly fine. But lets say I added some patterning matching and prediction AI that could make really good suggestions about what command I will type next, at some point you could promote these suggestions so that they always execute when the AI predicts you would use it. This is exactly how the brain works.

Most of my ideas of applied AI revolve around improving our HID interfaces to computers themselves. For example, I'd like to reduce our chances of developing RSI related to keyboard/mouse/trackpad usage and I feel that chorded keyboards are more ergonomic. But chorded keyboards are slower to use and harder to learn. The mapping of the chords is where I plan to apply some AI, initially I'm going to randomize how chords are bound to character inputs, these random values will use many fingers to input 1 character. As the user learns and uses the chords to input characters the AI can identify common patterns and promote those patterns into chords that use less and less fingers to input, thus improving the input speed potential of the user. The next place to use AI in this project would be autocorrection and using the user's input context(program, filetype, time, surrounding text) to continue improving these bindings and making completion suggestions.

From Yann LeCunn. ---

Numenta, Vicarious, NuPic, HTM, CLA, etc.

Jeff Hawkins has the right intuition and the right philosophy. Some of us have had similar ideas for several decades. Certainly, we all agree that AI systems of the future will be hierarchical (it’s the very idea of deep learning) and will use temporal prediction.

But the difficulty is to instantiate these concepts and reduce them to practice. Another difficulty is grounding them on sound mathematical principles (is this algorithm minimizing an objective function?).

I think Jeff Hawkins, Dileep George and others greatly underestimated the difficulty of reducing these conceptual ideas to practice.

As far as I can tell, HTM has not been demonstrated to get anywhere close to state of the art on any serious task.

I still think [Vicarious] is mostly hype. There is no public information about the underlying technology. The principals don’t have a particularly good track record of success. And the only demo is way behind what you can do with “plain vanilla” convolutional nets (see this Google blog post and this ICLR 2014 paper and this video of the ICLR talk).

HTM, NuPIC, and Numenta received a lot more publicity than they deserved because of the Internet millionaire / Silocon Valley celebrity status of Jeff Hawkins.

But I haven’t seen any result that would give substance to the hype.

Don’t get fooled by people who claim to have a solution to Artificial General Intelligence, who claim to have AI systems that work “just like the human brain”, or who claim to have figured out how the brain works (well, except if it’s Geoff Hinton making the claim). Ask them what error rate they get on MNIST or ImageNet.

---- Reference:

http://fastml.com/yann-lecuns-answers-from-the-reddit-ama/

MNIST error rates are noise these days, but otherwise agree. These guys have been around for the entire deep learning boom of the last decade, and have never produced results to match their handwaving predictions. They're the Moller Skycar of the field.

That implies we have hit or at least know the exact capacity ceiling for, in particular, recurrent/dynamic networks (I do agree that with static networks it seems more clear cut). While it will not be general AI, I'd assume, things like the neutral Turing machine, pointer networks, and many other open research areas leave room to speculate that we have not yet "found" that ceiling.