Claude Shannon Demonstrates Machine Learning (1952)

Catastrophic interference, also known as catastrophic forgetting, is the tendency of an artificial neural network to completely and abruptly forget previously learned information upon learning new information.

https://en.wikipedia.org/wiki/Catastrophic_interference

Of course neural nets can update their weights as they are trained, but the problem is that weight updates are destructive: the new weights replace the old weights and the old state of the network cannot be recalled.

Transfer learning, online learning and (deep) reinforcement learning are as susceptible to this problem as any neural network techniques.

This is a widely recognised limitation of neural network systems, old and new, and overcomging it is an active area of research. Many approaches have been proposed over the years but it remains an open problem.

Sorry, I didn't get what you mean about the dynamically evaluated reward function.

Seems plausible, that's worth an upvote for you.

Mechanical arms under the maze not much different than an XY plotter, move the mouse similar to Pac-Man but without needing a game controller or human player attached.

Under the maze an auto position (re)locator, followed by directional XY servo advancing plus a N/S/E/W impact sensor provide the input data for the memory to learn the maze from. This stuff actually all could be implemented in hardware with no memory needed.

All you need to do is place the mouse anywhere, grab the joystick and head on directly to the target like Pac-Man without any enemies or time limit. This would be a really easy game so far.

If you didn't see that wall coming and hit one anyway, the gameboard lets you know. At this point you already know which direction you are going so now you really know where that wall is. The gameboard doesn't need to tell you whether it's a N/S/E or W impact, surely you could figure it out.

Or if you did have just enough memory for a map of the maze but no _software_ hooked up to the sensors you could run the joystick yourself whatever direct path you might want and that would be enough of a maze map, whether you entered the data manually or could automatically somehow.

Then if the gameboard were able to play the movements back from memory when the mouse was placed at the identical starting position, that would not require code either. You could even play back from less than the full path on a previously memorized trajectory.

While you're at it memorize the entire maze anyway, this is just a preliminary training session, later there will be a quiz.

Then, without looking at the maze any more, just do the exact same series of joystick movements needed to get from anywhere directly to the goal as long as you know where the mouse is at the start. The gameboard itself tells you exactly where the puck is dropped and you just take it from there. If you can't remember and take a wrong turn, the gameboard throws a flag and lets you know it's bad. So you can refresh your memory. But you'll have to remember which way you are going, which way you were coming from and which ways were right or wrong, even better than when you were looking and hit the wall anyway. It hardly ever happens and it was just a little wall wasn't it.

Well, that's training. When you get really good then you don't need to look at any maze to begin with, and you can handle any combination of maze rearrangement including during real time. As long as it's truly solvable and you've got enough time.

When the machine actually did learn without a human or a joystick, that's ML. The final result was simply the full details of the entire landscape as if it was static. No actual higher machine intelligence necessary, the intelligence was naturally still contained within Shannon.

He made sure the machine would move in the useful ways and collect the desired data as it goes no matter how long it takes to memorize the maze.

His program allowed any maze to be memorized automatically, then perform without obstacle. The memory alone could guide the performance perfectly after that, but in real-time an unexpected obstacle had to be "learned the hard way" again, however after the memory was updated the new obstacles were perfectly dealt with after that. It only learned what it needed to learn and knew when it had enough. The rest of the time it just ran perfectly until more learning was needed. And it always got it right.

Keep in mind how many full-speed collisions it takes for the mouse just to memorize enough of the maze to get to the goal the first time.

No forgetting anything but a single element at a time, and only when it undoubtably needs updating.

In this case a single bit, each changed bit only allowed if it gives a more realistic representation of the maze, building only on what is absolutely known.

He had more digital memory to work with than almost anyone in history at the time, even though it's just one relay per bit somebody was bound to think of numbering the possible states 0 and 1 sooner or later.

The actual electro-mechanical processor and relay memory banks were much bigger than would ever fit under the gameboard. Tonnes.

All making up a static program which reads the gameboard sensors in real time, controls the mouse along as much proven path as there is before searching, and builds the variable maze map in memory systematically but mindlessly. Every time the routine is complete on a static maze it's always correct.

So the program itself is good enough to be burned to a ROM and you need almost insignificant RAM.

RAM is only for the intermediate variables needed by the program which can always start zeroed, plus a single variable in memory of the exact position within the current maze, and the maze itself at any one time to whatever degree it is known.

When it seems fairly intelligent but the machine didn't learn a damn thing compared to how smart it should be, that's current AI.

Shannon had an auto-learning ML machine including its immutable _software_ which on any static playing field eliminated all mistakes then after self-learning did exactly what was expected without mistake from that point forward. Other features built on top of that were the only thing that made the mouse hit a wall again. Even though a human could do it the first time without hitting any walls if looking at the gameboard. When the human can not see the gameboard and has to depend on the sensors alone, he will make mistakes the pre-programmed robot will not make. But the machine is not expected to learn the exact same path to success that the human does even when the exact same goal is reached. Shannon did it without ever having anything of consequence be unknown in memory except what could be easily audited in full detail whenever needed because it was small and to the point.

Kind of intelligent the way he did that.

Seems like maybe a lost art.

That's neither here nor there though. The brain doesn't have to follow a specific model (e.g. the shaky model ML uses, and I alluded to), for the analogy to work.

It just has to have memory and remembrance as a core attribute of learning, which it does.

Whether this happens through neurons and weights or some other mechanism, it's still remembering. Your counter-argument focused on implementation details.

>Remembering certainly plays a role in learning, though it is but one component. For it to be what learning is, everything has to be exactly the same with every instance.

Well, remembering doesn't just "play a role", it plays the biggest role in learning. It's not even remotely optional: without remembering there's no learning.

And inversely, discovering ourselves what we learn or applying it are both optional. I can listen to a professor tell me about something someone else discovered, and never apply it, but as long as I remember that information, I have learned it.

And, of course, as I already wrote, the contraption doesn't just remember but also discovers the solution (and can even re-apply it).