Hacker News Reader

Comment by XuMiao | Hacker News Reader

One key question is whether symbolic AI is the right model of the world. It underperforms vector based AI on many specific tasks. But human experts heavily reply on it to communicate with each other. If symbolic AI is not the right model, P vs NP problem might be just irrelevant. Human philosophy is full of craps. We will lose a lot of beliefs. Elon will be right, we will abandon human languages, and connect through a cable in the brain. Everyone will relearn every thing from NN.

If symbolic AI is the right model, but difficult to build algorithmically. Vector based model just help to make it faster and better. Then we humans are fine. We simply proxy the lower level optimization to AI. Our functionalities will be shifted just like what happened when engine was invented hundreds of years ago.

View on HN

Of course symbolic AI is the way to go. We communicate with text messages that consist of words that we internally convert to word2vec style vectors to detect similar words. One more thing we do in our heads: we build a graph of those word2vec symbols. When I read in a book "a cat is sleeping on a tree" I instantly build a small graph where nodes Cat and Tree are connected with an edge labeled Sleeps. I may visualize this as a picture, but that's not necessary for AI. In fact, some people can't visualize anything, but they can definitely think. How? They can still build this knowledge graph. I believe that this graph representation is the limit of our intelligence: there are many facts out there that we can't possibly think about because they don't fit this graph model. It's like the set of real numbers can't be squeezed into the set of rational numbers: most of the numbers are irrational.

Does one’s knowledge of how to ride a bike have anything to do with such graph structures? Or is intelligence unrelated to such skills? It seems that the intelligence involved in having a basic conversation would engage a lot of such skills, even just social tact.

I think intelligence isn't involved in riding a bike. Understanding the theory about riding a bike is a different story, and that understanding is a knowledge graph.

We perceive ourselves building a knowledge graph but at the physical level how does that happen? Does this graph materialize physically as neuron connections? Or is it just an abstraction the brain makes, a part of the subjective experience of thinking?

It's certainly true that we think in symbols but they exist somewhere in the mushy goo of neurons, could symbolic thinking emerge from large ANNs in the same way?

This graph is a high level abstraction, of course. How exactly neurons store information is interesting, but hardly relevant here. My guess is that one symbol is stored in a very sparse subset of neurons and each neuron acts a bit like a node in a DHT. All together these neurons implement a fast DHT where a word2vec graph node acts as a key. On top of that this "wet DHT" can quickly find keys nearby, i.e. in can instantly return all neighbors of word2vec("apple").

I think ANNs implement only the word2vec function that translates images or sounds into symbols and vice versa.

Reply on news.ycombinator.com