A Spreading Activation Theory of Semantic Processing (1975)

researchgate.net

13 pointsabrax31413 comments

If I'm interpreting correctly, this is a psychology paper taking examples from early algorithms in AI research and then applying them to constructing a psychological model of the human mind with them. Presumably to inform other psychological techniques?

Although that's structurally approximately correct it understates the influence that SA had, and continues to have (IMHO, see below). Mathematically, Spreading Activation is essentially contextually driven probability integration, sort an unruly version of Bayesian Networks -- iterative matrix multiplies of a weighted connectivity matrix given a set of initial "context" weights. My hypothesis is what embeddings are doing for the GPTs can be understood as essentially spreading activation over distributed representations connected to each token. In classical (GOFAI) SA, per Collins and Loftus, you spread to/from the tokens, whereas in modern SA (by my hypothesis) you are spreading through a high dimensionality network represented by the embedding vectors, instead of via the tokens.

To say it in more plain words, "spreading activation" describes the activity through a weighted graph, that could be a good, old-fashioned, semantic network (but with weights) or a neural net.

It's fundamental and foundational connectionist stuff and for once I'm not using the term dismissively. It seems like spreading activation goes to the heart of connectionist ideas about storing knowledge in a graph, what used to be known as a Parallel Distributed Process model, apparently.

Reply on news.ycombinator.com