How do machines ‘grok’ data?

I think its shorthand but still "accurate"

all of the following are on the same line through the origin.

    8+2 1+1 2+8 3+7 4+6 5+5 6+4 7+3

and the set of lines through the origin is double-covered by the output embeddings. which means there are twice as many slices in the output embedding as in the input embedding.

so its not that 1+1 bisector points at 1 and so the sum is 1. it points at 2 because of the double cover.

this double cover is easier to see if you examine step 2.2

    | cos(wk(a − b)/2)|(cos(wk(a + b)),sin(wk(a + b)))

and set a=b and then increment around the circle.

when a=b the leading |cos| term is 1 and the vector is roughly cos(2a),sin(2b). now go from a=b= 0 to 2pi and youll see the embedding will loop around twice as it varies from 0 to 4pi

the outside of the circle is labelled twice with 12 at top and bottom. the line through the origin picks this labelling. not the input embedding.

                     , - ~ 12~ - 1
                 , '               ' 2
               ,                       3
              ,                         4
             ,                           5
             ,                           6
             ,                           ,
              ,                         7
               ,                       8
                 ,                   9
                   ' - , _ 12_ 11 10

Yep. Computers are just binary-state machines. Everything you can do with a computer, you could do with water channels and pulleys and gates (if you could assemble trillions and trillions of them). No one would ask whether the water-channel system 'groks' things, but because they are miniaturized and (to 99.9999% of people) esoteric in their actual workings, people treat them like magic.

Clarke was right, but sadly it doesn't take alien technology for people to start anthropomorphizing computers or thinking that they magically have transcended mere (though obviously very complexly-arranged) logic gates, it just takes a GUI that spits out algorithmically-generated pictures and text.

I thought the article did a very good job of explaining why the term 'grokking' was used to describe this emergent behavior, and how it fit Heinlein's original definition. I'm curious which part of their explanation you feel is incorrect.

Regularization as a concept is taught in introductory ML classes. A simple example is called L2 regularization: you include in your loss function the sum of squares of the parameters (times some constant k). This causes the parameter values to compete between being good at modeling the training data and satisfying this constraint--which (hopefully!) reduces overfitting.

The specific regularization techniques that any one model is trained with may not be publicly revealed, but OAI hardly deserves credit for the concept.

Firstly, thanks for linking the paper.

I think you’re being unfair. I say that as someone who has done his share of charlatan hunting.

They explicitly list their contributions in the paper. They’re not saying they did something they didn’t. It’s not like that bogus "rat brain flies plane" paper that was doing something simple under the hood and then dressing it up as a world changing discovery in order to gain funding. They are doing something simple and studying it carefully. This is grade A science as far as I’m concerned, every bit as good as studying the motion of the stars and trying to find patterns.

It did not occur to me that NNs might be able to extrapolate a tiny training dataset into complete solutions. ML scientists are taught from an early age that you need to have a large dataset to make any progress. I don’t know whether this paper counts as a "discovery", but it was certainly a fun read, which was nice enough for me.

In some sense this paper is a proof of the idea that NNs can extrapolate, not merely memorize. This is in contrast to recent work where researchers have been claiming otherwise.

The weight decay study was also a nice touch. It’s no discovery to say that weight decay helps, but it’s a reminder to use it. I haven’t. Adam has always been reliable for me, and now I wonder if it was a mistake to shy away from weight decay. (We wanted to keep our training pipeline as simple as possible, in case any part of it might be causing problems. And lots of unexpected parts did.)

Now, I haven’t read the article submitted here, only the paper you linked. Maybe they’re claiming something more than the paper. But if so, then that is a (very real) problem with scientific journalism, and not necessarily the scientists themselves. It depends how much the scientists are leading or misleading the reporters. It’s important to separate the criticism of the work from the reporting around the work.

I’d also be curious if you have any citations for your claim that if an outcome space can be represented as a sequence of conditional probabilities, then NNs are guaranteed to find a solution after some unknown amount of training time. This is a surprising thing to me.

The interesting part is that with training you can go:

random -> generalization -> overfiting -> better generalization

The last step was unknown IIUC.

The relevant trick is "regularization":

> This is a process that reduces the model’s functional capacity — the complexity of the function that the model can learn.