> if nobody's surprised, then nobody has learned anything
Really? You're saying that as long as you assume something is true, there's no value in finding out if it's actually true or not?
Really? You're saying that as long as you assume something is true, there's no value in finding out if it's actually true or not?
I am equating learning to surprise, though you could disagree with semantics.
However, there isn't time to test out every single assumption we could generally have.
Therefore, the more worthwhile experiments are ones where we learn something interesting no matter what happens. I'm equating this with "surprise," as in, we have done some meaningful gradient descent or Bayesian update, we've changed our views, we know something that wasn't obvious before.
You could disagree with semantics there, but hopefully we agree with the idea of more vs. less valuable experiments.
I'm just not sure whose model of LLM dynamics was updated by this paper. Then again, I only listened to a couple minutes of their linked YouTube discussion before getting bored.