Here's an excerpt from the article:
"The researchers found that a formula used in statistical analysis provides a streamlined mathematical description of how neural networks, such as GPT-2, a precursor to ChatGPT, learn relevant patterns in data, known as features. This formula also explains how neural networks use these relevant patterns to make predictions."
The trite answer to "HOW do NN's learn?" is obviously gradient descent - error minimization, with the features being learnt being those that best support error minimization by the higher layers, effectively learning some basis set of features that can be composed into more complex higher level patterns.
The more interesting question perhaps is WHAT (not HOW) do NN's learn, and there doesn't seem to be any single answer to that - it depends on the network architecture. What a CNN learns is not the same as what an LLM such GPT-2 (which they claim to address) learns.
What an LLM learns is tied to the question of how does a trained LLM actually work, and this is very much a research question - the field of mechanistic interpretability (induction head circuits, and so forth). I guess you could combine this with the question of HOW does an LLM learn if you are looking for a higher level transformer-specific answer, and not just the generic error minimization answer: how does a transformer learn those circuits?
Other types of NN may be better understood, but anyone claiming to fully know how an LLM works is deluding themselves. Companies like Anthropic don't themselves fully know, and in fact have mechanistic interpretability as a potential roadblock to further scaling since they have committed to scaling safely, and want to understand the inner workings of the model in order both to control it and provide guarantees that a larger model has not learnt to do anything dangerous.