On a related note: recently, I released a visualization of all MLP neurons inside the llama3 8B model. Here is an example "derivative" neuron which is triggered when talking about the derivative concept.

https://neuralblog.github.io/llama3-neurons/neuron_viewer.ht...