The thing is, it's not gibberish. A sufficiently small language model can be understood by humans: https://twitter.com/karpathy/status/1645115622517542913
The explanation is perfectly sensical, just too complex for humans to understand as the model scales up.
The thing you're looking for - a reductive explanation of the weights of a ANN that's easy to fit in your head, does not exist. If it were simple enough to satisfy your demands, it wouldn't work at all.