Haven't seen this train of thought discussed. Takes some interesting comments from Karpathy and others to argue that transformers can distill down things well beyond just text and images