> In principle I could replace a large language model with a map from all possible input strings to output token or output token distribution without any weights.

What is an output token distribution except a set of weights?