> vision, audio, proprioception and more... what will happen when these modalities are incorporated

First of all, there's no evidence that a token-based model, as you yourself call it, can be made to work on visual or sound data where you have more than one dimension to the data. Second, even if you manage to make it work, it needs to be able to relate the different senses somehow, and no training data is going to be able to give it information about how things relate, it will have to use heuristics to determine that and I can only see that failing spetacularly because much of the relations between what someone sees and how someone acts can take very large amounts of time or never materialize at all (e.g. studying maths VS actually using it to solve a real-world problem).