was watching some youtube about the text/vision models and the trouble with language as a descriptor; their novel idea was to use coordinate systems on visual imput so the model would map out an image, then it tags what's looking at with coordinates or boxes, and then think with those box coordinates, providing a level of disambiguation.

If you tried to explain the same stuff using purely text, you'd also need to come up with some kind of "language", which you know is programming.

So 1: you'd need to fine tune a model to actuall succeed; they're not reaching AGI any time soon with the current batch. 2: you need to develop a lingual DSL for it, as they'll never do much of anything without some kind of glue and disambiguation.

IDK, sounds like it has brute forced my password already.