Any idea how ascii art is even parsed by these models? I know simple language models would basically just disregard it. Does gpt have some built in parsing that treats each symbol in ascii art as a token to try and actually learn a model for it? This would be an interesting thing to try and build. I feel like I remember seeing a vision transformer model that you could give the start of an image to and it would make up the rest, I guess this would be a similar concept. Overall it's not necessarily a logical extension of a language model though
Each special symbol is a distinct token.
This is probably the extent of ascii art it was trained with. There’s a link to an old geocities site that probably made its way into the pile