They are low hanging fruit. This is the sort of thing it ought to be good at, and it's not particularly surprising that it is. But it is also not what is being promoted by LLM advocates on the public stage.

Investors are not putting billions into OpenAI to crack historical ciphertexts. This is supposedly a trillion dollar general purpose artificial intelligence, but still can't reliably tell me how many p's are in 'raspberry'.

Cracking pre-computer era ciphers with LLMs is like me claiming I have a super-efficient hypersonic precooled hybrid air-breathing rocket engine that will revolutionize all forms of transportation, and then for a demo bragging about how nicely I can grill with it at my backyard BBQ.