Yes. I think we may have enough for "full finetuning" and erasing to a large extent the previous knowledge. But that's still very far off for pretraining.

"RomeGPT" is next on my list of Monad successors and to give you a general idea, we have on the order of tens of millions of words in classical Latin (and biggest source will… Augustine). There was a BERT Latin project that was able to collect roughly 500 million words in all with mostly early modern and modern Latin.

In comparison I'm currently part of a project to pretrain a French model and we need… 140 billion words.

There's a sufficient number of old texts - far more than necessary - if we were willing to converse with it in Latin*! Perhaps we could make a translator LLM to converse with the 17th century LLM...

* I'm not talking about Roman-era Latin, Latin was the scholarly/international language for Europe for a long long time. AFAIK, most European Latin literature is untranslated...

Sounds like something right up the domain of synthetic data.