It’s at least a million times slower than non-encrypted computation. 1000x or 100x would be a huge progress.
Accelerators are being developed that claim to get down to 10x, though i think they will be more like 100-1000x, which would still be a huge improvement considering how people use LLMs today for basic tasks like string matching.