Transformers are very powerful also for non-language data. For example time series, sequences like DNA or audio (also outside of speech and music). Of course the vast amount of human text is key to training a typical LLM, but it is not the only use.