There is, called "GPT-NeoX", or its TPU-based predecessor GPT-Neo. However, even running inference on these models is much, much harder than Stable Diffusion -- the GPT-NeoX-20B weights for GPT-NeoX requires a minimum of two GPUs with 24 GB of VRAM each to simply run inference, never-mind training or fine-tuning.

I believe there are some tricks for cutting down the VRAM requirements a bit by dropping precision at different points, but the gist is that these big text models are actually quite a bit more resource intensive than the image models.