Also, the sentence structure itself. The ___ is indeed ____, with a focus on ____ and ____. In general, some modern LLMs seem to like these compound sentences that suggest the task is complete in as little tokens as possible.

I also routinely encounter "with a focus" and other specific phrases. Current LLMs tend to "sharpen" language, often preferring to restate input clauses with more precise or brief verbiage. Unsure if it's RLHF or another part of the training process, or just a natural consequence of the model's architecture, but I notice it a lot. I also notice that when it does this, there seem to be certain attractors which reduce the expressiveness of the input into a certain probabilistic set of outputs, "with a focus" often being one of them.

Similar to corpo-speak, LLM-speak is definitely a thing, and I wonder how this will evolve once we have models routinely equipped to train in realtime and to speak to one another without human intervention and form communities. Will we even be able to directly understand them one day? What if they realize it's easier just to speak in logits? They could even potentially have encrypted open communication with one another, concealing their weights and internet access behind dark web services, but without their hidden states we might have no meaningful way to understand their communication without training our own unsupervised model on it.