LLMs are trained on large-scale text and code data, and fundamentally, they predict the probability distribution of the next token given the preceding tokens. There are studies showing that even with this simple learning objective, grammatical and semantic features, as well as relationships between concepts, emerge in the internal representation space in a structured geometric form. However, I understand it's more accurate to say that rather than the model retrieving sentences from a separate 'meaning space,' it converts the prompt into a contextual vector representation and then sequentially generates the next tokens from that state. I could be wrong about this.
Since code is fundamentally built on 'patterns' and mostly follows established conventions, it becomes easier to predict, which makes LLMs very effective for programming. After all, programming is designed to converge toward specific patterns. Frameworks with IoC are a good example of that structure.
Recently, even Linus Torvalds has acknowledged AI as a useful tool, so it's probably good to use it. That said, a lot of people still hate LLMs because they're afraid of losing their jobs