It makes me feel very uncomfortable to say this - I don't want to appear presumptuous or patronising, but it honestly sounds almost like you have little or no clue about this.
LLMs are not about code generation. Code generation is almost like a side-effect and there are so many different things LLMs are really good at way before you get to even talk about code - gathering data by its semantic fingerprint; classification and routing where the boundary is fuzzy: is this a bug, a config error, or a support question; semantic diffing; reading stuff nobody will read: a 900-line helm file, a dependency tree, six months of changelogs, a vendor contract, etc.
If you have never dealt with cases where the input is messy and the matching criteria are impossible to write as rules, yes, of course it would feel like you don't even need LLMs. But pretty much any serious software does have plenty of those, and LLMs are incredibly good at that. Not only they are great - they are quite irreplaceable, in the sense that, yes, I won't die if I stop using the Internet, but any reason for choosing that path obviously must have some drastic rationale.