I'm old enough to remember when this wasn't the case. Compilers back in the 80s and even early 90s had bugs. Heck, even Intel processors had bugs!

The bigger problem though with treating LLMs as a compiler is they're non-deterministic. The same prompt can generate two different outputs at two different times. One may be correct, the other may have subtle bugs. In the end, we should be focusing on what we should have been focusing on all along: what evidence do I have that this code is correct?