Why stop there? We could do long division before the calculator and hand write before the typewriter.
Because arithmetic itself, by definition, is.
Human language is not. Which is why being able to talk to our computers in natural language (and have them understand us and talk back) now is nothing short of science fiction come true.
LLMs are wrong infinitely more than calculators, because calculators are never wrong (unless they're broken).
If you input "1 + 3" into your calculator and get "4", but you actually wanted to know the answer to "1 + 2", the calculator wasn't "wrong". It gave you the answer to the question you asked.
Now you might say "but that's what's happening with LLMs too! It gave you the wrong answer because you didn't ask the question right!" But an LLM isn't an all-seeing oracle. It can only interpolate between points in its training data. And if the correct answer isn't in its training data, then no amount of "using it with care" will produce the correct answer.
They also don't use it at all anymore, they barely even care about your search query.
Google is successful, however, because they innovated once, and got enough money together as a result to buy Doubleclick. Combining their one innovation with the ad company they bought enabled them to buy other companies.
I was allowed to use a calculator from middle school onward, when we were being tested on algebra and beyond and not arithmetic.
Some schools have ridiculous policies. Some don’t. Ymmv. I don’t think that’s changed from when I was in school.