By default on a Mac, "smart quotes" is enabled that turns "" into “”. I don't think that's a sign of an LLM. I believe Microsoft Word on all platforms does this too.