You mean like a human teacher?
Of course, I agree with your point, but worth remembering that teachers can and do (knowingly or unknowingly) give incorrect information to their students.
I think one big difference with ChatGPT etc. is, as far as I've seen, it will pretty much always give a confident sounding answer (unless it's on a banned topic) rather than saying something like "I don't know", which a (good) human teacher should do rather than just making something up.