It's one thing if you know you are using an LLM tool, but when using a search engine, people won't expect this.
Honestly, I would not be surprised if they didn't think it was. But thanks their "Code Red" they seem determined to get as much out the door as possible to avoid someone else doing it first.
> I don't think you can reasonable expect all users to realize it is low quality nonsense, when it is presented at the top of the results.
And that is the problem, we are training a large number of non technical people that these tools are supposedly reliable. When the problems with them confidently giving very wrong results are well understood. But we just don't talk about it anymore outside of certain circles.
If we did keep talking about it, all the AI hype would die off or at least he seriously hampered.
This is why you can ask it a simple physics question for example, and the default might be to give a wrong answer based on a popular misconception. But tell it that it should answer as Richard Feynman, and it can give the correct answer. It "knows" that the first answer was incorrect, but it's still a more appropriate/likely answer given the default scenario, where it effectively assumes it is supposed to act as a random internet stranger when it answers.