Anthropic's Claude 3 causes stir by seeming to realize when it was being tested

Researchers are human and as such tend to say weird things due to various psychological biases or issues. We should be careful not to take all their statements seriously. The most reasonable comment is quoted in the article:

Noah Giansiracusa, Bentley University math professor and frequent AI pundit, tweeted, "Omg are we seriously doing the whole Blake Lemoine Google LaMDA thing again, now with Anthropic's Claude?"

I don’t know much about how these models work, but does this mean that a lot of ‘smartness’ that they show in how they articulate the answers are just them ‘parroting’ the contractors who ‘role played ai’?