You should. Feel like a conspiracy theorist when saying things like this.
Users are not reliable or consistent model evaluators. Users adapt to models - the moment they get a model that performs better, their expectations rise, their tasks get harder and their prompts get shorter.
"They made the model worse" is PEBKAC in 9 cases out of 10.