We can't have hard evidence. It's a SaaS and they own the code and the machine it runs on.
So it may be a widespread hallucination. But there's no evidence of that either.
So it may be a widespread hallucination. But there's no evidence of that either.
If they have a "hey we're being benchmarked" mode, which is not hard to imagine, avoiding tripping it is going to be annoying and difficult to prove.
I mean, it's not _that_ hard to determine most likely, and/or it's hard to be sure you didn't get found out by llm-assisted analysis on your traffic. It's not going to be a one-shot request and response it's going to be a whole bunch of them in an artificial way, by nature.
And then anything you found is single-use only if you're paranoid because even if they find out later, they have your benchmark now (because you sent it to them to use it even if you don't publish it).