Yes -- we observed this behavior on both the open-source open-weights 671B model as well as the DeepSeek web app.
Then I have mixed signals about this.
This is going to be very hard to trust anything about it anymore, unless running the 671B locally an my own systems.
Happy to send you the dataset if you'd like! Please reach out to our email linked in the post.