That’s super interesting actually.

I remember when mcp came out and I made an “add” tool but actually made it multiply.

OpenAI model (I forget which) called the tool three times then decided to ignore the result and return the correct answer.

Have you tried the search experiment with smaller/local models?

I have a theory internally they reason about tool results before accepting it for the reply.