This is a fascinating phenomenon that I've encountered before. It seems that some multilingual people cannot easily tell which language they are speaking and/or switch languages during conversation.
A friend who is learning Dutch once complained to me that Dutch speakers will switch to English the moment they hear you are struggling with the language. I suggested she should explicitly ask people to keep the conversation in Dutch.
In one instance, she did that, and the (native Dutch-speaking) person she was talking to agreed to her request, and then proceeded in English anyway! She asked that person multiple times, and they just kept speaking English without addressing her request. Baffling.