I believe this happens because language design involves a lot of trade-offs, so every language incorporates these trade-offs.
When one is comfortable in many kinds of languages, one is in a special position to see the trade-offs in the design of the language one is currently programming it.
The way I see it is slightly different then the way you describe. Say I'm programming in Prolog, for instance. In the beginning, I'll be like "Wow, such expressiveness". But inevitably I'll do something that is outside the scope of Prolog and it will be as slow as hell. That's when I think to myself: "if I could just fix this little part... I miss C". So when I'm back programming in C I'll think "now my code is fast", but then inevitably: "if I could just find a way to do this without so much repetition".
In the end, it comes down as a "right tool for the job" thing. One possible ideal would be do the things Prolog is good at in Prolog, the things C is good at in C (and also throw in some Python, Erlang, Lisp, etc). Except most of the time this is infeasible. Working with FFIs suck. Sometimes there are no FFIs, sometimes there are many incompatible ones, and the dream of calling any language from any language is just further and further apart. And don't get me started on the pain that it is to have multiple runtimes with slightly different semantics...
Even if it were possible, a developer would have to learn all these little languages enough to be working on them comfortably, and we all know this is not going to happen. So hardly anybody does this, as hiring someone for a polyglot project is a complete nightmare.