Fundamentally a perfectly rational world is unrealistic. Humans do behave irrationally, even insanely. Humans also behave sub-optimally because they are trapped in systems full of crazy perverse incentives. The world itself is full of paradoxes and chaotic feedback loops; the best intentions rationally and logically executed can and do lead to perverse outcomes that would not and maybe even could not be forseen.
Sometimes a rational investigation terminates with a big WTF. People see UFOS or Bigfoot all the time, and they're not all nuts. What are they seeing? Not enough information. The universe doesn't present you with all the evidence so you can tie a neat bow around it. It presents you with a dirty noisy incomplete data set so you can ponder it forever. (See also: bounded rationality. Bayesianism deals with this problem by defining truth as a float instead of a boolean, which I kind of like. Bigfoot might 0.0137384 exist.)
The universe gives us Fermi paradoxes and dark energy and Godel's incompleteness theorem.
That being said, I do like the aversion to the idiot plot. It's a big pet peeve of mine. I also like stories where all the characters have depth. To me that's just good characterization, something often lacking in genre fiction. Even if the villain (or the hero) is mad or irrational, there is some depth to it and they have their reasons however twisted these might be. But a lot of that is just good writing.
I also like the aversion to the deus ex machina, which is also lazy writing. If there is some kind of super-thing that intervenes, it should too be developed as a character with depth.