Take for example Einstein's Relativity of Simultaneity thought experiment. [1] This is akin to discussion about a bug among software developers. Like developers can agree that a bug exists after discussion alone, a software developer could read the thought experiment and conclude there's a bug in physics (i.e. the general consensus at the time). But physicists don't seem to work that way. In discussions with many super smart physics-minded people, I've found that most reject that thought experiment, nay any thought experiment, as having value. Only a mathematically-based argument suffices for them. Imagine showing software bugs only mathematically!
[1] In Einstein's words: http://www.bartleby.com/173/9.html