Personally I think that the physical reality can't be described in a finite Theory of Everything (ToE) for the same reason as that there is no ToE for mathematics? See http://plus.maths.org/issue37/features/omega/index.html
"we can distinguish a world which can be explained by science from one that cannot."
"The sentence, to me, should be "we can distinguish a world which can be explained by [mathematics] from one that cannot."
"Mathematics" and "science" are not synonyms. But I agree with Leibnitz that, to paraphrase, "line explains the dots."
And also, I would like to note that; physicists use "theory of everything" to mean two things; and they exploit this meanings anarchy that they created: Even in the same sentence; by "theory of everything" a physicist may mean "a theory that will conform three famously incompatible physics theories" and "a theory that will explain the entire reality." These two definition are not the same.