TDD is best when you're writing code that talks to other code. So APIs, database models, etc. Pure functions, and code that has dependencies you can inject and mock. You should never abandon TDD in situations like this.

It's true that it's harder to write TDD for code with side effects or that draws UI. It doesn't really make sense to use TDD for this.

You shouldn't conflate the two. Also, "always pass the majority of tests" is a trap. You should always pass all the tests.

Source: I've been managing and working in automated testing and continuous integration systems for 8 years, dating back to before the term was coined. I was the manager of the system, at IMVU, that coined the term "continuous integration". I've also worked on testing at Sauce Labs and Google.