You may not be so "out there" at all. There were two books that came out a few years ago, both basically lambasting the state of physics - especially string theory. One was titled
Not Even Wrong[1] and the other was titled
The Trouble With Physics[2]. As best as I can tell, of the two (and I'm afraid I can't remember which it was now) made the case that we might be completely wrong about the nature of time. The idea was that the reason we have had to resort to such (seemingly) byzantine theories as string theory, without seeming to make much progress on deciding
which string theory is correct (or if it's correct at all) is tied up in this misunderstanding of time. IOW, before we make any really fundamental breakthroughs in physics, we will have to re-conceptualize time itself.
Now that's a pretty bold claim, and I have no idea if it is correct or not. But it's a position held by at least one fairly well known and reputable physicist.
[1]: http://www.amazon.com/gp/product/0465092756
[2]: http://www.amazon.com/The-Trouble-With-Physics-Science/dp/06...