Ask HN: Fear of ToDo lists

I'm far from well-organised person, who remembers everything. To cope with this, I use a bunch of ToDo lists. However, I noticed that I'm afraid to look at my ToDo list. I start doing all kinds of small things just to get away from looking at (and following) ToDo lists. And that destroys my productivity even more than not having ToDo list at all. Then I do at least do what I remember :)

Anyone having similar problems? Any idea how to fix that?

P.S. I tried both paper and digital lists, various paradigms (GTD etc), tons of software.. No luck so far.