I get stressed out when I don't know what I need to do. A combination of scheduling appointments and writing down everything I need to get done dramatically increases my productivity by removing the "oh shit, I know I have to do something but I can't remember what it is" moments.

I agree with Jeff about the fact you should know the top three things you need to do each day, but lower priority items can - and do - find a way to slip through the cracks without some kind of to do list.