I finally found a todo system which worked for me, namely writing up every todo in a list. On Sundays I distribute the tasks to different days of my next week day.
If I don't get the task done for a weekday, it's completely fine. I just move it back into the long list, so it gets distributed next sunday.
If I notice a task has been cycled like that a few times, then I delete it, it wasn't important enough for me to overcome the guilt associated with the task.
I'm not sure if the system I came up with was what actually fixed my todo anxiety, but in case it was I figured I would share it.
What works for me now is writing my most important TODOs on a whiteboard and not erasing it until I complete it. The limited space forces me to prioritize. It's also obvious which items have been there the longest, either indicating lower priority or mental blockages. It helps that the ink is harder to erase the longer it sits there too :)
Identifying which "projects" you have lying around in your to-do list (to-do items that will require more than one atomic task to complete), and the weekly review process were particularly valuable takeaways.