It's not so much my issues with Toggl. I actually think it's a very good little app.
I've just found it very annoying working on multiple projects at once and constantly stopping and starting the timer due to distractions, phone calls from different clients etc. It's very easy to forget to stop or start it and then I have to go back and manually edit all the time which is guess work anyway.
Working on reducing distractions and allocating blocks of time better would probably help but I've gone back to just using a simple spreadsheet now and rounding up hours throughout the day. It just feels less pressured and it's something less to worry about.