Not sure if the most time consuming, but what really pisses me off is how much time I have to spend emailing/calling/etc clients to discuss project details, money, etc that I'd rather spend writing code. That's probably the worst part of freelancing
Have you tried setting up a contact form on your website with a survey that collects all the informations beforehand for you?
That would be good for first contact only... And believe me, non technical people are unable to properly explain what they want. Going by what they ask without giving constant feedback means you'll end up with something that has little to do with what they originally wanted