It took me a long time to realize this, but it finally clicked for me fairly recently. Record everything you do for a day, figure out which are of low or no value, and figure out how to get rid of them, create systems to reduce the time you spend on them, or delegate them. The most time-consuming things often can't be totally cut out, but they can be systemized or delegated. In the end, only you control your time (and if you don't, you have a bigger problem).
Edit: If anyone is interested in the resources I've found helpful in this realization and the implementation of taking actions to take back control of my own time, feel free to send me an email (in profile).