> So you end up in a situation where you have free time, but don't really have it, and you can't afford to focus on anything in case something more important pops up.

Nailed it. I have hobbies, but most occur on the weekend, or in the evening once or twice a week. When it's a slow week the empty time during the day hits me like a freight train- I've found myself slipping into a depression a few times.