Would "time away" be better do you think. I mean they are "off of work" as in not at their job.
If you took some time off to do something else equally taxing I would normally refer to it as time off, probably with a caveat in day to day situations, just like if they were on parental leave.
Thinking about it I probably wouldn't if someone was taking time away from work for reserves work (military, fire service, etc).