If you work the agreed amount of hours and you provide the agreed amount of services/deliverables, does it matter?

As long as you're doing what you agreed in your contract, it doesn't matter. It's only a problem if the job performance suffers due to your startup.