I don’t know of a study that estimates the maximum number of productive hours over a varied workload such as running a startup. There clearly is a limit, but it’s high — maybe 90 hrs/wk for motivated people.
In practice, it’s the neglecting of other things like relationships and exercise that kicks in before mental energy limits.