Athletic activity causes actual physical microdamage that takes rest time, nutrition, and hormones to repair. The elevated secretion of human growth hormone during REM sleep is a significant reason why athletes perform better on more sleep.
No such analogue exists for the human brain. Mental activities, at worst, deplete neurotransmitter levels. I have seen no evidence to suggest that massive dosages of sleep do a better job of restoring these levels than minimally adequate amounts.
The systems of the body are not neatly partitioned into modules.
I will absolutely concede that there is a threshold where sleep deprivation will negatively impact my (or anyone's) ability to perform, its just a question of what that threshold actually is.