I think a big factor is recovery time. When you’re young you can do some extreme activity and still be well rested the next day. Recovery time even for fit people just gets a bit slower when older. But if you’re not pushing yourself then you can usually recover fully by the next day.