For swimming at a college level, we got up at about 4:30, swam 5-7, lifted 7-8, then classes, then swam 4-6, then stretches/bodyweight exercises.
This was while classes were in session.
During the winter training trip, we would do the same, but also swam sprints in the middle of the day.
Sometimes other teams, including the football team, would try to use the pool for off-sport workouts. We would crush them. But when we ran, they would not outrun us.
And that's just one sport I have personal experience with. Please do not discount what other athletes are doing--any sport is an intentionally competitive environment, everyone is looking to gain advantage: that push to go harder is present and is not exclusive to football.