Are you sure you are cultivating his attention span, or were you both just born like that, and he is getting older and more mature?
The hours you spend with your children (5 hours a day would be really difficult with 3 kids) during a couple of years in their childhood is not that important to their adult personality. It’s a combination of traits from birth, which class mates they get, the teachers, if they get sick, infections, siblings, grand parents, media, culture...
Personally I enjoy spending time with my kids, so both me and my wife work shorter hours, but it’s to do stuff we all enjoy right now, not to cultivate this or that personality trait.
Don’t be a terrible parent, and your kids will turn out whoever they were meant to be. And do the things you and your kids enjoy.