There are more than enough years past the first one for the "father to build impressive things", though.
Also, the first 18 months establish a good deal of a person's attachment security, connection to parents, and emotional intelligence. The brain continues to be flexible for some time, but gradually becomes harder to rewire.
> Then from 4 or 5 upwards they're at school a lot and starting to lose interest in what their parents want to do
I had to disagree with this. Thought this might be a European experience, in the US there probably isn't a lot of time to interact with the kids during the typical work week.
I would say that interest in what we do has stayed steady if not increased after about 3 years old because now they understand more things.
At this rate I expect the "losing interest part" to start during the tween (8-12) or teenage years.
So long as it is safe, a kid even at 1 years old should be tagging along with the adults doing things. Like building, working out in the yard, helping friends with projects, whatever.
It is very settled science that brain connections happen rapidly in these years. Bubble wrapping your children is not the way to go in my opinion.
Everyone is going to have a different take on this, but for me the choice is immediately obvious. Kids should fit into your life (within reason), not have you change to revolve around theirs.
That said - I highly doubt a 1 year old did much more than watch and have short visits while being cared for by someone not doing the construction work. It’s a great bonding experience even with limited direct involvement.