As did I (well, a worker on one) and to an extent you're right. It depends on when the morning milking time is "set". Is it 05:00 summer time, which then becomes 04:00 in winter? Or 05:00 in winter, which then become 06:00 in summer?
We generally operated on the second one.
Also, over the period of six months, times can be gradually changed by one hour quite easily. Moving the milking time 10 minutes here and there adds up quite quickly.
No matter how hard you try, sometimes milking times slip anyway. Broken vacuum pump? Milking's waiting while you fix it.
Anyway, I just used dairying as an example. I think the dark winter start problem would be similar for many outdoor manual labouring jobs.