Farming today is a seven day, every day work week. There is no day off with livestock, no matter what century, and what season. If the sun sets, or the weather is bad outside, there are plenty of work to be done indoors.
I would presume it was even harder without all the automation and technology in the Middle Ages. Maybe they did not labour for the employer all the time, but all the 'free time' was spent labouring for sustenance, and other life's maintenance.
If I recall correctly, the 8 hour day, 40 hour week, and five day work-week are all 19th century trends.