I don't think it's meant to literally suggest they just went for the summer. Rather "summer job" ("sommerjobb") is commonly used in Norway to mean a second job done while you have time off from your regular duties. The "summer" bit included because the term is most often used for students who have an extra long summer vacation and so has plenty of time to take a short term job.

At least in crop agriculture the work tends to be concentrated during planting and harvest so there is a big period in between with nothing to do for most of the people involved in planting and harvest. The labor for at least some of the Egyptian pyramids were idle farmers who had free-time available for hire, iirc.

These guys were soldiers, right? When there wasn't fighting to be done in the North, they headed south to look for work.