From my perspective, anything regularly periodic in nature is used to track time: the Sun (years, seasons, days, day/dusk) and Moon (months, weeks, King tides). Consider even the starfield (astrology: years, decades+).

Four seasons makes sense in all of these contexts, and I hypothesize that there are a lot of patterns in nature that are well bucketed into 4 sequential, equal seasons. This is impossible for me to demonstrate, unfortunately. Also, division by by 4 and 12, a year into four seasons is like a month into four weeks. Weeks seem to me to be among our most subjective fundamental time unit.

I see weeks as being the most arbitrary. I always like to imagine what life would be like if weeks were 3 days long. Or 11. I just love this topic.