Physical Education, Art, Spanish, Music, History, Science, etc are scheduled around those accordingly.
The main reasoning is simply that math and English are the two subjects that build upon each other the most from grade level to grade level and because of that, they are considered the "core".