There’s basically no reason they should have a say at all about what someone else wants to do with their own property.
It’s also not like there’s nowhere else on earth to put a school campus either. They’d hardly be the first school with multiple campuses that are separated if they didn’t insist on building it where they currently are.
Telling people what they could use their land for (within building code) is only legal in the US because the SCOTUS said it was legal to in order to keep Black people out of neighborhoods. It should be unconstitutional, and would be if not for racism!
The idea is to put pressure on the locals - if you don't shape up we will move. A significant number probably like the college nearby overall.