The best balance I’ve seen involves centralizing a small number of essential services, ideally ones with lots of compliance and security complexity. Manage that well in one place, then let the departments use that infrastructure to meet their unique needs.
Face it - students have higher expectations now, professors also have higher expectations. This requires administrative staff to run. Back in the day school budgets were lower, but even when I went to college in 2005 they didn't have campus-wide wifi in every classroom. We had one professor who taught with powerpoint. Today, every student has a laptop in class.
Maintaining a modern campus takes a big IT department and centralizing it is the least wasteful way to do things.
There are things better done by a central IT team like university level WiFi, but you can make that smaller and also have departmental teams for things where more agility is needed. If the people are competent it's really great.
And yes 3-4 people only makes sense because it was a large department, but smaller departments with similar mandates, for example English/Literature and History, just have a shared departmental IT between them.