In my experience, small agencies (up to 20 people) usually cannot afford or justify giving substantial resources to web hosting. My agency is now fairly small (five people), and web hosting is so complex and fast-growing, that we decided not to deal with it. We found a good partner a few years ago and we refer some of our clients to them. Some of our clients do pay hosting to us, but it's more of an application hosting type of service (they rent apps from us and pay annually). We have our own dedicated server on which we only allow websites we've developed ourselves. This is a legacy thing: we've been doing that before and we're still billing some of our past clients for app hosting, but stopped selling any new web hosting.
If I was starting an agency from scratch today, I would steer away from web hosting and let the specialized partner handle that for my clients. There are simply too many things that could go wrong if a small agency does not have the necessary expertise. One wrong step and you can lose a six-figure client over a $10 domain. And since there are too many things to keep up with in your core business (web development), your time is better spent learning about your core business, not about the web hosting business.