He may think he doesn't have an 'HR department', but whoever holds the pen on his holiday policy, equipment policy, training & development policy, maternity & family friendly rights policy, expenses policy - well, there's his HR department.
When it all goes through a single person, there’s less of a balancing force there.
Both obviously can be abused.
Form a managers perspective I see HR essentially as your legal counsel, if the company wants to delegate that to the legal department sure but they’ll be likely overpaying those who’ll be tasked with dealing with HR issues all day long.