I mean, we don't live in a representative democracy. Not in the sense you're describing.
(Assuming you, like me, live in the U.S.) Civil servants are not elected, are not directly accountable to elected officials, and are certainly not drawn from a representative cross section of the public. You may feel like they represent you, but that is (to be frank) little more than an expression of your own place in society. Civil servants form a discrete and powerful class, with their own interests, unions, and lobbying groups. Ignoring this is, optimistically, naive.