You'd be hard pressed to convince me that someone who fought for their country in the military should not be preferred as a civil servant. It seems like a very strong qualification for these jobs.
This question obviously only applies to a subset of veterans. There's a civilian analog to many military jobs and the military's training and experience in those fields can stand on its own without preferential treatment for the most part.
A combat veteran may make a good post office manager or a good cop but it's probably better off for everyone if he's only given preferential treatment in the latter role.