I think you don't understand what a public good is. A public good is not something that is paid for by public funds. Here is a reference.
that is true only in the ideal situation of unlimited law enforcement resources and the type of crimes affecting everybody. In real situation going after specific criminals or crimes [esp. the ones affecting only specific subset of the society] leaves other crimes unpunished and laws unenforced. Thus any real law enforcement inplementation (not theory) demonstrates rivalness and excludability.