We would never have done it the first time if not for military competition with the USSR.
(Though there is potential for another military race there too - the first country to put a deorbiting system on a near-Earth rock wins.)
I agree with you when we get to the offensive use of weapons. Though the US is only one of a number of players in that field.
Military competitions is what gave us microwave tech. Doesn't mean that if world peace suddenly happened, we would stop producing microwave ovens.