Like: if I try and kill my neighbor, but my rifle jams as I shoot, there's no victim, right? Or: if my friends and I get together to plan the burning of a synagogue, there's no victim. Or: if I offer to pay you to kill that pesky neighbour, there's no victim (until you go through with it). Or: if I offer to buy goods off you that have you stole, what I am doing has no victim (the goods were already stolen).
The notion that "victimless" crime is somehow in a category of crimes that are lesser or "not proper" crimes is unjustified.
We have above: attempted murder, conspiracy to commit arson, soliciting a murder, receipt of stolen goods - most of which I think people would regard as serious crimes.
Societies criminalize things that are considered "bad", victim or no - and that reflects the contemporary moral viewpoint of the society. Sodomy (of the consensual variety) was a crime many places, and now it generally isn't.