Has no bearing on whether it's right or wrong.
Why isn't it? Those agreements don't restrict or infringe on the freedom of others to form different agreements, or avoid forming them.
> What about colluding on prices?
Same exact situation. Price agreements don't restrict the freedom of other parties to set prices contrary to that agreement.
> Aren't these all the same infringement?
Without getting into specifics... Yes.
> The truth is sometimes society has to take away the rights of the few for the sake of the many.
I truly commend you for your honesty about what these laws are.
> The fact is rules like this almost certainly make our economy more productive as a whole.
I don't see where that's been demonstrated here. They certainly aren't productive for the firms who are punished for, or prevented from, forming such agreements. I would argue that the fact that firms do form such agreements when free to do so, means that they are productive for the economy as a whole by definition.
By allowing those firms to make more money. They are the economy too. The economy does not exist for one persons or class of persons benefit.
> This was done to keep salaries low, so a handful of people could make more money.
People cooperated to reduce their expenses without restricting the right of anyone else to cooperate. Just because people make money doesn't make it wrong. Conversely, a handful of people (the defendants) colluded to make more money by keeping salaries high, except they did it by restricting the rights of everyone else.
It seems more reasonable that each human in a group will try to maximize their own economic outcomes as well as to assist the other humans that they personally care about in maximizing their economic outcomes. In the modern world, it's often expected that humans are able to extrapolate that the people that they don't know are similarly worthwhile human beings. So while it's generally considered acceptable to try to maximize things for themselves, doing so while knowingly making other humans worse off is seen as morally detestable.
It's important to remember that humans generally care much less about abstract ideals than they do about themselves, the people they care about, and their social standing. The "economy" as it exists in reality does in fact exist for the benefit of one class of persons: those persons who are willing to tolerate it and have the power to change it. When this real-life economy is no longer tolerated, it is changed, in ways like antitrust legislation, class action lawsuits against companies that participate in wage collusion schemes, and the French Revolution.
Also, please remember you can't look at a statement in a vacuum. "Why isn't it? Those agreements don't restrict or infringe on the freedom of others to form different agreements, or avoid forming them." Perhaps not technically, but realistically they absolutely do. If we had a million worlds with a trillion firms and easy travel between them then such actions would automatically correct, but we don't and thus people do not realistically have the option to turn them down. The whole point of capitalism is to reward and penalize decisions. It kind of breaks down (predictably) when one side has enough power to prevent the downside of a bad decision. Remember, free speech doesn't mean you can run into a theater and yell fire.