If it is also the case that patent holders purposely withold the very trade secrets that is the bargaining chips that gets you a patent according to the spirit of the law, then if it isn't a racket, it's still immoral and needs to stop being allowed.
So many patents these days are completely devoid of the secrets they are supposed to both protect, and publish. Little is thus gained for the public good, and much is lost.
It strikes me as odd that patents has such a strong support in the US, as generally there seems to be the case that the government isn't wanted where it's not absolutely needed, and I find it hard to see how one could argue that patents are so integral to society that they should fall in that category according to the metrics that seems to be used for other commonly government controlled areas.