Well, the incentives around research are a bit weird. I'm usually all-in on free market capitalism, but I have to admit that historically government sponsorship has a pretty great track record for research.
For your specific question though, the answer could easily be because someone paid them up front to develop a new cancer drug. There is no question that companies that synthesise drugs are incentivised to research new products and deploy them as a stream of new and more effective drugs entering the market will cause it to grow in size and hence increase their profits. It is a question of debate about whether the current American IP system or natural incentives would give a better result overall.
> And what about nuclear weapons? Are we supposed to think that it's better if every tin hat despot could just download plans for an ICBM?
Splitting hairs, but it is debatable if building ICBMs is 'private knowledge' for the grandparent comment. That sort of tech is probably more of a grey-area public-knowledge style. It certainly isn't protected by the IP system, because then someone would be selling ICBMs to tin hat dictators. It isn't used in products that are sold to private entities either (I hope!).