The difference is that "toxic food" directly implies food that is poisonous.

"Toxic masulinity" has no obvious or direct meaning. It's a made up term. It's marketing. It itself conveys no useful meaning other than to associate masculitity with toxicicity.

To tie such a negative term to any other social/gender group would be called out as the biggotry it is.