Fish do indeed often contain heavy metals (mercury, lead, cadmium, arsenic, copper), PCBs, dioxins, and furans, as well as microplastics.
Young dolphins are dying because the mother's milk may be poisonous for the same reason.
https://phys.org/news/2018-12-toxins-mother-young-european-d...
Banned toxins passed from mother to young in European dolphins
https://www.poisoningchildren.com/blog/2014/02/05/bioaccumul...
Bioaccumulation and Dead Baby Dolphins
"Because female dolphins off-load most of their toxic exposure to their first-born calf, their levels after sexual maturity are lower than males; but a very high percentage of first-borns die"
"It is not astonishing that native women in the Arctic Circle, who eat high levels of marine mammals, pass these chemicals on to their babies with dire effects"
"Childbirth and breastfeeding are some of the few ways the body can rid itself of persistent chemicals. It is usually still best to breastfeed, but children who are breastfed continue to inherit the mother’s exposures, as shown in a study of testicular cancer in Denmark and Finland"
Btw, human and cow milk has similar problems.