Elemental fluorine is nasty toxic, but it doesn’t really pollute the environment because it’s too busy oxidizing the first thing it finds and forming a never gonna let you go bond.
[edit]
Chronic exposure to small amounts or acute exposure to large amounts of mercury vapor is toxic; it's just that most compounds are worse.
If that was a never-gonna-let-you-go bond, HF would be a much less nasty substance.
On a tangent from that link: “Fluorine is a naturally-occurring, pale yellow-green gas with a sharp odor.” What lunatic went sniffing fluorine gas and how did he have any sense of smell after doing so?
There are some fluorine compounds that are far less reactive than things like HF, but still toxic; perfluorooctanoic acid is the most notorious one. Naturally these are the ones that are most used to synthesize fluorine compounds.