Whilst it's true that these concentrate in carnivores this is not a reason why we eat mostly herbivores.
Herbivores were plentiful and less dangerous to hunt. Then, herbivores are much easier and more economical to breed (you would need to breed herbivores first if you wanted to breed carnivores...)
You'll note that these decisions were made long before people even knew about heavy metals or had pesticides.