I would test if there is even the slightest concern of contamination. Eating a few garden veggies here and there likely won't have much of an impact on an adult. If you're regularly supplementing your meals and/or feeding children, you may want to be more cautious.
At my current house on a busier, older street, I wouldn't consider eating anything grown in the soil. Most near me seem to use raised beds or the community garden.
Because this is something we have some control over. Like trans-fats, no amount of lead is healthy (it can take the place of Zn and other elements in our bodies, and is especially harmful for young animals as they make foundational cells). If I want to grow food for my family (which includes a young human animal), I want the soil to be healthy enough (whatever we decide that is). If the soil that is present isn't healthy enough, I can import better soil (and contribute to it over time with compost from our kitchen).
I agree with you about the importance of not adding stressors, with the specification: don't add stress about things I can't control. For things we collectively have some control over I lobby my representatives, make comments on HN, and otherwise try to let it go.
> make comments on HN
This got a good laugh out of me. Good job.