I would imagine the argument for concern (not saying I agree with it) goes like this:
If Amazon Whole Foods is freed from the need to be profitable (like the rest of Amazon), and has the access to all of Amazon's other infrastructure, it will have a powerful competitive advantage against all other major grocery chains. Even if Amazon Whole Foods continues to focus on the upper end of the grocery market (which they may not), that will still make it even harder for other chains to compete in those markets, which I assume are more profitable than markets full of poor people.
If a given chain is only able to compete in the markets with the least purchasing power, that could have the effect of causing those chains to go out of business, which would in turn mean store closures in the poorest places. If I'm in charge of Ralph's as they're getting outcompeted in wealthier urban areas with high rents, I have to make up that profit by closing the least profitable stores, right?
Also, why assume that Amazon will keep Whole Foods' pricing or even brand alignment? Maybe they are simply buying a massive infrastructure for Amazon Fresh or whatever, and they intend to compete further down the line? I don't think there is any reason to assume that Amazon Whole Foods will have the same price range as Whole Foods.
By definition, there is no great business to be had selling groceries in poor areas. That's why there are food deserts now. If Amazon drives down margins and helps consolidate the grocery market, there may be even less likelihood that stores in poor areas will open or stay open.
Edit: read the actual letter, which makes very direct points about the potential pitfalls and merely asks for them to be studied (not for the merger to necessarily be blocked):
http://www.ufcw.org/wp-content/blogs.dir/61/files/2017/07/Lt...