Both, really. From all the people I know, there's only one person who cannot walk from his place to a nearby bakery. And in that case it was a conscious decision to "become isolated" and buy a house in an unlikely place. Walking every morning to buy bread for breakfast is a deeply entrenched cultural thing. I cannot really imagine living somewhere where this is not possible. Most of urbanized places are indeed downtown.
Sadly, bakeries in America are rare. Your daily bread comes from the supermarket. Much of it isn't baked on site, and it's designed to keep for several days. We don't have a tradition of daily fresh bread, which is too bad for a lot of reasons.
But Americans have plenty of food options so don't weep too much for us. If you want decent bread, it's available in many places, including many of those supermarkets and chain bakeries like Panera. It's not magnificent bread, but it's good. Magnificent bread is also available; it's just not a priority.
Americans do have a pretty dysfunctional relationship with food, and we'd be a lot healthier if we fixed it. But the mall isn't really part of that. That's more about the supermarket than the mall, and those are separate niches.