It's an unusual idiom to be sure. We talk about lots of fruits and foods every day without such qualifiers. Are we to assume OP meant possibly-stale bread because it wasn't explicit?
Plenty of people will only eat in-season tomatoes. I'm not one of them but I certainly understand it. The real thing is so different than the tomato-shaped-objects you get in the supermarket in January that it makes sense to treat them as a different food item. Honestly, I've been using canned tomatoes outside the season more and more, because they retain more flavor than the "fresh" ones that come from a hothouse or 3k miles away.