I think the issue with almonds is that by growing them in massive monoculture you create bee deserts for the majority of the year when the almonds aren't flowering. Once the flowers disappear, any remaining bees will starve until the next year. So this criticism could be leveled essentially against almost any large scale monoculture crop.
I'd be interested to know why almonds (and avocados) are usually highlighted here. Are other flowering crops typically grown in more diverse environments?