It's a funny thing - manufacturers of products want to "stand out" from competition - so they use unique shaped containers for their products...
...the same is true for the container companies! The want to stand out from their competition - so they introduce new products (containers) with new shapes or sizes for that purpose.
While I am certain that some product manufacturers can dictate or have custom-shaped containers for their brand only, for many they just choose an off-the-shelf version and brand using labeling and/or color.
So - while going with standard-sized containers would be great for a whole host of reasons, for the product manufacturers - both for what is being contained, and the containers themselves - it presents a huge loss of profit and market share.
I'm not sure what we can do about this; it seems like something that would have to be dictated by government, because I doubt any of the other stakeholders (not even the consumers) will sway the decision toward standardized containers.
Soda cans (and for that matter, soup cans) are almost the only examples we have (ok - plastic milk jugs and paper milk cartons?), but as you have noticed, that's starting to fragment (it had already started with beer cans long before).