> It would make a lot of sense to go back to a smaller more integrated union with the countries which want it and leave the ones which only want the single market be part of a less integrated union.

That option does exist, albeit in a bit of a roundabout way. Nothing is stopping a member country from leaving the EU and joining EFTA and the EEA afterwards. The fact that no country is seriously contemplating this is telling.