In the end, it comes down to the optics.
The (incredible) tiny sensors in smartphones aren't big enough to get past a certain fidelity. Combined with very small working ranges on the lenses and the limits are pretty hard. Hence the three lens set-ups now on the newer models.
Granted, the optics here are just the coolest things. 'Digital' lenses are some very slick stuff and the engineering is the best you can get.
But in the end, you just need longer working distances and bigger lenses. The optical physics are just the laws you have to deal with, same as Newton had to. No amount of engineering can get past them.
One thing they could do is to put a 'periscope'-like mount on the side, have the sensors slide up and down inside, and slot in a few lenses based on the optics. But the mechanics and likely slow speed of such a device would wear-out very fast and would be very susceptible to drops and dings.