You would certainly see an apparent slowdown due to phase shift if you consider a continuous wave that spans the entire medium, yes for sure and so this explanation and model does have merit, I don't want to dismiss it entirely.
But this model won't be useful for understanding pulses of light through a medium. For example if I emit a very short pulse of light through a medium and measure how long it takes the head of the pulse to traverse the medium, using this model won't work. A phase shift won't move the head of a pulse of light backwards, it will just adjust the amplitude of the head as it propagates through the medium.
You still need a way to explain how it is that the head of the pulse actually takes longer to travel through the medium and that's not something a phase shift can explain.