This can be refuted by observing that the refractive index isn't proportional to density, but is accurately predicted by charge mobility. Also, how refractive index changes with wavelength is related to the resonant frequencies of electrons.
Yeah, I was thinking about it and also lasers wouldn't work if it was 100% mini-Shapiro delays. I was thinking that maybe there's a small GR component to refraction, but the experiment would be tricky. You'd need to pin the electrons in the refractor down in a strong magentic field and then shoot some high energy photons at it, probably at least X-rays, and see if they still refract. (The high energy photons would be necessary to bring their wavelength closer to the diameter of the nucleus).