The structure of the photon (here meaning the fact that it can fluctuate into an electron positron pair) is what leads to the speed of light being c and not something even larger. So your intuition is somewhat correct, except that the effect is "already taken into account" when we first learned what the speed of light was. Also, a photon can do this fluctuating even if it doesnt have a lot of energy, since the electron positron pair can be virtual - i.e. they have almost no mass and live only for an instant before annihilating back into a photon