Thinking about photons is bad for you, especially if you are doing quantum-optics. It is almost always better to think about quantum fields, and in this case the quantumness adds nothing to the physics.
The "photon" is just a quantum of excitation of the optical medium (in this case the EM compled with the electromechanical state of matter in it). These exictations obey wave equations, just as classical waves do, and hence Snell's law.
It's true that you can also (almost) think of photons as point-particles travelling over every possible path in time-space and then do infinite-dimensional path-integrals. That point of view is important for a physicist to understand, but it is not needed to explain refraction.