The wallwart for the router is going to be a dead simple design that doesn't in the least bit care about how dirty the sine wave is. It goes into a step-down transformer (12VAC usually), into a bridge rectifier (now 12VDC), add some decoupling caps to smooth out the 12V. Inside the router, there will be LDOs that bring that voltage down to something usable along with more decoupling caps to filter the input and output of the LDO. LDOs have high ripple rejection especially at low frequencies so they can take on pretty dirty voltages.
Unless that little dip on the supply is constantly happening, I highly doubt that's causing any of the issues. That dip could be from in-rush current or from the router powering up, both of which would be happening before any data transfer. It means nothing without any sort of voltage or time scale.