Depends on your browser. On chrome, just long-press on the image until the context menu pops-up with the alt text on it.

On iOS, long press on the image.

Try using https://m.xkcd.com/ and click the comic to toggle the alt text