That's not necessarily true. My own site is a Jekyll site. To host that on S3, I'd need to generate it first and upload the generated files as opposed to my source files. Now that's not really a big deal, but I do enjoy the convenience of only having to do a `git push` to deploy my site on Pages.
That being said, I notice times similar to another commenter above, around 1-2s usually. I don't think I've seen a five second load time.