In my opinion, the primary advantage Github Pages has over a CMS like Wordpress is simplicity and lack of cruft. WordPress is an extremely powerful tool, but 95% of blogs, mine included, don't need the features that WP provides - and more importantly, the added code and serving overhead that comes with it.
When I've done WordPress development in the past, a lot of customization can be done on the theme level but sometimes you need to adjust the model around which a page is served and it can become confusing and complex with WP's expansive codebase. With Jekyll, it's easy to see every file and how those files combine to create my site pages. It's also easy to serve locally and I can rest assured that Github will build my site in the same manner as it's being built locally.
Finally, I would assert that I have much finer control over my source using Github Pages compared to WordPress - my site, in its native form, is literally a repository which I can move and rebuild on any number of platforms without any backend configuration. And that same repository is automatically open-sourced and built on Github without my having to maintain a build system. I never have to worry about migrating a database, I never have to worry about plugin updates breaking my page, and I never need to worry about PHP vulnerabilities compromising my data. When I look at Github Pages, I feel like GH really managed to abide by KISS principles, saying "what is the MVP for a blogging platform" and providing that in a clean & performant manner.