Jekyll is notoriously slow even for a very small website, unfortunately.
All of this is without cache, and with incremental builds disabled; it will obviously be faster with cache and incremental builds.
I don't know where this "Jekyll is notoriously slow" comes from; maybe it was (very) old versions, or maybe it's some plugin people use, or maybe something else, but I've used Jekyll for a number of websites over the last ten years, and I never really had any performance issues.
Certainly today, I think it's unfair to say that Jekyll is "slow". Yes, Go is faster, which should hardly surprise anyone, but for most purposes Jekyll should be more than fast enough, and you'll hardly notice the difference with something faster like Hugo or GoJekyll.