I would think so.
> Do regex libraries for different languages in-fact always implement the same algorithms?
I don't know. But I do know that when something is presented as an answer, it's good to know what the question is.
> suggests you are looking for something that tells us about "the relative speed" of those languages?
I can think of a couple of ways the two can compare. One is the relative speeds (and other metrics) of programs of a similar nature, written simple idiomatic code. This gives a general "feel" for what kind of workloads different languages may be suitable for. Another is comparing the performance of the compiler/interpreter on some microbenchmark of the same algorithms, preferably written by people who coordinate, and preferably without mixing too many variables (e.g. measuring parallel scheduling separately and regex separately). Either one of these is of some interest.
The benchmark game does neither. If there is some idea behind the site's comparisons, it's certainly keeping it a mystery.