IMO, the countries passport holders can travel should be weighted by: 1) their size and economic strength 2) attractiveness as a tourist destination.

For example, right now, Estonian and Polish passports have the same score. However, Estonians do not need visa to travel to the USA, while the Polish need it. This is arguably a significant difference.

How exactly countries should be weighted is a separate debate.