3000 matches should be more than enough for an Elo-style ranking system to converge. But the notebook[0] linked to from the site includes a section with an equal number of matches being sampled for each pairwise comparison, and the results are pretty much the same.