Hacker News Reader

Does this method build assumptions about the distribution of the evaluation dataset and make the bit-level reproduction of an evaluation unlikely?

Well, this method is based on the assumption that embeddings can accurately represent the texts and their structural relations are preserved.

So long as you have all the random seeds fixed, I think reproduction should be straight forward.