Accepting your analogy at face value, it's still not obvious to me that fixing a specific seed doesn't change things.

Take a recurrent PRNG for example. A randomly seeded recurrent function usually has degenerate cycles in its state space. For some functions, this might even describe the majority of the state space. This is why so many non-cryptographic PRNGs are max-cycle, so a different starting point is just further along the same trajectory.

I don't think LLMs have quite the same failure mode here, but recurrence + high dimensional spaces triggers my "here be dragons" sense.