There are many strict orderings that can be used, but we just need one.
Since each number is in a physical draw in a physical room, and no two draws can occupy the exact same space, there is a strict ordering of the drawers given by each drawers distance from a fixed point in the room (say a specific corner agreed on beforehand) in each of the three orthogonal dimensions sideways (x), backwards (y), and up (z).
We say a drawer A is before a drawer B if A.x < B.x; or, if A.x = B.x then if A.y < B.y; or, if A.x = B.x and A.y = B.y then if A.z < B.z
If A.x = B.x and A.y = B.y and A.z = B.z then A = B.
This is a strict ordering that will number the drawers from 1-100, and can be determined beforehand.
Now the prisoners take that order, and shuffle it randomly, without telling the warden what they are doing.