Slips of paper, write 1-20 (or whatever the die faces are) on them, shuffle them up, have people pick one. There's your randomness.

You could even insure an even distribution and have like 1-6 in there 4 or 5 times (if it's a d6), and don't return the slips back to the pile until the pile is depleted.

Or you can do what I've done for testing pen and paper game designs where a die wasn't handy, and do the above and write the series of results down in a list on a separate paper, and then refer to the next number in the list when you're playing.

I even did it where I put the numbers into several 4 by 4 square grids, and would simulate different games by picking the next number by from the grid but going from different directions (left, right, up, down, etc).