Current LLMs can’t do “random”.

There are 2 sources of randomness:

1) the random seed during inference

2) the non-determinism of GPu execution (caused due to performance optimizations)

This is one of those things that humans do trivially but computers struggle with.

If you want randomization, ask it the same question multiple times with a different random seed.