Hacker News Reader

Comment by gsinclair | Hacker News Reader

FWIW, I’d bet some coin that current CharGPT would provide a genuine pseudo-random number on request. It now has the ability to recognise when answering the prompt requires a standard algorithm instead of ordinary sentence generation.

I found this out recently when I asked it to generate some anagrams for me. Then I asked how it did it.

View on HN

In the context of gambling, random numbers or prngs can't have any unknown possible frequencies or tendencies. There can't be any doubt as to whether the number could be distorted or hallucinated. A pseudo random number that might or might not be from some algorithm picked by GPT is wayyyy worse than a mersenne twister, because it's open to distortion. Worse, there's no paper trail. MT is not the way to run a casino, or at least not sufficient, but at least you know it's pseudorandom based on a seed. With GPT you cannot know that, which means it doesn't fit the definition of "random" in any way. And if you find yourself watching a player getting blackjack 10 times in a row for $2k per bet, you will ask yourself where those numbers came from.

I think you're missing the point. Current incarnations of GPT can do tool calling, why shouldn't they be able to call on a CSPRNG if they think they'll need a genuinely random number?

It's not that they can't do that. It's how do you know for sure that they do it every single time?

I asked chatgpt for a ramdom number between 1 and 10. It answered 7, then i asked for anpther, and it answered 3.

I asked Gemini and it gave me 8 and then I asked again and it gave me 9.

exactly the same here, 7 first then 3.

I don't think LLMs can reliably explain how they do things.

Depends on how you ask it if course. Chatgpt:

  Output the random generation of a number between 1 and 10, 100 times

..chatgpt would only provide me with a python script and then offered to add scrolling numbers and colourful digits.

Tried again in a new session with:

  Generate a random number between 1 and 10, 100 times. Output only

4, 9, 1, 6, 10, 3, 2, 5, 7, 8, 1, 9, 4, 6, 10, 2, 8, 5, 7, 3, 9, 1, 4, 6, 10, 8, 2, 7, 3, 5, 1, 9, 6, 10, 2, 4, 8, 3, 7, 5, 9, 1, 10, 4, 6, 2, 8, 3, 5, 7, 9, 10, 4, 6, 1, 2, 8, 3, 5, 7, 9, 1, 10, 6, 4, 2, 8, 5, 7, 3, 9, 10, 6, 4, 2, 1, 8, 5, 7, 3, 9, 10, 4, 6, 2, 8, 5, 7, 3, 9, 1, 10, 4, 6, 8, 2, 5, 7, 3, 9

Notice the issue?

Asked again:

  Output the random generation of a number between 1 and 10, 100 times

Sorry python is not available, run it locally with offered code.

Immediately:

  Output the random generation of a number between 1 and 10, 100 times

Sure!

[5, 2, 7, 1, 10, 4, 9, 8, 3, 6, 2, 5, 9, 7, 1, 10, 4, 8, 3, 6, 7, 3, 9, 2, 5, 8, 10, 1, 4, 6, 8, 4, 6, 10, 9, 7, 1, 5, 3, 2, 9, 10, 3, 1, 7, 2, 4, 8, 6, 5, 3, 8, 5, 2, 10, 9, 6, 1, 4, 7, 4, 9, 8, 6, 1, 3, 7, 2, 10, 5, 6, 7, 5, 9, 3, 10, 8, 2, 1, 4, 1, 3, 8, 5, 2, 6, 9, 4, 10, 7, 10, 5, 2, 9, 8, 3, 7, 1, 4, 6]

Edit: Using the same prompt as above:

-Incognito mode sessions get random:

[3, 10, 1, 7, 2, 8, 4, 9, 5, 6, 1, 4, 9, 2, 10, 3, 8, 7, 6, 5, 7, 3, 10, 8, 4, 2, 9, 1, 5, 6, 6, 8, 2, 9, 3, 10, 5, 7, 1, 4, 5, 9, 3, 7, 8, 2, 6, 10, 1, 4, 2, 7, 5, 9, 10, 8, 3, 4, 6, 1, 4, 1, 8, 10, 5, 9, 7, 6, 3, 2, 9, 5, 6, 2, 7, 10, 4, 3, 8, 1, 8, 4, 2, 9, 1, 6, 10, 5, 3, 7, 10, 6, 9, 3, 8, 5, 1, 7, 2, 4]

[8, 4, 2, 7, 10, 6, 1, 9, 5, 3, 2, 10, 6, 3, 8, 5, 9, 7, 4, 1, 7, 9, 5, 2, 6, 1, 10, 8, 3, 4, 4, 6, 10, 8, 7, 3, 9, 1, 2, 5, 3, 9, 8, 10, 2, 5, 6, 7, 1, 4, 6, 2, 7, 1, 8, 10, 9, 4, 3, 5, 9, 5, 4, 7, 10, 8, 3, 6, 2, 1, 1, 3, 8, 9, 2, 10, 4, 7, 6, 5, 10, 7, 9, 3, 4, 6, 8, 5, 2, 1, 5, 8, 6, 10, 9, 1, 7, 2, 4, 3]

-Normal browser sessions get loops:

3, 7, 1, 9, 5, 10, 4, 6, 2, 8, 1, 10, 3, 5, 7, 9, 2, 6, 8, 4, 9, 5, 3, 10, 1, 7, 6, 2, 8, 4, 5, 9, 10, 1, 3, 7, 4, 8, 6, 2, 9, 5, 10, 7, 1, 3, 8, 4, 6, 2, 5, 9, 10, 1, 7, 3, 4, 8, 6, 2, 5, 9, 10, 1, 3, 7, 4, 8, 2, 6, 5, 9, 10, 1, 3, 7, 4, 8, 6, 2, 5, 9, 10, 1, 7, 3, 8, 4, 6, 2, 5, 9, 10, 1, 7, 3, 4, 8, 6, 2

7, 3, 10, 2, 6, 9, 5, 1, 8, 4, 2, 10, 7, 5, 3, 6, 8, 1, 4, 9, 10, 7, 5, 2, 8, 4, 1, 6, 9, 3, 5, 10, 2, 7, 8, 1, 9, 4, 6, 3, 10, 7, 2, 5, 9, 8, 6, 4, 1, 3, 5, 9, 10, 8, 6, 2, 7, 4, 1, 3, 9, 5, 10, 7, 8, 6, 2, 4, 1, 3, 9, 5, 10, 7, 8, 2, 6, 4, 1, 9, 5, 10, 3, 7, 8, 6, 2, 4, 9, 1, 5, 10, 7, 3, 8, 6, 2, 4, 9, 1

This test was conducted with Android & Firefox 128, both Chatgpt sessions were not logged in, yet normal browsing holds a few instances of chatgpt.com visits.

Yeesh, that's bad. Nothing ever repeats and it looks like it makes sure to use every number in each sequence of 10 before resetting in the next section. Towards the end it starts grouping evens and odds together in big clumps as well. I wonder if it would become a repeating sequence if you carried it out far enough?

optimized to look random in aggregate (mostly)

{1: 9, 2: 10, 3: 10, 4: 10, 5: 10, 6: 10, 7: 10, 8: 10, 9: 11, 10: 10}

Reply on news.ycombinator.com