Hacker News Reader

Comment by denton-scratch | Hacker News Reader

> so that if another person is shown your sequence of digits from 1 to 6, he/she should not be able to tell whether these numbers were produced by a real die or just “made up” by somebody.

That instruction is a flaw in the experiment. It's always impossible to tell, for any given sequence, whether it was produced by a fair die. There's nothing an experimental subject can do to make the impossible more impossible.

> the kind of sequence you’d get if you really rolled a die

Well, there is no such sequence. The instruction is incoherent.

You could take it as meaning "Construct a sequence that you think will convince others that it came from a random source". That would be coherent. And then it would be legitimate to eliminate responses that were all-heads ("clearly didn't even try"). But then what are you measuring? The comparative understanding of older and youger people concerning random sources, or the Gambler's Paradox? Their comparative expertise in human psychology? Their comparative willingness to move the mouse-pointer over the screen?

View on HN

You're wrong. Mathematically. Here's why.

When you throw a coin 100 times, each sequence you get is equally likely. However. You can look at properties of the sequence which are more likely to be one way than the other. For instance, it's more likely that the number of heads and tails are about equal than not. The reason is that there are more sequences, in general, where that is true, than those where heads or tails strongly prevail.

With the right property, you can make statements such as: This sequence is statistically likelier to be human made than random.

One such property is for instance the number of changes from heads to tails, or vice versa. In expectation, random sequences change heads to tails about 50% of all flips. For humans, the expectation is much higher. Hence, if you compare two sequences where one has 51% changes and one 63%, it is mathematically (/statistically) accurate to say that the latter one is likelier human made.

Your point doesnt refute OPs argument. Your final statement "you can say the latter is more likely random" is not the same as "you can say this sequence is not random". I think lots of people (especially programmers) who know about true RNG vs expectations of RNG might intentionally put in strings of same numbers, or not include the full set, because we know its what often happens during plain RNG. It isnt clear what the goal of the sequence is, hence the confusion in the comments.

Exactly, and their "good" dice roll sequence, 3 1 5 6 2 6 3 4 4 1 contained the full set which should only happen ~1/4 of the time for 10 rolls. It also contained no number more than twice, which should happen < 7% of the time. This looks to me like they purposely tried to make this sequence look like their idea of "random".

I'm curious about how they scored this section because my overall age was reported to be 60+ with the sequence 2 1 5 2 6 2 2 4 6 6.

1/4 of the time and <7% does not make it impossible. In fact thats kinda the whole point of RNG.

I also scored 60+ (actual age is in my 30s). I had similar thoughts and also did things like not use up all the numbers and repeat numbers more than twice exactly because I've looked at a lot of random number sequences in my life and I was trying to make it look like one of those.

> I've looked at a lot of random number sequences in my life and I was trying to make it look like one of those.

This is perhaps the difference between pseudo and statistically random. No idea which of those the study or the experiment is trying to validate btw.

And IIRC, interestingly they write that human capacity to create random numbers declines 25+. I can imagine that the older we are the more we look for something to make our decisions look more random based on what we've learned so far - more time, there was more time to look at more random number sequences - and the less random the outcome will be.

> And IIRC, interestingly they write that human capacity to create random numbers declines 25+. I can imagine that the older we are the more we look for something to make our decisions look more random based on what we've learned so far - more time, there was more time to look at more random number sequences - and the less random the outcome will be.

This is what they are testing, and at least based on the data they've got so far, it looks like it increases up to 25-ish and then stays pretty flat.

Another possibly interesting observation is that their preliminary data set (just eyeballing it, but) looks to have gotten

1) a flatter response

2) generally, less random responses

Which leads me to wonder if the live stats have been skewed more random as there might be some correlation between "interested in this sort of thing" and "has some idea what a random distribution ought to look like," and possibly this knowledge doesn't go away with age.

My opinion is that "real" randomness looks less random than artificial randomness.

That's why Apple changed iTunes random music shuffle to be less random because people complained that it wasn't random enough and replayed songs too close together.

https://www.cultofmac.com/181517/why-itunes-shuffling-order-...

> This is what they are testing

(human capacity to create random numbers declines 25+)

Not really; what they're testing is what kinds of response differently-aged people give to their question. So it's important what the question actually is; and it's important if the question might, for example, seem to older people to be a waste of their time.

They're not measuring what they claim to be measuring.

> This is perhaps the difference between pseudo and statistically random.

Not quite. Pseudo-randomness is defined as being indistinguishable from a uniform distribution, meaning if the next in a sequence is no more predictable than a statistically random selection.

I always pressed the same button. Let's say 10x 1. You get a rating below 60 then. Just in case you need the sequence to come towards your real age group when you redo the experiment.

Perhaps to guess random is also in the property of the age of someone clicking on a website? Perhaps someone should create an experiment that finds their experiment flawed ^^

What does that mean, "you can tell this sequence is not random"? If you show me a blue hat, and ask me if it's blue, I'll say yes, I can say it's blue. But there is always a chance it's not actually blue. It's very conceivable that I'm in a situation where I say with confidence that something is blue, but it isn't.

You always only ever speak in probability. Of course you can't say the sequence isn't random, because every sequence can be the result of a random process. "can you tell which is random" to me is equivalent as asking "is one of the sequences such that it is rational to choose it over the other as being random".

It's about rational decisions. Consider the frequentist view, where a probability p for an event A means that out of k trials, pk will show A (in expectation), and fruthermore if k -> infty, then the portion of events that show A will converge to p. If you want to choose the right sequence as being random as often as possible, it is rational to choose the one I described above, because it will, overall, be the one that is MORE OFTEN the random one compared to the other.

For a blue hat, it either is or isn't blue (and there's some rather strong evidence - whether or not it looks blue). Like, with a sequence of 6 digits, if you don't know whether the source was random or not, then that's like NOT showing me your hat, and asking me whether it's red or blue.

For a single sequence of six digits, it might or might not have come from a random source. You can't get any edge on that judgement by just inspecting the sequence. Only inspecting the source (the hat, if you like) can give you an advantage. Perhaps you're colour-blind, or the lighting is weird; so there's still uncertainty. But that's equivalent to examining the source of the digits, determining that it's really a random source, but making a mistake in your determination. That's uncertainty on a different level.

Red-pill blue-pill is a sort of meta-uncertainty.

> "is one of the sequences such that it is rational to choose it over the other as being random"

Most people don't care about this shit; it doesn't matter to them what random means, nor whether it's sequences or sources that can be said to be random. But for some people it does matter, and they have to try to use language precisely.

All [red|blue] hats are either red or blue. But no sequence is random or non-random; it's the source of the sequence (the process, if you like) than can be random or non-random.

If 111111 is emitted by a random process, then you can call that a "random sequence" if you like. If I emit 126692 from my ass (not a random process), that's not a "random sequence" in any sense, whatever statistical properties it has. You can't tell which is of random origin by inspection. The experimental subjects face an impossible challenge, and I can't see what conclusions you can draw from their responses.

Regarding "random process": (and sorry for commenting to myself)

I'm not taking a position on what a "random process" is; for these purposes, a PRNG, a LFSR or even the last three bits of the system-clock would do as well as radioactive decay.

"he/she should not be able to tell" isn't the same as "he/she should not be able to make a statistically-probable guess".

If it's light when I wake up, I would say that I can tell it's daytime, despite the possibility that it's still nighttime but a sufficiently near star has gone supernova or that the house next door is on fire.

I sense the sarcasm but I'm not sure which way you're intending it to go. What if you live in the arctic circle?

I wasn't trying to be sarcastic, just to give evidence against the statement

> "he/she should not be able to tell" isn't the same as "he/she should not be able to make a statistically-probable guess".

I'm in agreement with the sibling comment by baobabKoodaa.

By this logic the expression "being able to tell" should be banned from the English vocabulary, because no-one is able to tell anything with 100% certainty. Requiring 100% certainty as a precondition of using this expression is silly.

It depends on the framework. I can tell a geometric figure is a square because it’s a quadrilateral with right angles and sides of equal length. You could ask me a question like “Is a rectangle with a side of length 1 and a diagonal of root 2 a square?” and I can tell it is.

Ask me “Was 1 1 1 1 produced by a random process?” and it’s impossible to tell in the way I did with the square.

> It depends on the framework. I can tell a geometric figure is a square because it’s a quadrilateral with right angles and sides of equal length.

You're claiming to be able to craft a mathematical proof with 100% certainty. Although the thing you are proving appears to be obviously true (assuming a certain mathematical framework), the probability that you made a mistake is not 0%. You might falsely believe that the probability of making a mistake in a simple proof like this is 0%, but you would be wrong, and we have plenty of historical examples of mathematicians "proving" something and thinking that there is 0% chance of errors in the proof, only later being shown that they were incorrect.

You’re mixing up two layers of uncertainty. There’s an outer uncertainty. This would include things like I made a mistake, this is all a dream, etc. This outer uncertainty pervades all problems.

It’s often useful to ignore that outer uncertainty. We create a framework where we take certain things as true (shared reality, mathematical axioms). This framework may or may not have uncertainty inside of it, which we could call inner uncertainty.

Questions of probability have inner uncertainty. Questions of geometry do not. This makes them qualitatively different.

If you frame the initial task as something like “do your best to lead people to believe your sequence is random”, that makes sense. If the task is “make it so they can’t tell if it’s random”, that’s a bit off in some way. At the very least, it’s because you’ve presented the spotting of randomness as something that can truly be done to a logical conclusion (random/not or true/false). This violates both the outer and inner uncertainties of randomness.

Ask me "Was 19 19 19 19 19 19 19 produced by a random process" and I can say 'most likely not!'.

But then: https://www.dailymail.co.uk/news/article-2162190/What-odds-R...

Interestingly, the article computes the odds incorrectly: "... hit the same number on seven consecutive spins [...] the odds of which happening are 114billion to one...", which actually are the odds of having 7 consecutive 19s or the same (unspecified) number on 8 consecutive spins.

The process by which you came to hear about this particular spin of this particular roulette wheel was far from random.

> no-one is able to tell anything with 100% certainty

Including this very assertion? So it's possible that _someone_ could tell _something_ with 100% certainty?

Only a Sith deals in absolutes.

No seriously, humans can have 100% truth about core things that have an insane amount of empiricism such as e.g gravity being real. But it is accepted that for any non-empiritismed ad nauseam knowledge, when we use universal quantifiers, we tolerate generally some credible kinds of exceptions, contextually.

absolutely, probably you just did.

“Statistically likelier,” but still mathematically possible confirms the OP’s point.

This seems to be really testing for pseudo-random numbers. Relevant Dilbert (and article): https://www.lancaster.ac.uk/~blackb/RNG.html.

I don't think this has anything to do with pseudo random numbers :) PRNG are not perfect but their imperfections are impossible to detect by hand.

This is a good illustration of a binary epistemology vs a continuous one.

> It's always impossible to tell, for any given sequence, whether it was produced by a fair die.

Something like "you can't make any determination, because it's random". Whereas under the second worldview you can make statements about how likely things are, despite uncertainty.

For some reason the binary worldview seems to be incredibly common. My sibling commenter exhibits the same issue.

Could be. Though if you think long enough, with this worldview you can't decide anything, ever. And it's irrational. You can't say for sure which is random, but you can say for sure on which you should bet your money if you have to.

> you can make statements about how likely things are

Sure. And it's true that some sequences are more likely than others to have been emitted by a random process. [Edit] All sequences from a random process are equally likely. It's still true that some sequences are more-likely to have come from non-random processes.

The point is that randomness isn't a property of the sequence; it's a property of the process.

> For instance, it's more likely that the number of heads and tails are about equal than not

This isn't even true. Heads and tails being equal over 100 flips is something you'll see something like 8.33% of the time (not based on probability, I just ran a simulation of 100 flips 10,000 times and got 833 instances of them being equal)

edit: I missed the key word, "about". Sure, they're more likely within maybe 5-6 of one another than not.

Yes :) The exact probability for having 50%/50% is (100 nCr 50)/2^100, or about (as you found through experiemtation) 0.08.

The more often you run the experiement, the more likely you'll get a result close to 50%/50%, by the way. In the limit, you have a variance (i.e. spread of results away from the expected value, which is 50%) of 0. This is called the law of large numbers. As the generic name suggest, it's pretty central to mathematics haha.

The OP is essentially correct.

I was definitely confused and assumed it was about 'looking like' randomness.

But I did a lot of double clicking of things, because I felt that in 'real life' you're not going to get 1 roll of each number, but odd things happen.

But this is a bit moot - the people clicking 'all the same number' have obviously come to some different conclusion as the others - i.e. 'all possible values are the same' and therefore.

So what the study is really 'testing' probably, is how people react to the question.

They really need to change the question substantially in order to get randomness.

I don't see any insightful aspect in the experiment or the debate.

It's pedantic -> some people read the question differently and do different things.

I disagree. You want people to click numbers s.t. if you asked them 10k times, a uniform distribution would emerge. But that is not what's happening. They think all numbers have the same probablity, but if you click only 1, then the probability of your choice being random is low.

Edit:

Maybe this will convince you: You said each sequence of numbers is equally likely, hence, we can't tell. I'm going to disagree with that statement.

Let's say I give you a coin, and tell you: I've flipped this coin 100 times in a row, 10k times. And you look at the flips, and each flip result is 1111...111. Would you guess it's random, or biased? The probability of that happening is as high as any other sequence, but clearly, if you'd guess it was random, you'd be a fool. This is exactly what is happening here, just on a smaller scale: 111111111 being the result of the coinflip has a lower probability of being random than the result 100101101110.

11111 has to happen at some point if the experiment is really random. The probability that it happens with YOU is low, however. Thus, it is rational to decide that the sequence is not random. Because it most cases, it won't be.

> Would you guess it's random, or biased?

Well, I'd guess that it's not a coin-flip at all; even a biased coin won't produce 10,000 heads and no tails, unless it's a two-headed coin.

Let's go back to the actual case in hand: suppose you provide me with "111111", and not 10,000 1s. I simply have no way at all of determining whether that is more or less likely to have come from a random source. So I would decline your bet. If it was 10,000 1s, then maybe I'd be a fool to bet it was of random origin; but you can't convince me that a string of 6 1s is or isn't of random origin. So no bet.

This is all irrelevant. We're discussing a single sequence of 6 digits. There are not enough samples to perform statistical analysis. Probability doesn't come into it.

>It's always impossible to tell, for any given sequence, whether it was produced by a fair die.

If the sequence is long enough you can model how likely it is to have been produced by a fair die. Are all numbers equally distributed? Are some numbers more likely to follow or not follow other numbers? Are some patterns repeating?

Of course any sequence can be produced by a fair die, but you can still create some objective metric that will tell you how truly random a sequence is, and the longer it is the more accurate it will be. It's what tests like Diehard are about after all.

Can you roll a fair die a thousand times and only get 6s? Well yes of course. It'll never happen though.

Any single sequence of numbers has the exact same probability of being produced by a fair die (that's how the definition of "fair" goes). The probability of getting all 6 is the same of any other one you get.

That's true given that we have a fair die. But the question is, given some results, what is the probability that the die is fair? And there are statistical tools for that.

The probability of a specific sequence occurring is different from the probability of a specific sequence being random.

For sure, but I think it's more helpful to this about "classes" of sequences. Sequences which have a uniform distribution of digits (within some margin of error), sequences which do not have repeating patterns, sequences that do not contain the same digit twice in a row etc... Any single one of these sequence is as likely as any other, but some "classes" are vastly bigger (and therefore, more probable) than others. By deciding which classes of sequence any result belongs to, you can decide if it's likely to have been produced by a fair die or not.

This intersects with the concept of entropy: assuming that you have a box containing a gas whose particles move randomly about the volume of the box, then at some point you take a snapshot of the position of every single particle in the box and you discover that they're all in the right half of the box, the left side being in a vacuum. Would you assume that it's just random chance? It could be. It certainly isn't.

Meanwhile any of the trillions and trillions of snapshots showing particles more or less uniformly distributed within the box are all more "random looking" and are what is expected from such an experiment. These configurations as a group occupy the vast majority of the phase space for the contents of the box.

You work with crypto according to your profile. I hope that when you see a random generator return a series of a hundred 6s, you go and check what's wrong with it instead of assuming you just got lucky this time ;-)

This is much more relevant in crypography than on statistiscs. If your PRNG always returns the same 4, it's buggy, but the really problematic outputs look exactly like random numbers.

(Looks like my profile is out of date, by the way.)

We disagree on the definition (or divination) of "dice" and "sequence" and "roll".

The sequence length of a single die roll is 1. The probability of any particular roll is the same.

Let's do the next part with two-sided dice because it will be quicker.

The sequence length of a roll of two dice is 2. The roll is a bag; the probability of either [0] or [1] is lower than the probability of [1,2] because it's a set membership and [1,2] has the same members and is the same as [2,1]

> Can you roll a fair die a thousand times and only get 6s?

Anecdote time:

A few years ago with my friends we were discussing how easy is to roll 5 dices simultaneously and get the same result in all of them. This is a possible way to win a popular game here https://en.wikipedia.org/wiki/Generala We estimated how often you can roll the dices and the probability, and we estimated that you must try during 2 or 3 hours to get that result.

We were young, had a lot of free time, so one of my friend started rolling 5 dices while he was talking with us and eating. After about 2 hour he got the 5 equal dices in a roll. (IIRC he tried again, with a similar result.)

(Note that 2 or 3 hours is consistent with how this outcome is used in the game to win automatically. It's possible to get this in a normal game, but it's not super common.)

Also, each time you add a dice, the time increase exponentially. With 6 dice it's 14-21 hours, like a day. With 7 dice it's like a week. With 9 dice is like a month. With 10 dice is half a year. 1000 dice will take longer (and x6 if you want to get only 6s instead of any repeated number).

(Note that parallelizing this to all humans only adds 14 dices. If everyone on Earth start rolling 24 dices, it will take like half a year until one of us get 24 equal dices.)

Saving a click: Generala is an ancestor of Yahtzee.

i clicked to upvote you and now I'm writing this text.

thanks for nothing.

They are studying this

https://www.quantamagazine.org/elegant-six-page-proof-reveal...

Saving a click: Generala is a relative of Yatzy.

>Can you roll a fair die a thousand times and only get 6s? Well yes of course. It'll never happen though.

Somewhat related, but humans are also REALLY bad at generating randomness and one of our big tells is an aversion to repeats. If you ask someone to pick 0-9 randomly, repeatedly, they will rarely repeat numbers. But in a truly random sample a repeat is likely 10% of the time, and a three-peat will happen roughly once every 100 numbers. An average person will rarely if ever repeat a number and sure as heck won't ever come up with it three times in a row.

Then again i had exactly this in mind and in OP tasks i did repeat a number here and there.

Yeah but even then most people cap the repeats if they’re faking. If 100 people have to roll 10 random numbers, it is very likely some people will have 3-4 in a row of the same number. It’s less likely none will. My brother in law teaches stats and he runs this scenario with a seminar he teaches. He then runs the results through a simple formula he built in excel and can ascertain who faked vs. did it for real with about 95% certainty IIRC. I love little games like that haha

And funnily enough, you'll often hear this trait as being desirable in a pseudo-random number generator. People often want something that will jump around fairly unpredictably but that will come close to outputting all possible numbers once before getting into re-runs.

I’ve coded up this exact algorithm, it’s really fun. It’s useful for “shuffling” in the music sense (not the cards sense).

I think it’s actually the prototypical real-world software engineering problem. User says they want X (random music). X is a term in software, so you give them that (you get a random song). They’re not happy. You dig and find out they really want A, B, and C (next song is unknown, songs don’t repeat too soon or too infrequently). This new problem is harder to verify (how soon is too soon?).

Editing in tips on solving this sort of problem. You can turn vague requirements into precise requirements. Rather than make the precise requirements exactly equivalent to the vague ones, it's easier to make them more restrictive. Is playing a song again within 50% of the length of the playlist "too soon"? Maybe. How about within 80% of the length of the playlist? Definitely not. We can give ourselves the requirement "Songs must always play again between 80% and 125% of the length of the playlist." Much easier to solve, much easier to test.

Sometimes the extra restriction make the problem harder (not usually I've found). Still, this is a great trade because understanding requirements is harder than solving well defined problems.

[To the point of this whole post] Requirements can be turned into testable properties even if it's not programmatic. "When I look at a list of chosen songs, there must be no obvious patterns." Who says what's obvious? You do! Then, have someone else do the same.

Consider extreme cases. Extreme cases tend to be the most or least important. If they're least important, create a new set of easier requirements or drop it all together. "If 3 - 10 songs, always play within double the playlist, no obvious patterns, never twice in a row. If 2 alternate, if 1 repeat."

I actually had an engineering ask from somebody to produce "random sequences" which turned out to be anything but random. Short version: letter sequences, no repeats in a sequence or in adjacent sequences.

Took months and a lot of patience to extract that crucial piece of information from the customer. I actually coded a recursive algorithm which to my surprise generated every possible (three letter) sequence in an acceptable sequence, enough for them for several lifetimes.

> Extreme cases tend to be the most or least important.

Yeah, that's my suspicion.

There's an overlap in meaning between random, strange and unknown. "This random guy came up to me..." Keeping that in mind helps when talking to people about "random".

Yes, it's a very desirable trait in https://en.m.wikipedia.org/wiki/Quasi-Monte_Carlo_method

Quasi-Monte Carlo has a rate of convergence close to O(1/N), whereas the rate for the Monte Carlo method is O(N^(−0.5))

For such applications it's best to use quasi-random numbers (a.k.a. low-discrepancy sequences) such as the Halton sequence or the Sobol sequence instead of pseudorandom numbers.

Thank you for the link - I had not heard of this kind of sequence. It looks like something I'd like to know about, but I think it's beyond my schoolboy-level mathematical abilities. Anyway, I guess I'll have a peek in the rabbit-hole.

What is an example of a situation in which this is desirable:

> come close to outputting all possible numbers once before getting into re-runs

In a dice rolling game, you want as close to true random as your PRNG can get. In card drawing, you typically want EXACTLY all possible cards once before getting "repeats". Where do you want something in between?

> dice rolling.... card drawing...

What you're talking about is "with/without replacement".

Last time I encountered this was creating random IDs for things, either as a random string of characters or when selecting from lists of attributes and animals like all the tools that'll name things like "curious possum". It's not actually a hard requirement to hit every possibility before repeating, but if you do see 2 or 3 clusters in a sample it gives people the impression it isn't random.

Assuming the die is fair, that outcome would be exactly as likely as any other completely specified outcome.

> It's always impossible to tell, for any given sequence, whether it was produced by a fair die. There's nothing an experimental subject can do to make the impossible more impossible.

That's just not true.

Or feel free to play a game with me. We'll roll a 20 sided die. If it comes up 20, you give me a ten. If it comes up any other number, I'll get you a dollar. Nice EV on that!

Oh, the die has come up 20, 20, 20, 20, 20, 20, 20 the last seven times. Do you play?

It can still be a fair die.

The context implies it's not, but it's only unlikely, not impossible.

See https://youtu.be/8Ko3TdPy0TU

It introduces a 10 billion human second century i.e. 3.1 x 10^19. If your chance times it is greater than 1 then it's definitely plausible to be done by someone, somewhere.

I would; would you? The Gambler's Paradox says you shouldn't (I'm assuming your 20-sided die isn't crooked).

Incidentally, you haven't made your case that you can ever tell whether a given sequence was produced by a fair die. You've just asserted it, and then suggested a game that doesn't illuminate anything.

It would be incredibly stupid to take the bet, as it's way more likely that the sequence was not produced by a fair die than that it was (i.e. the dice is rigged in the example)

Just because it's impossible to know for certain, doesn't mean you can't make a prediction with very high chance of being correct.

I'm not really a gambling man, but I'd expect a crooked 20-sided die to produce a biased sequence, not a running straight. I don't know if it's possible to make a die that always rolls the same, and I'd expect any such die to fail a superficial inspection (all sides but one bulge; one side is larger than the others; the die has a weird magnetic field; the die is heavily weighted on one side).

So I'd still expect a running straight to be rare, even with a crooked die.

Of course, if I watched the die produce 7 20s in a row, and was then asked to bet on the next roll NOT being 20, I'd be stupid to assume the die was fair without inspecting it.

All this is beside the point; the instructions invite the subject to produce a sequence that they think will convince people it was produced by a roll of dice. But there is no sequence that SHOULD have that power to convince.

> All this is beside the point; the instructions invite the subject to produce a sequence that they think will convince people it was produced by a roll of dice. But there is no sequence that SHOULD have that power to convince.

If you genuinely believe that, we can easily set up a sequence of bets where you will win infinite amounts of money from me. But of course, you don't genuinely believe that, so you aren't interested in making bets around your supposed "beliefs".

I guess my "supposed" beliefs must be the beliefs you suppose I have. Whatever.

If you're offering me a bet, and you can easily set it up, then what bet are you proposing? You haven't been very specific. I'm no Turf Accountant[0], but I can spot a three-card-trick when I see one.

[0] https://en.wikipedia.org/w/index.php?title=Turf_accountant

Fine. You made this statement:

> the instructions invite the subject to produce a sequence that they think will convince people it was produced by a roll of dice. But there is no sequence that SHOULD have that power to convince.

Let's gather a random sample of people 20 people. I will produce 10 manually generated sequences of dice rolls and 10 actual dice roll sequences. The sequences are added to a list and the list is shuffled. We will present each person with a sequence from the list (sampling without replacement), and the person should guess whether the sequence was manually generated or produced with a dice roll. For each person who correctly identifies a manual sequence as a manual sequence, I will pay you $1. For each person who mis-identifies a manual sequence as an actual dice-roll sequence, you will pay me $1000. As you said, you believe no sequence should have the power to convince a person of such a thing, you will obviously never have to actually pay me $1000, you would just collect 20 x $1 from me. I'd be happy to continue this up to infinity in batches of 20, so you will eventually get infinite dollars from me.

I will need escrow.

> For each person who correctly identifies a []manual sequence[] as a manual sequence, I will pay you $1. For each person who mis-identifies a []manual sequence[] as an actual dice-roll sequence, you will pay me $1000.

Payment only happens for the manual sequences here.

> you would just collect 20 x $1 from me.

So they'd only get 10 x $1 per batch.

No bet!

You promised me "infinite amounts of money", I only stand to win 20 bucks.

Also, you have specified that these are 20 random people; so I guess I don't get to brief them in advance that they MUST say manual each time. So you have replaced me, the bettor, with a panel of 20 people whose average IQ is 100, and who don't have my interests at heart. Why would I take that bet?

If my random panel say manual each time, you stand to lose.

But as I say, I don't bet often. Only once a year, only on gee-gees, and only as much as I'm willing to lose (because I always lose).

Look, you're merely pretending to disagree. You're pretending to believe something akin to "it's impossible to craft a sequence of numbers that convinces observes of its randomness". But you don't actually believe this [or whatever minor variation of that statement that you'd find agreeable in rhetoric]. If you did believe it, you would find a wager that we could do to settle this disagreement. But no such wager can possibly be formulated, because you don't actually believe what you pretend to believe.

It's not about my beliefs, except my belief that the die is fair.

I made a statement about a simple bet, using a fair 20-sided die, on the outcome of the 8th roll after the die has just come up 20 seven times. I'll take the bet that it doesn't come up 20 on the 8th roll. This scenario with 20 random people and 10 sequences appears to be a different scenario. I'm not sure how the odds work in that scenario, and I don't fancy that bet. That's all.

> I made a statement about a simple bet, using a fair 20-sided die, on the outcome of the 8th roll after the die has just come up 20 seven times. I'll take the bet that it doesn't come up 20 on the 8th roll.

How is this related to the topic at all? Our disagreement concerns the ability of humans to produce numbers that look random, our disagreement doesn't concern the ability of a fair die to produce numbers that look random. Of course a fair die is going to produce numbers that look random, that's not related to this discussion at all!

It's related only to your challenge with a contrived and complicated betting scenario.

"The topic" is whether the challenge faced by the experimental subjects makes any sense. It doesn't; the challenge is to produce a string of six symbols that others can't distinguish from randomness. There is no such string. Your contrived betting scenario doesn't illuminate the issue; it's an attempt to distract attention, and IMO it's not in good faith.

> "The topic" is whether the challenge faced by the experimental subjects makes any sense. It doesn't; the challenge is to produce a string of six symbols that others can't distinguish from randomness. There is no such string.

I strongly disagree with that statement. On a surface-level inspection, some strings appear to have more entropy than others ("can be distinguished from randomly generated strings"). This absolutely is a real thing, and it's measurably real. If you genuinely believe what you say, then we can easily wager on it and find out who's right.

> Your contrived betting scenario doesn't illuminate the issue; it's an attempt to distract attention, and IMO it's not in good faith.

You asked me to produce a specific betting scenario, and I did my best to entertain you with that. I crafted the scenario in good faith insofar as I tried my best to answer to your request. I suppose you can still argue that it's not in good faith because I never had any expectation that you would take the wager. But like I said before, that's because there is no way to formulate our disagreement as a betting scenario that you would accept, because you don't genuinely believe the claim you are making here, so you will simply weasel out of any wager.

I withdraw my accusation of bad faith, and I apologise for that remark.

I call on you to withdraw your claim that my remarks were made in bad faith.

I still don't know why you felt the need to contrive a complicated betting scenario, when we were discussing a "simpler" scenario that already involved an icosohedral die. If we're using betting scenarios to model good faith, then aren't simple scenarios more useful than complex ones? Ergo, coin-toss is the most appropriate.

But trying to talk about this stuff in terms of physical things like coins or dice inevitably turns into discussion about unfair coins and crooked dice, or whether the caster can influence the outcome; so argument by analogy quickly leads to dead ends, in this area.

> I call on you to withdraw your claim that my remarks were made in bad faith.

I don't want to offend you, but I still do not think you believe the claim you are making. If you actually do believe it, we should be able to formulate our disagreement in the form of a wager and use the scientific method to determine which one of us is correct.

> I still don't know why you felt the need to contrive a complicated betting scenario, when we were discussing a "simpler" scenario that already involved an icosohedral die. If we're using betting scenarios to model good faith, then aren't simple scenarios more useful than complex ones? Ergo, coin-toss is the most appropriate.

The die scenario you formulated is not suitable, because it is unrelated to our disagreement. We're looking for a wager that illustrates the disagreement, e.g. you would be taking one side of the wager and I would be taking the other side of the wager. Your die scenario is not like this, because both of us would be taking the same side of the wager in that scenario; it has no connection to our disagreement at all.

I tried my best to formulate a simple scenario that would illustrate our difference and allow us to wager on which one of us is right. Sure we could use coin-toss instead of dice. Feel free to formulate a scenario.

> But trying to talk about this stuff in terms of physical things like coins or dice inevitably turns into discussion about unfair coins and crooked dice, or whether the caster can influence the outcome; so argument by analogy quickly leads to dead ends, in this area.

The disagreement concerns whether a person can produce a sequence of numbers in a way such that another person will/will not not be able to determine whether that sequence of numbers was manufactured or generated by a random process. A generator like coin toss or dice roll is very appropriate here.

This is the telltale sign of someone who is wrong but refuses to admit it. They begin bringing up completely tangential points and ideas to distract from the original argument that they now realize they lost.

> I don't know if it's possible to make a die that always rolls the same

The easiest way would be to put the same number on every side, which would probably fail a superficial human inspection (but might pass a surprising number of machine inspections).

Hadn't thought of that!

B-but statistics taught me that the events are independent! Take my 10!

The prompt told you it was independent. Not statistics.

Neither the prompt nor statistics told anyone it was independent. The prompt just says a 20-sided die, not a fair one.

It's just a question about what we perceive as random. It has nothing to do with the probability of the sequence being produced by a die, only with the probability of the sequence being produced by a human. A 20 20 20 sequence is not less random, it's just more likely to be produced by someone with incentive to cheat.

How did the researchers measure the "randomness" of a particular sequence in this experiment?

> Formally, the algorithmic (Kolmogorov-Chaitin) complexity of a string is the length of the shortest program that, running on a universal Turing machine (an abstract general-purpose computer), produces the string and halts.

Oh, interesting.

I think we want (and expect) RNGs to produce sequences with high algorithmic complexity (which we regard as "random") and ignore the (almost impossible) possibility that they fail to do so.

An impossible to create modified RNG which always produces sequences with high algorithmic complexity would better meet our expectations of randomness, but would be less random because it could not produce uniform sequences (among others).

The obvious thing to do is then conduct a physical test on the die, for example trying to spin it on an axis with the 20 face near the top of the axis. An obviously loaded die will not be mass-symmetrical and will not spin well.

In general this is called an independent test for systematic bias and it's something often left out of statistical arguments.

Yeah except that can happen. Its vanishingly a small probability but it's not impossible. I would feel that the die is crooked but feelings are not proof

You’re comparing apples to oranges. The correct analogy is: you pick any other sequence of numbers between 1 and 20 and then tell me you’re more likely to win because your sequence is more random.

Hey, out of curiosity, why did you choose a 20-sided die? I assume it was just to match the "ten bucks" bit. I'm not snarking, I just began thinking about 20-sided dice.

I do not believe there is a solid with 20 faces, and all faces congruent (I haven't checked). So we end up with a solid that is roughly ball-shaped, with faces of different sizes and shapes.

[Edit] I checked; I suspected I'd made a fool of myself. An icosahedron has 20 congruent faces, of course.

A ball-shaped die is much more likely to topple, and so more likely to be influenced by small differences in weight distribution, selective corner-shaving, whatever.

I can't think of a way of judging the fairness of a 20-sided die other than casting it many times, and analysing the results. I'd be much more confident in my ability to judge by inspection the fairness of a 6-sided die.

> Oh, the die has come up 20, 20, 20, 20, 20, 20, 20 the last seven times. Do you play?

If you let me float the die in a cup of water and spin it to determine its not weighted so the 20 comes up, or have some magical means of assuring me the die is not weighted 100% yes I would play.

It's entirely possible that a fair die rolls 20 7x in a row, but it's more probable that you're cheating.

> That instruction is a flaw in the experiment. It's always impossible to tell, for any given sequence, whether it was produced by a fair die. There's nothing an experimental subject can do to make the impossible more impossible.

Baloney!

Say you measure the traffic to your website in the morning and the evening, every day for a week. And this is what you see:

    day:      1  1  2  2  3  3  4  4  5  5  6  6  7  7
    time:     M  E  M  E  M  E  M  E  M  E  M  E  M  E
    visitors: 51 73 58 72 50 78 55 74 55 77 52 73 55 76

It could be that the traffic you get is uniformly random, and each day the number of visitors you get is a uniformly random number between 50 and 80. Sure, all the morning numbers are less and the evening numbers are more and there are suspiciously no numbers in the 60s. All that could be a coincidence.

You know perfectly well, though, that you're getting more traffic in the evenings.

You've answered a different question. The question is: can you construct a sequence of dice-casts that an adversary can't distinguish from a real dice-cast? Answer: you can't.

> The question is: can you construct a sequence of dice-casts that an adversary can't distinguish from a real dice-cast?

Yes, obviously:

A: 2 4 4 3 5 2 3 6 4 2

B: 5 5 5 6 4 1 1 2 4 5

One is a real dice-cast with a d6 I had lying around. The other is the (100% deterministic) output of:

  echo 1651434259 $X | md5sum | grep -o '[1-6]' | paste -sd' '

for some X. Feel free to explain[0] how you are able to distinguish which is which.

The real issue is that most people don't bother to produce random numbers in a way that's actually secure (which, to be fair, is rather tedious if you don't have a computer handy, and downright prohibitively impractical if you want to do it all in your head, so why would you bother?), either in the study or in general.

0: If you'd like a more black-box distinguishment, I can provide a longer list; obviously a adversary can get the right answer 50% of the time just by chance.

Not sure that I'm disagreeing with you; but:

Can you contruct a sequence that an adversary CAN distinguish from a real dice-cast?

If you can't distinguish a spoof from the real article, then that blade has two edges. It's impossible to distinguish them, so the instruction in the pudding test that you are to make a sequence that is indistinguishable from a dice-cast is meaningless, because any sequence is indistinguishable from a dice-cast.

If you ask people to do impossible things before breakfast, then it's not sensible to do an analysis of what they end up doing. It's a waste of time.

> Can you contruct a sequence that an adversary CAN distinguish from a real dice-cast?

With high probability (over the distribution of possible dice-casts to distinguish it from), yes:

A: 6 4 3 2 1 4 5 1 6 5

B: 1 1 1 1 1 1 1 1 1 1

Same procedure (A=constructed,B=10d6,xchg A <=> B if another d6 is >3), but this time I clearly don't need to explain how the constructed sequence was constructed.

The adversary will sometimes get it wrong because 10d6 came up "1 1 1 1 1 1 1 1 1 1" itself (or something equally-plausible like "6 6 6 6 6 6 6 6 6 6"), but they'll do much better than random guessing. Whereas being able to do better than random guessing against a CSPRNG/stream cipher means the cryptographic primitive is completely broken. (I'm not sure it's a direct break for a hash function, but it's still pretty bad.)

Now I'm curious: in your opinion what is funny about the line "Nine Nine Nine Nine Nine Nine" in the following cartoon strip, vs. something like "Two Nine Eight Three Seven Eight":

https://dilbert.com/search_results?terms=Random%20Number%20G...

Not the person you asked, but,

P(9,9,9,9,9,9 | Not random ) > average over all values of (sequence) of P((sequence) | Not random )

(under some reasonable assumptions about how likely you consider different processes to be the process producing the output)

While

P(9,9,9,9,9,9 | random ) = average over all values of (sequence) of P((sequence) | random )

Therefore, P( Not random | 9,9,9,9,9,9 ) > P( Not random )

Martin Gardner had a wonderful demonstration in one of his columns, where he had people write out 36 "random" numbers, then fold them into a 6x6 square. there were inevitably doubles or triples of the same digit going vertically, and almost never horizontally, since repeating a digit didn't seem random enough to the person generating the sequence.

I don't think it's a flawed instruction -- i.e., I believe most participants will read it as intended, and it will produce interesting results. I read it as follows:

"Suppose we show a person your sequence and a sequence generated via an actual random number generator. Minimize the average probability that that person correctly guesses which sequence was generated by a human, across all possible real random sequences."

I guess most other people would read it that way, even it is not formally correct from a statistical perspective. And I believe it gets at what they're trying to measure.

> I guess most other people would read it that way

Not me. But as I said upthread, that's the only way I can construe the question as being coherent. And I agree that the question thus construed invites the experimental subject to make predictions about the behaviour of others; so the question tests the subject's accuracy in psychological judgements.

If you construe it as being a coherent question.

I think it's silly to construe it that way; I just think it was a crap question, and you can't draw any conclusions at all from the results.

Too deep topic and maybe I don't understand it compared to the megaminds here on hacker News but as far as I get it when it comes to randomness every sequence has the same probability which is simply the inverse of the number of sequences for any given length.

In that case it seems to me impossible to tell from only the information of a single sequence.

But if you have multiple sequences where one is the binary coding of a Mozart symphony and another is the binary encoding of Shakespeare sonnet and another is the binary encoding of days of the week or a DNA sequence if you have enough data points like that I think you can start to say Well they're probably not random because we see coherent patterns.

The second aspect is coherent pattern depends on perspective. So what seems random to us maybe a very common physical constant to an alien civilization. so if we see a binary sequence that represents pi, maybe we say oh well that's not random because it's accurately pi to like a thousand places... but if we see another sequence that looks random maybe it's a physical constant to an alien civilization that's accurate to a thousand places as well. But we don't know that so it looks random...to us. So randomness doesn't just depend on entropy, I think in a realistic point of view, probability is not enough, it also depends on something which Maybe it's a little bit harder to measure perhaps which is perspective. Random depends on the context.

I think your way of looking at it requires an explosion of alien civilisations to find one in which our random 1000 digits is their special number. If you generate 1 extra digit, you'd need 10x as many civilizations to expect a match.

Getting back to the experiment, obviously it's measuring the human mind so the context is the person's mind. They might enter their bank account number which looks random to the researchers but is actually cheating because they're not using their brain as the source of randomness.

That's an interesting point about the random sequence selected what's the chance it matches an alien civilization constant. I didn't think about that way. I guess I thought about it more from your other example point of view, where say an alien civilization pretends to be a human and gives you a number that they say is random (just like the guy giving you his bank account number) but actually the alien civilization gave you a physical constant.

It's interesting that there's a difference but it's sort of like the difference between you picking a random sequence from a random source and the probability of that being actually random versus you being given a supposedly random sequence by somebody and the probability of that being actually random. I think.

Yea I guess we couldn't tell if somebody gave us a number that was special to them but appeared random. Afterall, the alien's physical constant (or rather, the arbitrary definitions of their units) would have effectively been chosen by a random number generator just like ours are.

If you pick a random sequence from a random source, then it's truly random by definition, isn't it? Even if it happens to be 1111111. It might score badly on a randomness measure but no randomness measure is truly perfect.

I think this is the idea you are looking for: https://en.m.wikipedia.org/wiki/Kolmogorov_complexity

Let's not confuse P(observed_rolls|used_dice) with P(used_dice|observed_rolls). P(observed_rolls|used_dice) is always the same, independent of observed rolls, assuming the dice are fair. But P(used_dice|observed_rolls) can vary, because other ways of generating rolls which are under consideration may be biased towards certain answers, and this allows you to perform inference.

For example, suppose the rolls you will be shown were either generated by fair dice or by the program "always return 4" [4]. The rolls you are given are "4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4". Are you really thinking you'd make the same prediction for this sequence of rolls as you would for the sequence "1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1", or is there perhaps some SMALL INKLING OF A HINT as to which answer is correct?

The simple fact of the matter is that, in the real world, if you see a sequence like "4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4" you can be surprisingly confident that it was not generated by die rolls. This is because there are plenty of other ways to get sequences and those other hypotheses didn't just pay a Bayes factor penalty of a trillion. Seeing the instructions as incoherent is the mistake of trying to over-isolate the study to the abstract mathematical realm, instead of the world people actually operate in. If someone tells you their luggage combination is 1234, do you really think it's meaningless to guess that it was the default combination as opposed to being generated by secure die rolls? Do you not form opinions about whether or not someone is using secure randomly generated passwords when you find out their password is "password2"?

4: https://xkcd.com/221/

All sequences are equally likely to be produced by a fair die but humans are very biased in the kinds of sequences they produce. It might be impossible to ever be certain but you can certainly look at a sequence of all sixes (or that contains more complicated patterns) and estimate that it was much more likely to be produced by a human than a die.

I would have thought that “looking random” would have been calculated by just checking if there is a bias in people’s answers. If someone chooses something that has never been seen before vs something that has been picked a hundred times, then it might be “more random” as the bias clearly comes from humans

Here are some numbers:

123456123456

Did I get them by rolling a die?

Almost certainly not.

Given a circle with diameter AB and a point on the perimeter C, is ABC an equilateral triangle?

It's impossible for ABC to be equilateral. Maybe you meant right triangle.

> It's impossible for ABC to be equilateral.

Exactly! This is why you can answer "no" to my question, not "almost certainly not".

There's no way of telling. I'll guess you didn't, but that's a guess about people, not about sequences of digits.

See, it's not about how likely it is that OP rolled 123456123456; it could have been any "unlikely" number (where I suppose "unlikely" is a psychological quality). So OP keeps rolling until an "unlikely" number comes up, and exclaims "Wow, how unlikely was that?". Well, it's impossible to know, without knowing how many "unlikely" numbers there are; but it's much more likely than 123456123456 is.

Aren't all sequences really equally unlikely? So whatever you roll, it's as improbable as all-1s?

Is there a null hypothesis that some other mechanism has a higher probability of producing that sequence?

Reply on news.ycombinator.com