Test your “free will” at the Aaronson Oracle
people.ischool.berkeley.edu
people.ischool.berkeley.edu
f f f f f f d f f f f f f d d f f f f d f d f f f d f f d f f f f f f d d d f f f d d f d f f d d f f d f d d f f f f d f d f d f f f d f f d f f f f f f d d d d f f d d d f d f d d d f f f d d f d d f d f f d d f f d f d d f f f f d f d f d f f f d f f d f f f f f f d d d d d f d d d d f f d d d f d f d d d f f f d d f d d f d f f d d f f d f d d f f f f d f d f d f f f d f f d f f f f f f d d d d d d f d d d d f f d d d f d f d d d f f f d d f d d f d f f d d f f d f d d f f f f d f d f d f f f d f f d f f f f f f d d d d d d f d d d d f f d d d f d f d d d f f f d d f d d f d f f d d f f d f d d f f f f d f d f d f f f d f f d f f f f f f d d d d d d f d d d d f f d d d f d f d d d f f f d d f d d f d f f d d f f d f d d f f f f d f d f d f f f d f f d f f f f f f d d d d d d f d d d d f f d d d f d f d d d f f f d d f d d f d f f d d f f d f d d f f f f d f d f d f f f d f f d f f f f f f d d d d d d f d d d d f f d d d f d f d d d f f f d d f d d f d f f d d f f d f d d f f f f d
It's fairly easy to debug/modify the javascript to do this
Not sure how "free will" comes into play, if there was indeed free will, I could have freely decided to only press one, and the Oracle would have had close to 100% certainty.
Rather, what it measures is randomness or predictability, which is not the same thing as free will (especially "after the fact", e.g. after the choice is made).
The "free will" quote comes from a Berkley quantum computing student, who chose keys as independently of pattern as he could [1], and this seems to take it out of context. It might make sense to retitle this submission to something marginally less clickbaity, like "test how predictable you are."
Did they predict which method I'd wire it up with and the exact source of the program I'd use to do so?
I'm going to hazard a 'no', so it looks like I have free will.
Being quite unpredictable just means that you do a lot of random things -- but says nothing about how you came to will them.
If someone, for example, was threatened and ordered to follow for years what a program with millions of suggestions told him to do, he would be very unpredictable, but totally without free will.
And the inverse: one could choose to do the same routine all his life, and thus be very predictable, but it would still be his free choice.
But yeah, it's not the same as being unpredictable: that would look like seizures, not "gosh which ice cream flavor will he pick?:
The former is the metaphysical idea that all events are causally linked, a belief which we can never confirm or refute. The latter describes something's observable degree of predictability.
They're not related concepts at all. This demo has nothing to do with free will, Determinism, or Compatibilism - but it's an interesting study in predictability.
If we are talking about an abstract computing device, and whether it is deterministic, we mean whether its next state is always uniquely logically determined by the current state, correct?
Hmm, if an abstract machine behaves in a way that is uniquely logically determined, but which is not computable, do we consider such machines to be deterministic? I would think so, but I am not sure. We would be unable to predict it, does that make it not deterministic(computing) ?
Ghost in the Quantum Turing Machine https://arxiv.org/abs/1306.0159
RE free will itself, I'm thinking it has more to do with metacognition and general complexity of thoughts. People may have free will, but they're running the same "free will algorithm" - give them the same inputs, and most of the time you'll get the same outputs. A lot of people don't want to consciously accept that, but it's a fact. It's e.g. why economics works at all - because people are predictable at scale.
But when do we most often reference the concept of "free will" in practice? In evaluating degree of responsibility for decisions. "You did X, which was the most obvious and beneficial in the short-term option, but you could've done Y or Z, which would be better - therefore you will suffer consequences". Or, "I won't do X even though the situation tries to manipulate me into doing it; I notice I'm being manipulated so I'll do Y instead". Scenarios like these refer to the human ability of a) non-greedy optimization (long-term planning), and b) being able to go meta many levels up, to notice their own patterns of thoughts and use them as an input to the thinking process. A feedback loop over metacognition if you don't mind.
This is still all pretty deterministic. The apparent randomness - I think - comes from the fact that a) two people never have perfectly the same set of inputs, because the internal state is dependent on one's life history, and b) in some cases slight variations in those inputs cause huge variations in outputs (butterfly/hurricane and all that). In a way, the main connection between "free will" and randomness may be just that a human doesn't have capability to perfectly predict the thought process of another human. We can do that to inanimate objects, we can do that to algorithms we write - at least in principle. But we know that other humans will always suprise us (at least because they can notice we're trying to predict them and start acting random to mess with us).
That's not quite how economics work. It's more like, outputs from a given set of inputs can be characterized in a statistical way at scale (which is subtly different from saying that individuals are predictable at scale.) It's not that every person will choose to buy pizza if it's $11.99 but not if it's $12.00, but instead that if you offer pizza for $11.99 you'll generally get between X and Y sales per day, and if you increase that to $12.00 you can expect to lose about Z sales, and you can mess with that curve to try to maximize revenues or whatever.
I would say that what makes "free will" an interesting concept is that it's about how one chooses to prioritize different values -- how they express meaning by their choices. In the sense of a "free will algorithm", it's how you choose to weight different factors in your algorithm. How do you weight comfort, pleasure, stimulation, challenge, avoidance of pain, etc.? How do you weight those things for others? In practical usage, we talk about "free will" when someone makes a choice that surprises us, showing that their value system or their evaluation matrix is different from ours.
It means you were temporarily a statistical outlier.
df d
dd d
fd f
ff f
This algorithm is trivially defeated (chance of guessing 0) by the string
ffddffddffdd
Since that holds a pattern just long enough to get the algorithm to guess based on it and then switches.
For a little while, either intentionally or not, you must have been doing a more sophisticated version of this.
With perfect knowledge, it should be possible to get it down to 0.
> I couldn’t even beat my own program, knowing exactly how it worked. I challenged people to try this and the program was getting between 70% and 80% prediction rates. Then, we found one student that the program predicted exactly 50% of the time. We asked him what his secret was and he responded that he “just used his free will.”
> I wrote a review of Steven Wolfram's book a while ago where I mentioned this, as a basic consequence of Bell's Theorem that ruled out the sort of deterministic model of physics that Wolfram was trying to construct. I didn't call my little result the Free Will Theorem, but now I've learned my lesson: if I want people to pay attention, I should be talking about free will!
So, apparently, my brain's PRNG is pretty random. But I still have absolutely fuck-all idea whether or not I have "free will".
Fun two-and-small-change minutes, though.
Don't ask me what 'trying hard' means -- I don't know! :-)
Fixate on a number fairly late in the sequence (the millionths place seems to work well). If that number is 5-9, push 'f'. Otherwise, push 'd'. Keeps it pretty consistently around 0.5.
But seriously, the intuitive concept of free will, something like "unconstrained choice" leads to the idea that free will means simulating randomness - but once you're there, it looks more or less like nonsense - "acting randomly"
> In a class I taught at Berkeley, I did an experiment where I wrote a simple little program that would let people type either “f” or “d” and would predict which key they were going to push next. It’s actually very easy to write a program that will make the right prediction about 70% of the time. Most people don’t really know how to type randomly. They’ll have too many alternations and so on. There will be all sorts of patterns, so you just have to build some sort of probabilistic model. Even a very crude one will do well. I couldn’t even beat my own program, knowing exactly how it worked. I challenged people to try this and the program was getting between 70% and 80% prediction rates. Then, we found one student that the program predicted exactly 50% of the time. We asked him what his secret was and he responded that he “just used his free will.”
I was doing better by just trying to continually break whatever pattern I had fallen into, and occasionally throwing in some long predictable patterns (i.e. all the same key) to simulate the low chance of a long predictable string occurring. That had me down in the ~.4 range for a while. (Edit: Down to 0.23529 this try)
3.141592653589793238462643383279502884197969399375105
Now, the interesting question, how accurate is my recollection after all these years? Ack, one digit off!
0 1 10 11 100 101 110 111 1000 1001 1010 1011 1100 1101 1110 1111 10000...
(Granted, that is a transcendental number -- the Champernowne constant for base 2 -- but there's no real need to think of it that way.)
Why?
var simulateKeyPress = function(character) {
var evt = document.createEvent("KeyboardEvent");
evt.initKeyEvent ("keypress", true, true, window,
0, 0, 0, 0,
0, character.charCodeAt(0))
document.body.dispatchEvent(evt);
};
var num = 0;
var loop = function() {
console.log(num);
var binary = num.toString(2);
num++;
for(var i = 0, len = binary.length; i < len; i++) {
simulateKeyPress(binary[i] == '0' ? 'd' : 'f');
}
};
setInterval(loop, 100);pi (base 9) = 3.12418812407442788645177761731035828516545353462652301126321450283864034354163303086781327...
The keypresses I entered were not predictable by the Oracle. I generated the keypresses by first picking an arbitrary integer in [0,59]. I then (using simple mental arithmetic) used that integer to seed a PRNG by George Marsaglia [1] to generate a stream of pseudo-random decimal digits. For each digit, if it was in [0,7], then I took the 3-low order bits of its binary representation, in order from least significant to most significant bit (0=f, 1=d). If the digit was 8 or 9, I discarded it. The PRNG I used has period 59, and only 20% of the output digits would have to be discarded. Therefore one could use this approach to generate up to 141 pseudo-random bits.
[1] Marsaglia, George. "How to generate random number sequences (in your head)" https://groups.google.com/d/msg/sci.math/6BIYd0cafQo/Ucipn_5...)
See results at http://imgur.com/iaHBNNr ("Your nth attempt" is primes % 4, "Coin nth attempt" is the Marsaglia PRNG). Note that primes % 4 has too many runs, its prob(H given H) is too low, and its prob(H given T) is too high.
Maybe a collatz sequence?
You're right, apparently it's called Chebyshev's bias.
The 2/3 of the average guess game is an interesting illustration where if you take the game theory approach and recursively apply it you'll end up with 0 all the time: https://en.m.wikipedia.org/wiki/Guess_2/3_of_the_average
Instead of trying to pick randomly between f and d, I picked numbers between 0 and 9, with an intentional bias toward lower numbers, and those would determine the length of the next sequence. So if I picked 4, 3, 7, 1, 3, 2, 2, 5, 0, 4 as my numbers, that would translate to
ddddfffdddddddfdddffddfffff[0=don't change]ffff.
I suspect a more sophisticated program would be able to pick apart my strategy, but it worked really well against this particular algorithm.
.... Turns out it was because I was pressing the wrong buttons on my Dvorak layout keyboard!
After all, it's pretty impressive when it appears to correctly get Frank Sinatra out of "does your character play video games for a living."
(They were using the grating slogan "the neural net on the Internet" when neural nets were the pariah of AI. It's kind of amazing.)
Actually, an interesting challenge might be to construct a sequence that gives the Oracle as low a score as possible (is there a nicer simpler way to describe this than simulating the Oracle's code and always choosing what it doesn't expect?).
Some searching tells me this already has a name, see https://en.wikipedia.org/wiki/De_Bruijn_sequence
So yes, just take a De Bruijb sequence B(2,6) (algorithms in the link can generate them) and repeat it.
For example, the 4-bit Gray code (under this interpretation) when concatenated into a single bitstream begins "00000001", which then contains the 4-grams "0000", "0000", "0000", "0000", "0001", so the predictor gains an advantage right at the outset. (I didn't check whether it then loses its advantage over time... that's an interesting question.)
I'm not going to take the time to test it, but if you flipped a coin before each key press, this algorithm would eventually get to 50% accuracy. But in the short term it would probably be <50%, because it assumes it won't actually get random input.
So you may need to make a yes-no decision where you decide on yes about 65% of the time, when you consider lots of those decisions.
His solution? Glance at your watch. Divide the current minute into proportionate parts. See in which part the seconds hand is standing.
dd if=/dev/urandom count=1024 bs=1 | xxd -b
I mean, ever heard of compatibilism?
1. Unpredictability is possible even when the user doesn't have any choice about input (type keys based on a PRNG)
2. Predictability is possible even when the user can choose from both available options (human user)
It's an interesting study in our ability to be random, but it doesn't tell us anything about the macro-level philosophical concepts of determinism or free will. Compatibilism is not relevant to this demo.
Also, there is a huge assumption there - the assumption that there is a "self" that can have this free will. There is no self, just a stream of experiences and actions. The "self" is an intuitive concept, a reification that is useful in social dealings. Taking it from the social context and putting it in the realm of physics could be just a sleight of language - it cannot be the same concept in both domains, because the two domains are so far off. And trying to formally define a self in physics is impossible for us.
So does that mean it's not able to predict with more than 50% accuracy?
This is like showing someone an elaborate mirror, and saying, "See? My copy of you does exactly what you're doing to do! You don't have free will!"
All you have to do is put another mirror in the middle and show them that it doesn't work as they claim.
Actually the quickest way I found to "break" it and hover at ~50% is to simply aim for the dead space between the "F" and "D" keys. If it was using non-adjacent keys like "F" and "J" instead it would be better for its purpose. "Q" and "P" might work even better since it would mostly force you to use 2 hands, making it much harder to score close to 50%.
"In a class I taught at Berkeley, I did an experiment where I wrote a simple little program that would let people type either “f” or “d” and would predict which key they were going to push next. It’s actually very easy to write a program that will make the right prediction about 70% of the time. Most people don’t really know how to type randomly. They’ll have too many alternations and so on. There will be all sorts of patterns, so you just have to build some sort of probabilistic model. Even a very crude one will do well. I couldn’t even beat my own program, knowing exactly how it worked. I challenged people to try this and the program was getting between 70% and 80% prediction rates. Then, we found one student that the program predicted exactly 50% of the time. We asked him what his secret was and he responded that he “just used his free will.”
Maybe there's some randomness lying deep down in what we experience as free will, but that doesn't mean that we're 100% unpredictable in everything we do.
There's a fringe view (also help by some of these physicists) that "The more radical group holds that the agent who determines his own will is not causally influenced by external causal factors, including his own character."[1] Start with section 3.2 of the Stanford Encyclopedia of Philosophy and move on to perhaps the whole article or the referenced primary sources.
My take on the link between the Aaronson Oracle and the ultimate origin view follows like this.
Suppose that we take the radical definition of free will and try to devise a test that demonstrates an individual making a choice not influenced by external causal factors. One thing that comes to mind is to ask that individual to make a choice between two options. If the a choice is not causally influenced by external factors, then we can expect that it will not be predictable.
It's interesting that a number of commenters have reported prediction rates very close to 0.5. Typically in similar hot hand experiments[2] that's not the case and I suspect it's a reporting bias (why should those closer to 1.0 feel the need to report their findings).
There's also a bit of a counter-example in the whole space of pseudo-random number generation that's very much an example of a deterministic system with nothing but external causal factors, but will pass the Aaronson oracle test. On the other hand, truly random sources without any identifiable causal sources (ie radioactive decay) will also pass this test. Both of these passing support an argument that the test has no power to discriminate at all.
My personal take is that I like the idea of thinking about having free will as a spectrum or degree rather than a binary proposition. In the ultimate origin conversation, I don't see external causal factors as being completely avoidable nor unavoidable so having a test whose output is a range of predicability is nice to see.
This might easily be more about formal education in statistics than about culture, though. I'd expect people from any culture we know about to have the intuitive expectations I mentioned -- identifying randomness with chaos.
1. Totally clear your mind of the previous letter and make each letter a separate decision. That way you only have to avoid being biased toward using one letter or the other too much. This is probably pretty hard.
2. Allow yourself to be aware of your previous answers, but thwart your own biases by including patterns that cancel them out.
I went with the second option and consistently got between .45 and .55. I know our main bias around randomness is not realizing how many runs and patterns exist in truly random data. So I occasionally included runs like "fff" and "dddd" and, rarely, patterns like "dfdf" and "fdfd."
Free will is in play as and to the extent that my future is determined by my decisions, rather than by other factors beyond my control. For example, if I'm walking down the street, I have free will in the matter of whether I go North or South, but not in the matter of whether I go up or down.
I tried typing some letters because I was curious to see what the output would be. But for some reason, the page didn't work on my browser; no output was produced. Thus, my attempt to exercise free will was foiled.
As arguments against free will go... I think people with no short-term memory make a pretty solid case. You'd think with all the complex activity in our brains we'd at least behave a little randomly based on some chaotic processes, but without your short-term memory you begin to sound like a broken record.
I think its flawed - Whatever I press it never changes by more than 1%
How can the prediction for 'f' be 0.764 and for 'd' be 0.763 - they don't add up to 1.
Not exactly, seems to be a running average with a specific window.
It's making a (hidden) prediction about what key you'll press, and it's displaying the number of times it was correct over all your key presses.
So, the more you've pressed the keys, the less each successive press will change it's overall success rate.
Also it's broken in Safari.
My choice was taken away from me.
Has anyone done a control test with an RNG to see what happens?
It would be interesting to play with more modern branch prediction algorithms (e.g., Andre Seznec's recent work) with the same interactive interface. I haven't seen anyone do this before...
What, does the IRQ for 'F' fire first or something?
This algorithm should work on that as well... :(
d 3 times f 1 time d 4 times etc.
... and got 50% almost exactly.
I am using KeePass 1.31's Random Password generator with minus and underline. The oracle is getting 59%. Hmm. Also, when I generate a 64 character string with two possible characters, KeePass claims I have 72 bits of entropy. Hmmm!
But the most recent guess is at the top.
Often what I'll observe is they try to mimic a random distribution by selecting a number far away from their last selection, but sometimes they somehow catch on and start saying things like the same number (rare) or a number close to it. This gets interesting because they usually start cycling between picking a number far away and close, but they do so with a particular pattern. I'm able to guess their numbers well beyond a random distribution, and it all comes down the the idea that they are trying to make a random number or trick me, which produces a blueprint of behavior.
It seems likely that the probability of guessing correctly would be around 1/9, since people rarely pick the same number, but is a sustained 1/8 or better really possible?
Good forward planning to allocate a variable to hold a reference to the participant in case you need to refer to them later in the story, though in this case it turned out not to be needed. I need to start doing this in anecdotes. "So I was chatting with a colleague (C), and he mentioned..."
But I have to agree with the posters who say that free will and predictability are not the same thing.