A solution to Newcomb's paradox
fakenous.net
fakenous.net
In other words, the problem as stated is that if you choose both boxes there will be nothing in the opaque box, and if you choose one box there will be a large payout in the opaque box.
Redefining the problem to say "they've already made the decision and filled the box or boxes, so your choice won't change the outcome" is sidestepping the point of the problem.
I personally wasn't convinced by every paradox solution offered in this book, but a few of the solutions were truly stunning, and left me with the feeling that some famous philosophical paradox had been completely and unambiguously resolved (esp. in the first chapter).
(I love shilling for Michael Huemer [author of this book/blog] because his philosophical books & essays have been extremely influential/clarifying in my own thinking).
The key to one-boxing is to commit to it early and don't waver when the time comes.
edit: Oh hang on, this is a machine, not Omega, so I can't say that Omega can just choose to not give you the offer if it can't reach 90%. Idk then. Probably it means that at least 9 out of 10 people don't throw coins.
If you recast the paradox as "There exists a machine scientists made that will determine if any other intelligence will either choose the box or not the choose the box. You are that other intelligence. Do you choose the box or not choose the box?" then you can see that the same principle holds; if you incorporate the logic of the predictor into your own intelligence, you can twist the original machine's logic back on to itself in exactly the same way the Halting problem does.
A subtlety to note about the halting problem is that while it is phrased in terms of that particular machine that can twist back on itself, it is itself a generalized proof of impossibility, and via Rice's theorem and lots of other work over the years it extends out into the impossibility of all sorts of other machines as well. The proof simply provides one machine that unambiguously can not be created, it is not limited to that one machine.
Similarly, while a human brain that encompasses the same logic as the box-prediction machine may have technical problems in the fuzzy real-world land, the fact that such a brain would be fundamentally unpredictable means that the prediction machine can't exist.
If you try to fix up the thought experiment by limiting the size of the predictor and the predictee in various ways, I suspect it isn't that hard to show that the predictor must be exponentially more complicated than the predictee in order to function, and something "exponentially larger" than the human brain doesn't really fit in the universe. And then if you try to escape by allowing arbitrarily large mathematical systems, you're back where I described above. If you try to bound how much larger the predictor has to be than the predictee, you are going to be encountering some really serious mathematical problems doing so (of the "busy beaver" mathematical sequence sorts).
Given that the problem intrinsically encompasses a delay between the scan and the decision, I can simply take an ad-hoc hash function of the experiences of the last day and now the predictor needs to have also had information of my entire last day as well, and a simulation sufficiently detailed to have predicted that, too. Even if it can predict the hash function I would use (itself no guarantee since that is also conditionalized on the intervening day, potentially), it can't predict the input going into it.
I think most people intuitively sense that the predictor can't really exist; I think people's intuitions are correct. It would require the predictor to have physically impossible amounts of additional resources vs. the predictee if you try to embed it in real space and time, and if you put it in math space it's nothing more and nothing less than the halting problem restated.
The paradox arises around the 100.00000000...% accurate algorithm.
Is it? As I said, my decision up-front is to flip a perfectly fair coin (or substitute in your favorite CSPRNG or TRNG). So your machine could have whatever claimed accuracy percentage, but I can't see how there's anything to predict beyond 50% and any machine that does so would have to be magical (i.e. outside the rules of science/mathematics).
I used that as a stand-in for my intuitive thought that such a machine's accuracy is time-dependent - the further out from the original scan you are, the more "random coin flips" your brain has experienced & thus you degrade very quickly to somewhere around 50% accuracy. You could try to mitigate that by having a super accurate model of how your brain works & trying to measure all the inputs it has encountered since, but ultimately the machine has to degrade to 50% at some point in time unless "magic". I guess you could do worse than 50%, but given this is a binary choice, any such result I think would mean you could just take the complementary result so the lower bound should always be 50%? Not sure about that last point.
A MIRI talk mentioned a hypothetical scenario when an AI gets duplicated. In this case the AI has a very high certainty about the state of its own duplicate. Also Nate argues that many everyday problems humans face are Newcomb-like. (And of course he talks about an AI that knows that humans know its source code.)
https://www.youtube.com/watch?v=3Sn0stHiNh4 (from 12 min 30 sec)
https://intelligence.org/2017/04/07/decisions-are-for-making...
Also, why you should 1 box: https://www.youtube.com/watch?v=Tzi3PCi0WBU
If it's just that 90% of people choose one, then it can choose that one and be right 90% of the time without even taking your brain scan into account. Or maybe 90% of males go one way and 90% of females go the other way, well that's an easy thing. Or maybe nobody ever chooses one when it's raining, and there's a 10% chance of rain tomorrow. Or, maybe it does a perfect simulation and knows exactly what you'll choose, but flips it 10% of the time just for fun.
Any of those would have different ways of optimizing, so it doesn't make sense to speculate without knowing exactly how it works.
The accuracy of the predictor does not impact the two-boxer's argument from causal independence, which I think is a serious problem for the two-boxer.
It doesn't feel right for there to be one argument that applies below 100% and another at exactly 100%.
Exactly. The reason to choose one box holds whether the predictor is 90%, 99%, or 100% accurate. I think lowering the accuracy of the predictor makes two boxing look more appealing though.
I think regardless of whether you agree with the author's conclusions about what is correct strategy, they're correct the paradox arises in that it pits the optimality of two different strategies against one another, where the strategies depend on the causal processes involved. I think the paradox raises a lot of questions about prediction versus causality and free will, and it's fair to ask (as another poster suggested) whether or not the machine in the paradox can ever exist.
The problem is that in the case where the machine is 100% accurate, it is still not causing anything. So although it's fair to conclude in that scenario that optimal strategy is the 1-box decision, it doesn't make sense from a causal perspective because you shouldn't have any causal agency over the decision the machine made.
Here's something interesting to think about: assuming the machine could exist in some possible world, how does causality work in that world?
To be certain about your future behavior, the machine would need a "time oracle" that allowed it to view the future. The machine consults its time oracle, it sees you making a box choice, and it puts money in the boxes based on the choice you made.
But this is literally you having causal agency over the machine's decision, isn't it? After all, it acted because of what it saw you do.
More importantly, regardless of how good the computer is, the difference is always $1,000. This is the c term in the expressions at the bottom. If the computer is always wrong, with 2 boxes you always get $1,001,000. If the computer is always right, you always get $1,000. And the one box strategy always averages $1,000 less.
"If it predicted that you would take only B, then they put $1 million in B. "
If it predicts that you will take only B, and they put $1M in B, and 90% of the time it's correct, why isn't the one-box strategy 90% likely to get you $1M?