Hacker News Reader

Comment by PeterisP | Hacker News Reader

Well, there's the old 'exploit-vs-explore' tradeoff, and it's well known in decision theory that without perfect information (and in real life, there's never perfect information) it's not optimal to always take the 'best' decision according to currently known information, sometimes you should take a potentially worse choice just to gain information about how good that other alternative is.

The multi-armed bandit problem (https://en.wikipedia.org/wiki/Multi-armed_bandit) is a theoretical simplification of this concept.

View on HN

In particular, the study mentioned here does not seem to take this into account.

In summary, it took the form of a simple game, for which many players deduced the optimal strategy. They then modified its behavior, without notifying the subjects, so that the optimal strategy remained the same, but was less successful, and many players strayed from the optimal behavior. So were they exploring? Had they simply lost confidence in their earlier assumptions about how the game worked? We don't know why they strayed, because "while the answer to that is beyond the scope of this study, Krajbich said, it likely takes a lot of mental energy and planning to always make decisions based on your knowledge of the environment."

Notice, in that quote, the willingness to speculate about a cause despite an experimental design that could not deliver evidence on the issue... (To be fair, I probably would not have noticed that inconsistency if Krajbich had picked a putative explanation that I speculate is more probable.)

Opportunity cost seems to be at least one of the factors it's not accounting for. If you know a strategy that produces a reasonable reward consistently, then trialing a strategy that produces a higher reward less consistently is a risk, and you can't know whether it will pay off in the long term until you've already committed to forgo the consistent rewards for the period of your trial run. This study could have just as easily been about risk aversion.

Then you also have the opportunity cost of what you choose to optimize. Perhaps making sub-optimal choices is fine in areas that aren't important to you for whatever reason. You have to make choices all day, you don't have enough time or energy to optimize all of them. If you want to succeed in whatever your highest priority goals are, then I'd suggest that you should only bother to optimize things that are actually going to help you achieve them. Devoting significant effort to optimizing things that don't matter is usually called bike-shedding, and I think a lot of us would have experienced how that can come at the expense of things that actually matter.

I'd also suggest that focusing on overly narrow set of factors can skew the results. The author talked about commute routes, and I know that my usual commute route is sub-optimal from a time perspective. I know there are routes that can give me 5-10 minutes in time. But I usually don't choose them, because I'm usually not optimizing for time on my commute. After a days work, I'll usually prefer to take the slow mindless crawl down the freeway, over the 20-left-turn route that gets me home some inconsequential amount of time faster.

Agreed. The risk of the decision seems to be very important in this experiment yet not mentioned. With high risk exploration is out of the question, but with low risk exploration is very logical behaviour (with potential high reward)

This phenomenon is observed in poker all the time.

this seems to be the underlying principle of creativity, and presumably intelligence and wisdom, as well. the brain makes wacky connections all the time and creativity emerges as a random subset of those that rise into the conscious mind.

That’s such a cool idea. I’ve never heard about it in that way before.

Thank you!

This is particularly important problem when playing poker.

Reply on news.ycombinator.com