We Know When We're Being Lazy Thinkers

Like everything from Science Daily, this is a pure reprint of someone else's press release. Here's the original from Springer:

http://www.springer.com/about+springer/media/springer+select...

and here's the article itself, from the lead author's website:

http://wdeneys.org/data/De%20Neys_substitution_revision.pdf

Incidentally, the article addresses mistercow's criticism: "For example, some critics might spontaneously argue that since our control bat-and-ball version is easier than the standard version our findings are trivial since they simply show that people are more confident when answering an easy question than when answering a hard question. It is important to stress that this critique is begging the question. The crucial question is of course whether or not people realize that the classic version is hard. That is, the control version presents the easier statement that participants are supposed to be unconsciously substituting. What we want to know is whether or not people note this substitution. If people do not notice it, then the two problems should be isomorphic and they should be considered equally hard. In other words, arguing that people notice that the classic problem is harder than the control problem underscores the point that they are not oblivious to the substitution."

I'm not sure quite how convincing I find that, for two reasons. Firstly, since it's possible to ask equivalent questions in different ways, it seems like someone might (1) genuinely think the question they've been asked is the same as the easy one, but (2) have found it harder to parse or something. Secondly and (I think) more importantly, isn't it possible that asking people about their level of confidence provokes extra reflection? Participants may have substituted obliviously, but then -- only when asked "how confident are you in this answer?" -- realised on some level, to some extent, that something wasn't quite right.

Quite of a funny coincidence - I was just discussing this with my colleague literally a few seconds before seeing this link.

We were both going through the same book recently - Thinking, Fast and Slow by Kahneman. Interesting read if you want to find out more about those phenomenons.

This seems to be extracting a lot of inference from not a lot of evidence. The difference in confidence could simply arise from the fact that the "2 dollars more" question sounds harder than the control question. It's a more complicated question, so it stands to reason that people would be less confident about it.

What I really want to know is this: how confident were the people who got the harder question right?