The Marshmallow Test does not reliably predict adult functioning | Hacker News Reader

The Marshmallow Test does not reliably predict adult functioning

srcd.onlinelibrary.wiley.com

519 pointssuperposeur400 comments

The findings of the original study were called into question by a larger 2018 study[0]. The original study had 90 students. Some folks did a study with 900 people. They found the same correlation that the original study did. But when they controlled for household income, they found most of the correlation disappeared.

The obvious conclusion is that household income is a predictor of both:

- inability to delay gratification, and

- higher academic achievement

This makes sense when you consider that someone growing up in a poor household may have both:

- less reliable/continuous/predictable access to material things, meaning they would rationally seize immediate opportunities rather than taking the risk of a larger future opportunity, and

- less academic support

Now, this new study (OP) goes even further, finding that the correlation itself is weak.

[0] Watts, T. W., Duncan, G. J., & Quan, H. (2018). Revisiting the Marshmallow Test: A Conceptual Replication Investigating Links Between Early Delay of Gratification and Later Outcomes. Psychological Science, 29(7), 1159-1177. https://doi.org/10.1177/0956797618761661

Psychology is messy. If you assume that impulse control and the ability to delay gratification is an inherited trait, then the income of parents becomes supporting evidence rather than a confounder.

Time to do some GWAS to see if there is indeed a genetic component :)

Maybe the only possible conclusion is that you need much more specific experiments to conclude anything useful.

I believe that any experiment that involves the whole life of someone is doomed and useless.

Why do people make up inherited traits and apply them as if that is a legitimate critique?

The entire reason they did the marshmallow study is because most studies on impulse control cannot avoid confounding factors.

Time value money matters if I am offering you money now vs later. E.g. if you are in debt money later effectively involves the interest earned on that money at likely 25% or worst case 900%. If you aren't in debt the alternative is investing at 7% with risk or 2-5% without risk.

Trust is incredibly important. Money now is money now, money later might be money later if they actually fulfill the promise. And this isn't income agnostic as the risk of this varies wildly based on the impact of the money. A "get back on your feet" amount of money today or a slightly larger amount in a year implies a lot more risk than some spending money on either case.

Additionally while genetic markers have sometimes been effective at predicting even those have trouble with the random nature of gene transference.

All psychological traits are highly heritable. See eg https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4739500/

All traits, psychological or otherwise are heritable. The hard question to answer is how predictive those inherited genetics are relative to other factors.

Yes, but tiny variations in phenotype will foster rather largely different outcomes. People forget that a lot of your genetic traits are there as potential, but not as fact. Its not even necessarily and upbringing thing. Epigenetica systems, multi generation genetic markings (like famine). Its very hard to say how a certain genetic trait becomes an observed psychological trait.

> Time to do some GWAS to see if there is indeed a genetic component :)

Um, what exactly are you trying to say?

Seems like they're trying to say their could be a genetic component and a GWAS would help determine if that's true

Right, but I think in this case people mostly about whether there's causation. Because, if there is, then you can do an intervention to train willpower.

The obvious conclusion is not that obvious. You can have genetic traits which affect self-control, for example.

It doesn’t even have to be genetic. Parents which are able to raise their children to be functioning adults probably were raised by functioning adults and were able to find a job that leads to higher household income. We‘re talking about statistics here, so outliers are not relevant. Unfortunately this often prevents lots of meaningful discussions, because that would imply that a) it’s not just „you need to work hard to be successful“ (which one side of the political spectrum does not like to hear) and b) where and how you grew up is very predictive of how capable you are (which the other side does not like to hear).

In regards to a and b, wouldn't someone who thinks the former likely be someone who also thinks the latter? Those don't seem contradictory and, indeed, one is a possible explanation of the other.

I think in general, those divisions do exist.

People who lean right, tend to think movement from lower classes to higher classes is possible with hard work and that a person’s starting point doesn’t matter as much.

People who lean left tend to think where you start is the biggest predictor of where you will end up regardless of how hard you work. Hence the reasons one side favors the social safety net more than the other side does.

That has been my observation at least.

Yes, this is exactly as I see it - but as you can see in the downvotes many people very strongly think just one of these is true and very aggressively disagree with the other one.

That would be true if these issues were discussed in rational terms, but unfortunately because it's predominantly political, it means rational terms are not the basis of these discussions. That is presupposing either point A or B is even true.

I think that the difference is probably that one side thinks that being a parent that raises healthy functional adults comes down mostly to personal factors. The other side believes that societal/structural factors play a large part.

So they don't exactly disagree on what the circumstances for success looks like as much as they disagree on the degree to which those circumstances are under an individual's control.

The premise of personal responsibility is surprisingly partisan.

> Parents which are able to raise their children to be functioning adults probably were raised by functioning adults and were able to find a job that leads to higher household income. We‘re talking about statistics here,

What is a functioning adult, and where are those statistics from?

Start with a low bar. Since we are talking kids: married parents, employed.

https://eadn-wc01-3158345.nxedge.io/wp-content/uploads/2022/...

Why would marriage matter? Marriage may be an indicator of something, but there are plenty of successful single parents. I can think of several exceptions I know, all of whom are reasonably successful - one a somewhat well-known academic, one, last I checked, a rabbi somewhere, one, now deceased, a...actually, I forget what she did, but she managed to have a home in NYC and didn't come from a rich background, so she must've done something for a living (her kids were both academics.) When I can think of that many examples of reasonably successful people who break your rules without thinking too hard, there's probably something wrong with your theory, or at least my interpretation of it (sorry if I'm misreading what you wrote!)

It's a question of what is the rule and what is the exception. Single parent households correlate extremely strongly with many negative factors, relative to 2-parent households. This doesn't mean that somebody can't live a good life coming from a single parent household, but that on average they are much less likely to do so than somebody coming from a 2-parent household.

And while correlation is not causation, many of these factors are obviously causal. For instance 2-parent households will either be able to provide a much higher average income in the case when both parents work, or a more supportive environment when only 1 parent works.

> And while correlation is not causation, many of these factors are obviously causal.

You might think so, but the negative factors are sharply divergent between single parent never-married households and single parent widowed households.

Why would marriage matter?

"For instance 2-parent households will either be able to provide a much higher average income in the case when both parents work, or a more supportive environment when only 1 parent works."

I knowed a single-mom-family of 20 (twenty persons most under 21y), so if that mixed comment seem to look like 'that 2 parents build a safety-net for their children, if one becomes ill - for example - no the heck if you think how worse it is when a child got ill, 'horrifying!', think in that family of 20, on a regular basis _all_ got ill. At the same time. So you've to get a complete medical "lazarett"-team (doctor, sisters, helper,...) but to underline it (for the extreme... with the 'working parents') or there may be a need for an nanny-state-scenario... that may be called 'social'...

And for the original question: "Why would marriage matter", cos marriage seem to be often about rule and expections, not? Um But if i remember correctly, there were some academics, centurys ago speaking about "Moralstats" ("Moralstatistik" in German), where one finding was that not been married correlates with bad-tooth.

regards,

Jeff Bezos and Bill Gates would like to have a word.

Maybe that was misleading. The statistics I’m talking about are from the article, and the sentence before was just my guess - but I really think this is common sense, isn’t it?

I agree. There very clearly is a causation link between certain behaviors and long term success. Household income is something people have direct control over. I can go out and immediately cut mine in half tomorrow if I choose to. Doubling it would be harder but i imagine I could do that too if I took the appropriate actions.

They have a certain percentage of direct control and the remainder is what causes all the problems.

being able to double the household income is a privilege you have. what makes you think anybody could do it but is chosing to live with half the income instead ?

> you need to work hard to be successful

Hard work is only a path to success if you're working on the right things. For example, if I decided to be an Olympic athlete, and worked like the devil, I have zero chance of making the team.

Yes, and likely with environmental reinforcement. High income/high IQ -> self sorting, and assortative mating. That will lead to households with kids who have both genetic traits, and an environment that's going to teach and reward self control.

people really don't want to accept that beyond the most extreme cases(starvation, lead poisoning, complete neglect, no school access at all, etc.), environment really doesn't play that big of a role. Twin studies have shown this for literally decades

Lead poisoning isn't an extreme case, it's a very common problem.

Twin studies don't exclude environmental causes because twins have the same maternal environment.

Yeah, I never got how the womb doesn't count as an environment, and even the circumstances that led to the twins being separated.

While commonly thought, this isn't how twin usually studies work. The way it works is you look at the correlation between identical and non-identical twins on some given thing. If there's a much stronger correlation between the identical twins, then it's probably primarily genetic. What this does is helps to eliminate environmental factors because identical vs nonidentical twins will both be raised in basically the exact same environment.

So take height. If identical twins have identical heights while non-identical twins have varying heights, then it's safe to assume height is largely genetic. Interestingly separated twins would actually be worse in many cases because you reintroduce environmental deviation. For instance with height, differences may well be down to e.g. nutrition, but when you have them in the same household you can usually assume roughly identical nutrition.

That's not true, for example metacognitive ability studies have shown environment plays the dominant role. Twin studies on trust provide the same, in which genetic component while large at 33% certainly doesn't indicate what you're stating that it's only "extreme cases". Even in studies re-assessing conventional twin studies and educational attainment, the conclusion was that while some is genetic (sometimes even a large portion) the correlations between a mother and father's educational attainment points to environment playing a large role (unless you have the belief that the mother and father are siblings I suppose).

You'll be extremely hard pressed to find researchers conducting these twin studies who minimize the role of either genetic or environmental impact on certain aspects in the way you did.

There's a tricky (and super interesting) thing with IQ studies. Environmental factors play a dominant role early on, but genetics becomes more and more dominant as a person ages; significant privilege or disadvantage earlier in life notwithstanding (excepting major physical impairment by nutrition, lead, etc)! Most studies tend to find the heritability of adult IQ at around 80%.

Any research on this area is walking on egg shells and so researchers are highly incentivized to overemphasize possible environmental explanations. Nature formalized this threat/risk with their relatively recent announcement [1], but it seems to have been an unspoken 'rule' for decades at least.

[1] - https://www.nature.com/articles/s41562-022-01443-2

Heritability is not genetic determination; your environment is inherited as well.

The Nature link you've provided doesn't address anything you wrote in your comment.

Heritability is, literally by definition, the measured difference in some value (like IQ) between people that is attributable to genetics alone. There's quite a lot of clever ways to control for environmental factors, like twin studies. And I simply think you're not engaging in good faith whatsoever if you don't see how that Nature article creates a huge chilling effect on any discussion of genetics.

Heritability is the ratio of genetic variation to total variation. Lipstick-wearing is highly heritable; the number of hands and feet you have is not. As I said, and as you can see, the heritability of a trait tells you nothing at all about its genetic determination.

I don't know what to tell you about the cite you gave, since it simply doesn't say what you said it says.

You sound like you're thinking of things colloquially, to put things in a kind way. Heritability is not a colloquial term and in biology/genetics refers exclusively to genetic factors. In particular it's the percent of difference in some value that cannot be explained by non-genetic factors. So e.g. if my IQ is 130 and yours is 90 then we'd have a difference of 40 points. With an adult IQ heritability of 80% we'd expect that about 32 points of that would be unable to explained outside of genetic factors.

Heritability does change over time because environments change over time. For instance in an area where starvation, lead poisoning, and malnutrition was common, the heritability of intelligence (or height) would generally be quite low, because the aforementioned environmental factors would be able to explain a large chunk of the differences between populations. But in a society where everybody had practically identical relevant upbringings and opportunities, the heritability of intelligence would be 100%, because the only difference between people would be genetic.

One of the many ways to test for heritability is twin studies. You'll likely find the correlation between lipstick wearing between identical and non-identical twins would be near to 100%. This means that the heritability of the trait would be near 0%.

No, I just provided you the literal technical definition of "heritability". It should be immediately apparent to you why lipstick is (highly) heritable and hand count isn't†; if it isn't, you're the one working from the "colloquial" understanding.

† lipstick: highly dependent on XX vs. XY; hands: set by highly conserved Hox genes, variation virtually entirely due to environmental factors.

I'm going to assume you're not trolling, and I am also going to assume you're the type of person that would take Wiki as a reliable source, so here you go:

---

"The concept of heritability can be expressed in the form of the following question: "What is the proportion of the variation in a given trait within a population that is not explained by the environment or random chance?"[2]"

---

The within a population part is critical. I'd encourage looking up twin studies to understand one way this percentage is estimated. I assume you think you know how they are done - you do not, because it directly leads into an understanding of how things like lipstick wearing would be near 0% heritable, while things like handcount at birth would be near 100% heritable.

[1] - https://en.wikipedia.org/wiki/Heritability

I'm not making up that lipstick is heritable; it is a classic example of a non-genetically-determined heritable trait, as is hand count (in the other direction). I think you need to do some more reading: you are clearly using the term as a synonym for "genetically determined", and that is simply not what the heritability statistic tells you.

For what it's worth, you can mechanistically work out lipstick and hands from first principles; just plug rough numbers into the formula.

When you say things like lipstick wearing is a classic example of a non-genetic heritable trait, you sound like a guy claiming that a cat is a classic example of a non-reptilian reptile. It's somewhat of a contradiction in terms that doesn't make much sense.

"Heritability is a statistic used in the fields of breeding and genetics that estimates the degree of variation in a phenotypic trait in a population that is due to genetic variation between individuals in that population."

You are using the term in a colloquial or folk sense. Do you think accents are highly heritable? After all, a child will almost always share the accent of their parent. So in the colloquial sense this would be a 'heritable' trait, but in the genetic sense the heritability of an accent is zero.

That’s not what the twin studies have shown. It’s not only the most extreme cases, it’s anything short of the very good circumstances. For example, the stable homes families have to prove they have to adopt.

Cyril-f “twin”. https://www.robkhenderson.com/p/no-one-expects-young-men-to-...

Wut? The causation can flow the other way as well. Having high tike preference results in lower household income. And time preference is probably genetic. They literally controlled for the variable they were testing for....

Why is it instead not obvious that delayed gratification is a predictor of household income?

How would a 4-year-old's ability to delay gratification increase their parents' income?

Perhaps if they inherited the ability from a parent, the parent is more likely to have an income as a result of investing in their education for example.

I mean, maybe, but this is definitely doing causal modeling backwards

Yes, it's possible that there are strong genetic predictors of household income, as a lot of people seem to want there to be for some reason, but when predicting the behavior of a child, their immediate circumstances are a much more parsimonious explanation for their behavior than some genetic factor strongly predicting both the circumstances and the behavior. I'm not saying that genetics being somewhat causally upstream of income is an inherently bad hypothesis, but this kind of correlation analysis doesn't support it as well as it does an environmental influence on time preference

This is the crux of the controversy. Some people think behaviors have 0% genetic inheritance and some people think it’s >0%. To assume low income parents can only cause low future orientation, but not the reverse, you must be in the former camp.

Some people desperately want poverty to be due to individual moral failure, as opposed to a systemic failure.

> Some people desperately want poverty to be due to individual moral failure, as opposed to a systemic failure

There are zealots on both sides. Brilliant people are poor because they were never given an opportunity while rich nincompoops accumulate aristocratic power. At the same time, plenty of people are poor because they can’t make good decisions or have zero emotional self control while a few go from rags to riches. The problem is blended, and it shouldn’t be beyond reproach to question whether some factors are heritable, whether genetically or through cultural transfer.

But one of those opinions flows power from the many to the few while the other the opposite. It is in and of itself political.

It of course shouldn’t be beyond reproach to do the research but it seems reasonable to be more critical of research that implies some implicit reinforcement of the current power structures because that’s what we’d get from bad research too.

> one of those opinions flows power from the many to the few while the other the opposite. It is in and of itself political

This is true of many things. That doesn’t mean asking the question is tainted. Anyone using either hypothesis as the basis for policy is similarly flawed in my view.

> seems reasonable to be more critical of research that implies some implicit reinforcement of the current power structures

There are massive power structures that benefit from the promulgation of either hypothesis.

> Anyone using either hypothesis as the basis for policy is similarly flawed in my view.

This is a tacit endorsement of the current power structures. I don’t think that is _wrong_ just a political position. One I agree with which is uninteresting given the forum.

Your second hypothesis is one we might test. Can we formulate an experiment that asks how often brilliant minds cross class boundaries? Or idiots bring their house down?

> This is a tacit endorsement of the current power structures

Not really, rather than assuming either theory is right just test giving resources to poor and sees if it improves outcomes. If it does and the resources were well spent, preventing people from becoming criminals or other burdens on society has massive value so continue and maybe do more of it. If it did little then don't, why spend on stuff that doesn't help?

That is much better than just assuming one is true and implementing measures mindlessly, like many governments do today. For example there is no evidence that diversity training improves any metrics, yet it is still required by many governments.

When people just assume one explanation you get a lot of effort put into things that doesn't improve the lives of anyone.

And some people are open to the possibility/probability that personal actions and choices have an influence over the propensity to experience poverty, in order to understand (and intervene where feasible) to break the observed cycle in a structural way.

The four-year-old's income predicts the four-year-old's low time preference.

Because we have no evidence of a genetic predictor for household income (or delayed gratification capacity). We do have evidence of class mobility.

More simply: people who grow up hungry learn to eat whenever they can because being hungry is awful

It doesn’t even need to generalize. This is just a basic food security thing and is part of the reason why obesity is counter intuitively common among people who suffer from food insecurity.

Is that from the study or your interpretation? Being hungry is a lot more of a threshold thing than income.

I’m not sure what you’re trying to claim. Poverty is obviously well correlated with food insecurity.

That depends on the country. In rich countries it is rare for poor people to go hungry because food is cheap compared to median income and poverty is defined relative to median income.

Counterpoint: I've visited the US. Poor children go hungry there, at least in WV. I've talked with a teacher who baked white rice everyday to give it to 3 children in her class, since they couldn't afford the school's restaurant and didn't had lunch.

Every public school in the US has free school linches for those in extreme poverty. This is especially true in more poverty striken areas (like West Virginia):

https://www.fns.usda.gov/nslp

Some of these programs require household application, which may be to high a barrier for some students.

That teacher's support might be vital to a hungry 8yo whose parents can't/won't/don't fill out the form?

Call CPS if it's that bad

"Extreme poverty" in the US is such a crazy low bar though. A huge portion of America is literally working poor. Not technically under any poverty bar, but that's not enough to pay rent and also buy enough food for everyone.

For those people, the $2 a day that school lunch costs (or even 50 cents for reduced price lunch) is literally too much money, because it is. So those kids go hungry. I knew plenty of those kids that all of you are insisting don't exist (for some reason) and no, the existing welfare is just not enough, even in an extremely low cost of living area, in a state with significant state level aid.

The bar for free school lunches in America isn't "extreme poverty". You've responding to HideousKojima's characterization of the program as though it is accurate, but he's wrong and if you're actually familiar with the subject matter you should have recognized that he's wrong.

Most working class families qualify for free or reduced price lunches. If you're at or below 130% of the poverty line your kids get free lunch. At or below 185% of the poverty line gets kids a reduced price lunch. How many children a family has is part of this calculation; the poverty line is set higher for larger families. This program spends something like 5 billion dollars a year giving lunches to tens of millions of kids. It's not a niche thing few qualify for. There are many schools where virtually every student is in the program.

Furthermore, schools will not allow kids to skip lunch. They are given lunch whether or not they have money and the matter is settled with their parents later. The few times I tried to skip lunch in school it turned into huge dramas and food was given to me even though I had chosen to not eat it.

As far as I know schools provide free lunches to children that need it.

In my area it is a school policy that a kid would not be denied food even if they had no money in their account - and every place I lived had organizations that would eagerly step up to address such a problem if and when it manifested.

For what it is worth, during COVID and a few years after, in my state every kid got free lunch in school regardless of their income.

I'm pretty sure the parents have to fill something for their children to have lunch, and I'm pretty sure the parents were too gone to care.

I met the teacher following a photograph who tried to document the opioid epidemic, crica 2018 (we were young and naive, video is probably the only communication medium that's worth anything when you're independent, photography is harder), while I kayaked/hiked/rafted/climbed everywhere I could for the two month I was there. I think she work for a magazine now. And I'm still convinced West Virginia is only lacking a huge lake or a sea to be the best place on earth.

what I am calling out is that it feels like you are generalizing after learning about one specific and possibly isolated incident.

it is very uncommon to have children go hungry in America, because food is plentiful, and numerous charitable organizations would eagerly step in.

Bad parenting may happen everywhere in the world.

For some brownie points, googling seems to indicate that Germany has a hunger crisis:

https://www.cnn.com/interactive/2017/09/world/germany-food-b...

It is rare for poor people in rich countries to starve to death, but missing meals and going through periods of hunger is unfortunately not uncommon at all.

In a recent study by the USDA, Household Food Security in the United States in 2022[1], it is estimated that "17.0 million house-holds were food insecure" (meaning that "they had difficulty at some time during the year providing enough food for all their members because of a lack of resources") and "5.1 percent of U.S. households (6.8 million households) had very low food security" (meaning "food intake of some household members was reduced, and normal eating patterns were disrupted at times during the year because of limited resources").

That's 17 million HOUSEHOLDS that struggled to provide food and 6.8 million HOUSEHOLDS that had to skip meals. I wouldn't call that rare.

[1] https://www.ers.usda.gov/publications/pub-details/?pubid=107...

>In rich countries it is rare for poor people to go hungry because food is cheap compared to median income

People say this as if we don't have direct statistics about how hungry poor americans are.

Maybe your ASSERTION that poor people should have enough food because "it's cheap" isn't right and you should investigate that.

Clearly, but they're saying that only attempts to explain why those particularly in poverty had the associated results. When referring back to the original study's actual claims it was more than just correlation with poverty incomes, it was claimed for all incomes and not so obviously linked to food insecurity.

OP says:

"Some folks did a study with 900 people. They found the same correlation that the original study did. But when they controlled for household income, they found most of the correlation disappeared."

So original study didn't control for income. If the original study claimed it across all incomes, and then it mostly went away when others controlled for income, then the delayed gratification strong correlation wasn't really for all incomes, right?

Food insecurity is in the definition of poverty in most places.

No, you are the one claiming something in your original comment. What the person you are responding to is insinuating is that what you are claiming is more of the same pseudoscience as the original study.

Could you list some specifics because I cannot follow you at all

This is your comment:

> More simply: people who grow up hungry learn to eat whenever they can because being hungry is awful

> It doesn’t even need to generalize. This is just a basic food security thing and is part of the reason why obesity is counter intuitively common among people who suffer from food insecurity.

Share the study that backs your assertions. If you don't have a study then everything you've claimed has no scientific basis.

https://pubmed.ncbi.nlm.nih.gov/37040535/

> Share the study that backs your assertions. If you don't have a study then everything you've claimed has no scientific basis.

How about “that sounds interesting, could you cite a source?”

yeah but that's too simple. as an adult who wants to live as long as possible, if I have to choose between eating one meal today or being able to eating two meals next week, I'm going to ask who the psychopath is that's imprisoned us, because we live in normal world and why would I ever be in that sort of a situation? Seriously, think back to as early as you can possibly remember, then as the marshmallow question. then ask it again to yourself, again and again, until your parents bringing you into a strange room with some weirdos, saying these are your parents now and have a marshmallow, isn't traumatizing

An extreme personal observation of this was my grandmother, who grew up during the Great Depression with 9 siblings. They managed better than most and helped out neighbors (having both land and descending from farmers); she would in her later years (when I knew her) have massive stockpiles of processed long-shelf life food stored everywhere. Also, all the pickled/preserved food from her garden that was left over. For the longest time, I thought it was common to have multiple freezers, and a couple fridges, in a garage stocked with food. Including, months old leftovers from cooking enough of whatever soup/stew for 4-5 full families.

It's true but it winds up carrying in weird ways. (my family history is awkward but suffice to say the main reason we weren't called out for being broke all the time was a factor of my mother being very involved in communities... perhaps it was her penance for her actions in causing the situation in the first place?) On top of my family history, treatment for what was then-misdiagnosed ADHD caused me to literally lose just about everything but my job and car in 2011... Lots of months eating ramen, cereal on sale, and baked beans.

As an example: my parents bought used Dodge/Plymouth/Chrysler products and my brothers first used Jeep wound up being a hot mess that caused a large blast radius. I purchase the cheapest 'new' vehicle that fits my needs [0] and only bought one used car (as a backup for all the fun accidents my ex-wife got into.)

As another more painful example: My first few years in the workforce alongside student loan debt, then alongside the 2008 crash, on top of my adolescent observations an helping my now-ex-wife through college, caused me to wait way way way to long to start contributing to my future retirement.

Semi-positive counterpoints:

1. I buy stuff that lasts, it takes more research and sometimes more up front but as I get older it saves me more and more money compared to people who live in a more disposable culture. I'm not afraid to shop/wait for deals and I make sure to think about every major purchase I make. I take good care of stuff I own.

2. I've been able to learn how to fix a -lot- of stuff (working at a bike shop helped) and it has both saved me money and save waste in general.

3. I can fit all of my mementos -and- important stuff including work desk (aside from bed/couch/etc) in a portable storage unit if needed.

[0] - Except the WRX, that was a 'my life is in a terrible spot but I survived a year'. OTOH I got a base model with only a couple options and it was <30k before taxes.

> part of the reason why obesity is counter intuitively common among people who suffer from food insecurity.

Bigger elephant in that room is the nutritional content provided to people in that category as well as access to that nutrition.

True, but we are living through an obesity epidemic, and it’s not just the rich people who are fat.

In Table 3, being white is a very significant predictor (p=0.007) of being able to wait at least 7 min, but in Table 7 they don't report white among the races at all. Does this mean the difference among white kids is entirely explained by SES and other covariants? Conversely, does this mean being black has an effect not explained by other covariants? That seems pretty controversial.

> Conversely, does this mean being black has an effect not explained by other covariants? That seems pretty controversial.

Controversial? There's a major effect in every context. It'd be hard to get less controversial.

>> less reliable/continuous/predictable access to material things, meaning they would rationally seize immediate opportunities rather than taking the risk of a larger future opportunity

THIS — the environment definitely changes what is the most rational behavior.

In economics, this is Counterparty Risk — the risk that the other party will fail to fulfill their obligation. E.g., as a vendor it is rational to accept a piece of plastic from a complete stranger without a word, because the issuer of the card is good for the money, and has taken on the problem if the buyer doesn't pay their bill that month.

For kids in affluent stable households, it is rational to expect that they'll get the second cookie in 20 minutes.

For a kid in an unstable household, being told by someone who neither looks nor talks like they do, that they'll get two cookies in 20 minutes, it's often rational to take what you can get NOW.

The marshmallow test measures mainly environmental counterparty risk in everyday events.

Seems when controlling for those factors, the 'marshmallow effect' disappears.

This is good science. Discover an effect. Generate a hypothesis. Keep testing until you find the limits of that hypothesis, and/or hidden variables.

It's more of the same slop we've seen for decades from the social sciences. We know most traits are heritable. We know that low time preference yields higher lifetime earnings. Of course an experiment is going to show no effect when it controls for the effect! What else would you expect?

And they never factor in time and place. Cultures vary annd change. Every 10-20 years the US is a different country.

Yeah. The reality is exactly the opposite of the headline. The study is confirmed, not refuted. Low time preference is heritable no matter what you believe the mechanism might be.

I don’t know about children but it makes sense that poorer adults have worse impulse control. Poorer people have more worries and less to look forward to. Maybe a 12-hour workday rather than an eight-hour one.

I’m sure that overworked wealthy people have plenty of vices, i.e. “poor impulse control”. But they have successful careers so those things are coping mechanisms, really. It’s compartmentalized.

In any case my cynicism would just be vindicated if a study just turned out to rationalize (as an emergent property because, duh, there is a population overlap between researchers and this group) the position of the upper-middle class.

The acoup blog series on medieval agrarian societies made an interesting point that resonated with me.

Things that look like failures in long term planning (to people with resource surpluses) can actually be optimal decisions (to people without resource surpluses).

In that case, it was the observation that maximizing currency profit from farming in a society that experiences arbitrary taxation and repeated famine is useless -- if no one has food then no one will sell you any, assuming your saved currency hasn't already been seized.

In contrast, optimizing for familial and social ties were much more reliable ways to see yourself through then-common perils.

If you're poor and don't have a stable living situation, delaying gratification presents risks to you. (ones that people with money and stability don't have to consider)

Sometimes people are dumb, but sometimes they're optimizing for factors others are oblivious to.

People might reply, "Well, take the risk," without acknowledging that risk is two-dimensional. There's a range of success and and range of consequences. Something traditionally considered "high risk", like investing millions of a billionaire's money in a start-up, might have low consequences for the direct investor. It sucks if the start-up fails, but they're still a billionaire, probably. Their lifestyle doesn't change substantially. Compare to a low-income worker choosing a car: maybe a cheap used one that could break down at any time, or a more expensive one is less likely to break down (if you stay up-to-date with expensive maintenance), or a much more expensive new car that is unlikely to break down (but that puts them in a substantial amount of debt). In every case, there's a way for the prospect to go sideways in a manner that would likely end with the worker losing their job, with (statistically-speaking) no savings cushion. However you rank the risk of each (at least one being the lowest financial risk), you have high negative potential consequences.

My impression is that people who say "take risk" literally always mean "take bets where you have little to loose and a lot to gain". And they look down on people who took actual risk and lost.

What "take risk" means is that you should try to be entrepreneur in situation where you can fall back and be well paid programmer again if it does not work out. Or that you should risk someone elses money.

EDIT: I guess good rephrase would be that "take risk" usually means "overcome irrational fear when you are in perfectly safe situation". That is what actually people mean.

This nails it. If Elon Musk loses 220 billion, he still does just fine. If a subsistence farmer in a remote place loses his couple hectares cassava crop, he and his family risks famine.

Wow, that’s an insane fact. Right now, the internet says he’s got $232 billion. So he could lose $231.9 billion and still be really really rich.

They also look down on people who could not take the risk, calling them cowardly, lacking boldness, entrepreneurial spirit, etc.

"Sell everything and fund my startup" is fine when you've got a 12 month emergency fund, living in a paid-off home, , and you parents are able to bail you out. It's not fine when failure means you're penniless and homeless.

Further: it's much less of a risk when you have a metaphorical rolodex of wealthy friends from university or business school. You're not cold pitching - not even slightly. Doors open for you, instead of being slammed in your face.

Yet they'll tell you shit like "Try hard enough and you will succeed! I worked long and hard, people saw that, and were willing to invest. Good hard honest work is rewarded."

The people working on their startup 12 hours a day who don't have connections have no idea that the bit left out is that "people were willing to invest in my idea because we chugged beers together in the basement of Sigma Chi."

> Something traditionally considered "high risk", like investing millions of a billionaire's money in a start-up, might have low consequences for the direct investor. It sucks if the start-up fails, but they're still a billionaire, probably.

That's not so much about being a billionaire, but about diversification.

Fortunately, even people of very modest means have access to diversified index funds these days.

If you have a few thousand stocks in your index fund (eg like VWRA or VT), then in doesn't matter how risky any individual stock is (like the startup in your example), as long as holding them has positive expected value.

Ref: https://acoup.blog/2020/07/24/collections-bread-how-did-they...

> But because these households wobble on the edge of disaster continually, that changes the calculus. These small subsistence farmers generally seek to minimize risk, rather than maximize profits.

> [...] Consequently, for the family, money is likely to become useless the moment it is needed most. So while keeping some cash around against an emergency (or simply for market transactions – more on that later) might be a good idea, keeping nearly a year’s worth of expenses to make it through a bad harvest was not practical.

> In that case, it was the observation that maximizing currency profit from farming in a society that experiences arbitrary taxation and repeated famine is useless -- if no one has food then no one will sell you any, assuming your saved currency hasn't already been seized.

> In contrast, optimizing for familial and social ties were much more reliable ways to see yourself through then-common perils.

Social ties are no more helpful in a famine than currency is. People who can't sell you food can't give you food either. They are insurance against the case that your crop fails, not that everybody's crop fails.

Currency is fine in that scenario. The tendency not to grow the most profitable set of crops is a method of avoiding crop failure, not a method of mitigating it when it happens.

From Terr_'s helpful digging up of the link: https://acoup.blog/2020/07/24/collections-bread-how-did-they...

See "Risk Control", "Banking the Yields", "Banqueting the Yields" for why currency was a terrible idea and relationships were superior.

Under "Risk Control":

> Avoiding risk for these farmers comes in two main forms: there are strategies to reduce the risk of failure within the annual cycle and then strategies to prepare for failure by ‘banking’ the gains of a good cycle against the losses of a bad cycle.

> If you only farm one crop (the ‘best’ one) and you get too little rain or too much, or the temperature is wrong – that crop fails and the family starves. But if you farm several different crops, that mitigates the risk of any particular crop failing due to climate conditions, or blight (for the Romans, the standard combination seems to have been a mix of wheat, barley and beans, often with grapes or olives besides; there might also be a small garden space. Orchards might double as grazing-space for a small herd of animals, like pigs). By switching up crops like this and farming a bit of everything, the family is less profitable (and less engaged with markets, more on that in a bit), but much safer because the climate conditions that cause one crop to fail may not impact the others.

Under "Banqueting the Yields":

> The most immediate of these are the horizontal relationships: friends, family, marriage ties and neighbors. While some high-risk disasters are likely to strike an entire village at once (like a large raid or a general drought), most of the disasters that might befall one farming family (an essential worker being conscripted, harvest failure, robbery and so on) would just strike that one household. So farmers tended to build these reciprocal relationships with each other: I help you when things are bad for you, so you help me when things are bad for me.

You might notice that this is exactly what I said above. What did you want me to see?

> Social ties are no more helpful in a famine than currency is. [...] Currency is fine in [a famine] scenario.

Social ties are more helpful than currency, even in a famine, because at that point resources are shared based on relationship rather than value. Triage, not optimizing future gain.

See also all the bits about why currency at that point was a terrible store of value.

Currency wasn't a terrible store of value. The essay says as much:

> Ok, so why not sell the grain and store something less perishable, like money? Sure, you can’t put it in a bank, but you can just keep it. And indeed, our ancients do this

The problems with currency are listed as:

1. Someone can come into your house and take it.

2. Ordinary farmers cannot afford enough currency to replace the value of a failed harvest. (An entire year's revenue.)

3. In the event of a famine, the price of food rises, making currency a poor hedge against famine.

Only #3 applies to how valuable currency is in a famine, and it remains superior to local social ties when the famine strikes.

But because of its unaffordability and undesirable correlation with famine, it is not the favored solution to famine. It's just a much better solution than local friendships.

You read the wrong essay; the standard defense against famine is discussed in the following one:

> So we’ve established what the big landlord gets out of working with the smaller subsistence farmers around them – they get labor to put more of their large holdings under cultivation and even a degree of labor flexibility with wage laborers and sharecroppers drawn from the existing rural population. In this sense – and I want to stress this – the large estates need the rural small farmers to survive. This is why, even in periods of rapid growth among large landholding estates (like the steady expansion of latifundia in the Roman late Republic and early Empire), there remained lots of these smaller farmers. But what do the small farmers get?

> Just like the smallholders could establish horizontal ties with fellow small farmers, they could also try to establish those ties with the big fellow in the big house. Of course the mechanisms for establishing the ties were different: few peasants could banquet an aristocrat and most aristocrats would be insulted by a suggestion of an alliance through marriage. Instead the ties were strictly vertical – that is they were unequal. They often began with the farmer working at least a bit as a tenant on the big farm, but also typically included political support, sometimes military support (that is, coming out to fight when the large landholder did, often as common troops in his retinue) and no small amount of social deference.

> In exchange, in theory, the large landholder could provide the ultimate backstop against catastrophe – even a catastrophe that might ruin an entire village or rural area all at once.

In the event of a famine, the farmer has two options:

1. Leave the stricken area, in which case he's dependent on his stock of currency.

2. Hope for relief to be delivered by his social superiors, which depends on their stock of currency.

> I’m sure that overworked wealthy people have plenty of vices, i.e. “poor impulse control”.

I once watched a movie about this called The Wolf of Wall Street

"worse impulse control"

or better grabbing-opportunity skills. If you repeat the experiment and the "doubling" of the marshmallow turns into "a teen barging into the room and stealing the marshmallow", who'd be the wiser kid?

The teenager is a good metaphor for what your creditors will do when they find out you have enough money to make a payment.

A factor in this study that I don't know was mentioned in 'trust'. Did the kids trust the adult to deliver on the promise of the extra mashmallows. If the kids had low trust in adults, its very rational to take the marshmallow you see rather the ones you don't.

This is very insightful.

Great insight. This also correlates with household income, as promises are more frequently broken with lower income.

Are you sure? Why? And with what statistics?

>In 2008–12, the rate of violent victimization was highest for persons in poor households (39.8 per 1,000) and lowest for persons in high-income households (16.9 per 1,000) (table 1). This pattern was consistent across all types of violent crime.

I also wonder, I think there probably a continuum between patient academic strategist and like driven tactical disruptor. I would say I deploy lots of tactical impatience to get shit done.

It makes me think of the scenario where mom gives the kids each a marshmallow. You decide to keep yours for later, the other kid eats theirs.

Later you decide to eat your marshmallow, but the other kid sees this and demands half. He goes to mom and she makes you share.

Lesson learned: either hide what you have or don't delay gratification.

I feel like this scenario is becoming more common in (US) politics these days (eg student loan forgiveness, housing bubble in 2008). Or it could've anyways been happening and I just didn't notice.

Poor people are better at cash flow management. That's why poor people like the dollar store. They're not dumb. They know you can buy soap in bulk at Costco, but buying a small $1 bottle is better than the 55 gallon barrel to manage cash flow.

Likely this is related to the marshmallow. I need to eat. I will eat it now. I cannot guarantee the marginal return on more marshmallows. I mean maybe if real life was like "wait one minute and double your money" people would do it, but typically it's like, lock up your cash for weeks, months, years at a time for margins, not for doubling your marshmallow count.

In real life, realized savings or gains of 1 or 2 or 4 percent for a 6 month wait is not worth the RISK of locking up that marshmallow (or T-bond) when having that marshmallow locked up may result in say, no housing.

This was posted 53 minutes ago and no one has given us the Vimes Boots Theory of Economics yet?

Don't believe everything you read. Fiction isn't real.

There's an opposing wise saying too: https://kottke.org/10/10/andy-warhol-on-coca-cola

Coke isn't comparable to boots in this case. Unlike boots, there is relatively little difference in price or quality in colas and the price is low enough that almost everyone can afford them. On the other hand, there is a dramatic difference in price and quality in footwear.

Sam's Cola is $8.54 for a 24 pack while Coca-Cola is $13.48. (Both prices taken from Walmart).

Cheap work boots are $30 at Walmart while Red Wing Super Soles are $250 while Iron Rangers are $350.

Skip the Iron Rangers and go straight for the Thorogood Moc Toe

"poor taxes" are an extremely real thing. google up the phrase to find tons of examples.

> I mean maybe if real life was like "wait one minute and double your money" people would do it

In that kind of scenario, everyone's savings are constantly wiped out; there's no difference between "wait one minute and double your money" and "wait one minute and all your money disappears".

> Poorer people have more worries and less to look forward to.

The authors in a book “The Poor Economics” make a similar assertion. It did make a lot of sense to me.

Are cause and effect reversed?

Does cause start from the Big Bang or from where you/I want it to?

It starts when you collapse the wavefunction /s.

> poorer adults have worse impulse control

I would dispute that line of thinking. Wealthier people who are used to getting what they want when they want it would have worse impulse control. Poorer people are used to having to wait already.

This is not the point. Wealthy people have no rush, because they are always guaranteed to get what they want when they want it as you said. Poorer people can't risk to miss the opportunity, even when they don't necessarily want it right now.

I don't think poor kids has less candy than rich kids, normally it is the opposite poor kids are fat since they eat too much candy. If it was money, sure, but this is candy.

And in reality there is no guarantee that the offered double reward actually materialize later. Better take what is offered now.

I think it's more complicated than both of these, honestly.

Past a certain level of desperation, there can be a hard-to-escape level of nihilism - "why bother saving, something's going to take it and I'm going to be fucked tomorrow no matter how much I do". Whether this is an accurate description of the situation or not is going to depend, but I have met a number of people who think like this even when they've not been that desperate in decades, and it bites them as soon as they stop making so much that it masks the problem.

And in some cases, it can be practically true - there's various systems that are designed with nasty edges where if you have enough resources accrued, you stop being given support, but the thresholds, by design or incident, are far below the point where you might be able to escape the pit, so you can't save your way out of it - you'll suffer a catastrophic penalty for accumulating wealth, and then be worse off.

Once you get past a certain level of instability, you start seeing gains again from saving if you do it, but not necessarily immediately - after all, if you're earning $9 an hour, at perhaps 160hr/month, and you're spending $800/mo on rent, that's $640 ignoring taxes to spend on anything else, so even if you somehow spent _none_ of that on taxes or food or w/e, it'd take more than a month to save one month's rent. So the benefits of saving are slow to accrue, when your income is not much past your expenses, and it can be hard to convince someone who's never had that level of safety and stability that it's worth it when it's going to take a long time to be worth it.

If your income is outsized enough to your expenses, then it can be more obvious much faster, _but only if you've ever had to think about it_ - if you've been externalizing your life expenses to your parents or a trust fund, it's even more foreign to you than the people described above who have concluded saving isn't useful, because you've never had to think about money as a resource in your life, it's just a thing you spend unendingly.

I'm reminded of one of the stories from "Poor Economics" (Abhijit V. Banerjee and Esther Duflo, ISBN: 978-1-58648-798-0).

Consistently, those in poverty (living on < $1 per day) do not simply 'try harder' to save and make better financial decisions by being restricting their impulses. Instead they find clever ways to outwit themselves.

One mother in India, intent on accruing $2000 (?) for her daughter's dowry, knew she didn't have the willpower to "just save $200 per year over 10 years", though her annual income was greater than $200.

But she had the foresight to take out a $2000 loan from a bank, immediately move it to a saving account, and pay back the bank. The interest on her loan was almost like an extrinsic-motivation fee.

I think this might be missing information? I don’t understand

The mother forced herself to save by punishing herself if she did not.

The $2000 in the saving account was earmarked for the dowry, and she would not touch it. So, she had to save every month to pay back the loan to the bank. The longer she took to do this, the more interest she paid.

This is a kind of non-rational trick that some people use to "trick themselves into doing the right thing". This is like paying some random stranger $100 if you do not accomplish some task.

Well, some people in rich countries take out a mortgage and buy a house on similar grounds.

With the differences of:

1. Immediate use of the home 2. Collateral that can be taken back of the loan isn't repaid 3. Often the collateral gains in value over the duration of the loan 4. The interest rate on the loan is way lower than a personal loan or a credit card.

I mostly agree.

However, (3) is a bug that's just commonly encountered (hopefully temporarily) in some countries that restrict construction way too much at the moment. One can hope.

The mother knew (from experience?) that she did not have the necessary impulse-control, or intrinsic motivation, to save money for such a long-term thing as a dowry.

I was really taken by this example because my initial reaction was "just try harder". My first-world perspective has always been "if you failed, its a personal short-coming. You need to become better".

This mother's solution is based off a fundamentally different philosophy; "I am not capable of that kind of self-control. I'll modify the environment".

This mirrors things I've read in Gregory's Savage Money.

Most people on this website live in cultures where the value system places prestige upon accruing money, so it's easy for us to do so because the incentives line up. In parts of India that's not so. To simplify a little, their value systems places prestige upon spending money on social events/life cycle rituals, so it's easy for them to spend money on e.g. a funeral but significantly harder to hold onto the money.

This, in Gregory's view (which I symphathise with) has nothing to do with being rich or poor. The rich in Bastar also spend a lot on life cycle rituals, but they outearn their spending.

For the interested, I plan on publishing a review on that book later this month: https://two-wrongs.com/book-review-savage-money.html

It's not just impulse control. That poor adult probably needs the "marshmallow" now while the wealthy adult has enough resources to "invest the marshmallow". There is also perceived risk. If the marshmallow is there for the taking, you are guaranteed to receive that marshmallow if you take it now. The promise of receiving more marshmallows now is just that, a promise not a guarantee. (And, of course, many real life future gains are not even promises. Many people invest a lot into their homes with the expectation that it will increase in value. For most it will. For some it won't.)

It may sound silly when everything is framed in terms of marshmallows, but it's probably a safe bet that lessons learned in life will carry over to an experiment unless they carefully consider what the experiment is asking of them.

I think if you're trying to make a choice yourself then making it sound silly is a good way to take the emotion out of the decision.

Another way I've seen people reason over things like this is in terms of an MMORPG/ARPG. 'You need to invest in your character to get stronger to beat the next boss. That will pay back in X amount of time, but if the value of the loot drops too much in that time then it's not worth it.'

> The obvious conclusion is that household income is a predictor of both

Could you please add if you think the prediction goes in the positive or negative direction?

One of the big psychology books mentions this study. Maybe Thinking Fast and Slow?

All it tells you really is whether the person has to grab what joy they can now because their life experience has taught them that promises about tomorrow sometimes do not come. You see that marshmallow, you enjoy it while you can.

And that’s also ignoring the joy of small things. Three marshmallows is as enjoyable as fifty. So now you need to decide if one is enough joy or if you should wait for ten or whatever the reward is.

> The obvious conclusion is that household income is a predictor of both:

> - inability to delay gratification, and

> - higher academic achievement

> This makes sense when you consider that someone growing up in a poor household may have both:

> - less reliable/continuous/predictable access to material things, meaning they would rationally seize immediate opportunities rather than taking the risk of a larger future opportunity, and

> - less academic support

While this is all true, there's another factor that no one ever brings in: wealthy people are likely to possess attributes that lead to wealth accumulation like conscientiousness, intelligence, ability to delay gratification, etc. Those traits are quite heritable, so their children are likely to have higher income.

People are allergic to the idea that outcomes have something to do with heritable characteristics. And allergic to the idea that economic success is related to positive personality traits.

It's both. Higher household income means better nutrition for their children (which also maxes out your genetic disposition), better education, more secure attachment, focus on careers, etc, in addition to the points you raised.

Good points. But you know what's even more heritable than those traits? Wealth.

Some people attribute success to raw ability. You get good grades because you know the material, you get the job because you're the best at doing the work.

There's also a component of giving others the perception that you're capable. Seeming smart, navigating the politics of a school or workplace, fitting in. A lot of this seems like it's obviously learned, and that affluent people will be ahead.

> wealthy people are likely to posses attributes that lead to wealth accumlation like conscientiousness, intelligence, ability to delay gratification, etc. Those traits are quite heritable, so their children are likely to have higher income.

It's just as likely that proximity to these characteristics is sufficient. As heritable as those might be, inheritance of assets is protected by law. What more heritability is needed?

> People are allergic to the idea that outcomes have something to do with heritable characteristics.

Heritable only through learned behaviors, imitating family. Too many playboys squandering grandpa's hard-earned wealth to think otherwise. The right lessons just weren't taught for some. Positive personality traits do relate to economic success, but too few who have those have the parenting skills to transfer those to the next generation. Their genes certainly aren't doing the heavy-lifting.

Assets are typically inherited long after the income generating years have been determined. You could argue that the promise of high likelihood of a future inheritance event affects choices (and I’d agree that a positive effect), but I think most people receiving inheritances of notable size are middle-aged.

> This makes sense when you consider that someone growing up in a poor household may have both:

the inverse makes equal 'common sense':

wealth begs complacency and indifference

yes, there may be a correlation but i bet it's insignificant as the factors playing into this are just too numerous.

> But when they controlled for household income, they found most of the correlation disappeared.

This does not really prove much, since attitudes to long-term gratification are probably shared within households due to the effect of idiosyncratic cultural factors, which might affect both income and academic achievement. One would need to look for a "natural experiment" where divergence in income was totally exogenous and not due to any shared factor in order to conclusively resolve the issue.

You could also look at adoption and twin studies.

And trust. Does the kid trusts the researcher they will actually get more marshmallows? Or is their life experience such that adults promise candy and then dont deliver?

>The obvious conclusion is that household income is a predictor of both...

Correlation, but of course, not causation. We need to be very careful about storytelling, especially when it comes to behavioral studies, where it's easy and intellectually satisfying.

Yup, I fell into the same trap I was pointing out :(

I should have said something like "it's just as plausible that ...".

My main point is that these 3 studies don't provide any evidence that teaching someone willpower will help with other life outcomes.

> But when they controlled for household income, they found most of the correlation disappeared.

Controlling for things is mostly bad statistics, although of course all social science is bad statistics.

Confounding variables are bad controls more often than they're good ones, so controlling for them introduces collider bias. Also, finding a result and then controlling for something is a multiple comparison fallacy.

The correct thing to do is to have a theory of causation and then design a study that's capable of detecting it, not the other way round.

In this case household income is the parents' household income, so it can't be affected by the child's (future) academic achievement.

Can controlling for household income introduce collider bias?

(Sorry I know the words you're using, and a few years ago I started reading Pearl's book, but I did not finish it and do not have a strong grasp of the concepts.)

> household income is a predictor of both: > - inability to delay gratification, and

I'm not sure we can draw that conclusion. Household income is a predictor of higher education -- that is well established -- and higher education as a child could mean you are more likely to have learned lessons on the benefits of waiting vs instant gratification (the principle behind savings and investments).

So higher education _might_ be correlated with delayed gratification, but not household income itself.

Alternatively, ability to delay gratification could be heritable and/or influenced by parenting styles, in which case controlling for household income doesn't make sense. Parents with high time preference tend to not do as well financially AND to have children with high time preference. I have no idea how much these factors are at play, everything is at least a little bit nature and at least a little bit nuture. But science must find out rather than acting squeamish. We know that patience can be improved through training and maybe eventually we will even have meds like we have for ADHD. If these things matter, we could improve millions of lives.

  We know that patience can be improved through training

Right, but AFAICT none of these suggest show a causal link between improving patience and life outcomes.

> household income is a predictor of both

Household income is a predictor[0] of a lot of things.

It strongly correlates zip code (almost identical) and from these you can make good predictions of race and even politics. A naive mistake a lot of people often make is thinking that removing explicit race data removes race from your model. It's still there, but now only expresses itself indirectly. This is not just true for things like race or politics, but for a lot of factors, which is why causal statistics is such a different (and harder) ball game.

[0] Predictor means correlates with, not causal. I see some people confusing this in the comments. I've never liked the word because of this result.

I don’t understand the racial part, since there are plenty of trailer parks and poor rural areas. Zipcodes in the Appalachias for example.

it's scale-invariant and self-similar. pick a big city or a sundown town, the demographics change but you're measuring a consequence of modern/historic systems larger and longer-lived than either place

It just is a thing, tbh. It manifests in the data pretty clearly.

In aggregate, in large data sets, race comes through - especially with a few datapoints. For example, when I worked at a fintech company: with household income and zip code, we could accurately target race with >80% accuracy [0]. Add a few more datapoints, and this would very quickly get closer to 95% accuracy.

That was an _actual_ party-trick[1] demo we did, alongside also de-anonymizing coworkers based on car model, zip code, and bank name.

[0] I worked as a SecEng and were trying to prove that we were(n't) inadvertently targeting race, for compliance reasons. In the end, the business realized the threat and made required changes to prevent this.

[1] We were doing this to make a case for stricter controls and stronger isolation/security measures for storing non-PII data. The business also saw the light on this. Sometimes we'd narrow them down to 30 or 40 people in their zip code, and sometimes (such as a coworker with an old Bentley), it was an instant hit.

Not accounting for race comes up in scary ways. I was part of a program that used a totally neutral database (race/gender were not in the database). People were selected by criteria and then emailed asking them to attend an introduction meeting online. Only when the webcams were turned on did we realize nearly every volunteer was none-white female. It was a very bad look. It seemed that we had selected them based on race/gender when in reality that data wasnt availible until the first video call. By ignoring race/gender we had somehow made it the most obvious selector.

(The program involved having children who were in regular contact with the criminal justice system.)

Did it turn out that the selection process actually didn't represent the makeup of the target audience?

If the participants did represent a subset of the target audience, I don't really see what the problem is if that audience happens to be heavily weighted towards a particular race, sex, etc. It seems like you'd be doing a disservice to the program to purposely control for those factors and end up with a population that physically looks more diverse at the cost of missing people who actually most need the program.

A lot of people have gotten into a weird place where they think that acknowledging that the descendants of slaves in the US are in a dire situation is a form of racism. Acknowledging that being injured has caused an injury has become either extreme right-wing bigotry (if you're a liberal who demands that every subset of people be a racially representative mix), or "the soft bigotry of low expectations" (if you're a conservative who can't admit to yourself that you inherited hundreds of thousands of dollars from a parent who also paid for your private school, car, and rent, then found you your first job.)

Sure, there are a lot of interesting and compelling angles to consider why these kinds of things happen. In such a specific case as the parent comment here, it really is more interesting in my opinion to see what was found.

For example, it stood out to the team that they group of people that were on the call paint a certain picture when considering how similar they were (specifically by surface-level factors like race or sex). Is that response justified when they dug in and found that the group didn't represent a similar makeup of the total population despite having completely ignore sex and race? Or did it turn out that the population is actually that homogenous? If it is that homogenous there are great questions to dig deeper on, like why that happened and what may be done to help correct that.

It's not racist show, point out, or claim that data racially skews in one direction. If that were true, then you couldn't even claim that minorities are under privileged. Right? Then how could you help them if you aren't able to recognize the areas that the biggest challenges? You're right when interpreting this way.

But the thing you do care when you want to attribute causality. In part this is an issue because people naturally associate correlation with causation (there is good reason but that's a long discussion. See Judea Pearl's The Book of Why). At the end of the day, we really are always after causal relationships, because we want to do things with the data (somewhere along the chain). So it's not that you want to remove race from data, but rather that you want to be wary and ensure that your variable is not confounding the real issue. Though this happens outside of race too.

And note that at times there where race does play a causal role. (I suspect not likely in the parent's case) For example, different races may be more prone to certain illnesses or genetic disorders.

If it helps, maybe it is easier to frame it as it's easy to be lazy, but the pressure around race makes us more likely to revisit our analysis and look for confounding variables. The thing is, this will improve your stats even for the non-minority settings because the truth of what you're (hopefully) doing, is just making better models.

I think we're getting at roughly the same idea here, I don't actually think its about racism at all though. The parent commenter clearly didn't select a population based in their race or sex. This may be an unpopular opinion, but I have always viewed racism as very much linked to the person's intent, something that may unintentionally have a bias isn't racist in my book and really lumping it in would cheapen the argument when the intent really was there.

That said, in this case I really am curious if it turned out that there was an unintentional bias somewhere. Say they ended up with 90% black women and in reality that is only say 40% of the total population of kids with regular interactions with the judicial system. There's certainly something there they missed and it'd be interesting data to understand how that happened even when they purposely ignored race and sex. It usually boils down to an otherwise benign detail of the selection process that makes way more of an impact than would have been expected.

> I have always viewed racism as very much linked to the person's intent

Certainly this is one of the most critical factors. I'd argue that this is critical in being not racist in the first place, since it is easy to misinterpret actions. Not to say that how it is received doesn't matter, but that we're in a globally interacting community and we're bound to step on one another's feet, so it is important to recognize that not knowing the dance is not the same as intentionally attacking nor that not knowing the dance is inexcusable in the first place. As this would need to go both ways and then we'd all be at fault. We should try to learn at least.

But I do want to push back a bit, and I think this connects to the prior point of the statistics. I do think it is possible to be unintentionally racist. Just like you can intentionally harm someone and you can unintentionally do so. In either case harm is still done, right? But it is with good reason we distinguish these in our legal system.

The world is much more complicated than it used to be, and this is the burden of advancement. As we advance, lower order approximations are no longer sufficient to solve problems, so we must become more nuanced, more forward thinking, and we must slow down so that we can move fast. The burden of our advancements is that we are now the gods who destroy cities without even knowing what we have done. The same way a butterfly does not know the hurricane it creates, because it's the interaction of its actions combined with so many others. But this does not change the end result. A more connected world means that our actions have more paths to travel through, and thus can do more. The question more is if we'll deny this or if we will try to do better. Maybe it is impossible to live without stepping on cities, but even if that is true, it doesn't mean we shouldn't try to look where we step and minimize the damage.

So what I'm saying is to me it didn't sound like they were trying to deny the correlation of whatever they were studying had to do with race, but rather that we've advanced as a society enough that correlation is insufficient. And at least in this domain we've recognized how it is easy to fool ourselves with data. Because I'll tell you, most people fool themselves with data, including experts. The difference is the expert always reserves some doubt. Most people confuse what data analysis does. It doesn't answer questions, it can't. Instead it eliminates potential answers. If you remember this, doubt is a natural consequence. If you don't, you'll always be the fool, lying to yourself.

I think it’s more reflective of the reality of living in the US than of your company’s selection process. I’m curious what you did after realizing this. Did you pivot away, or create a program designed to be useful for the volunteers? Assuming that the volunteer pool accurately represented the larger group.

> Zipcodes in the Appalachias for example.

You're overconstraining what I've said. You're perfectly right that zipcodes in Appalachias account for many poor people that are also white. But actually, you're correctly inferring that you can still infer race out of this, because you're inferring that the majority of these zipcodes are also white. Right? White people are also a race. You're correct that zip code is also able to strongly indicate poor white people. In fact, it is also even able to strongly indicate rich black people. Though you might guess not to the same degree as the overall rate is lower, but people do congregate.

Think about it in a different framing: zipcode strongly correlates with people congregating together who are culturally and economically similar.

I think this version should make sense (especially as the locality affects the culture), and that from here you can extrapolate to recognize that people of varying demographics aren't homogeneously distributed among zipcodes of similar economic bins. I part of this is easily explained by a simple fact: when people move, they like to move to where they have friends, family, or other connections.

You are obviously not gonna get rich by moving to a rich zipcode, but rich people move to rich zipcodes.

In that sense it is very much a causal relationship.

If you move to a rich zip code your child goes to a well funded school, hangs out with rich kids behaves like a rich kid. Moving to a rich zip code (and supporting your child’s education) is a fantastic way to get your family on the path to generational wealth.

It really is. In fact there's some good studies that suggest that this is even true with race, where you don't need to move zip codes but simply move schools (a la "Everybody Hates Chris"), if the kids are young enough.

But clearly the causal connections here are not zipcode, but schooling, culture (or rather the knowledge to understand and navigate a culture), social connections, etc.

No, that does not make it causal.

1) Causal relationships are directional. Which then follow your own logic. You can reason that it is directional because you can see that there are many ways to get into rich zipcodes without being rich.

2) You can find many rich people that don't move into wealthy zip codes.

So actually, it is just correlating. If we use Judea's ladder, then it is causal in that sense, but that's the lowest rung and not what we'd call causal in a colloquial (or even statistical) sense other than to be pedantic.

The direction of causality is just hard to determine. The “obvious” conclusion could be that ability to delay gratification is a predictor of: - the offspring’s ability to delay gratification (genetic; what is tested in the experiment) - household income - higher academic achievement (for the offspring)

Or that the ability to delay gratification led to higher incomes in their parents and was a hereditary trait passed down to the kids?

There is one big variable here that is being overlooked - which is the test itself.

I remember my mother doing the marshmallow test on me, aged 3. She put two packs of play-doh moulds on the stairs where I could see them - one with three in it, one with five in it.

She told me that if I didn’t touch them, then she would give me the pack with five for my birthday a week later, and the pack with three to a friend. If I did touch them, she would give them both to said friend.

After passing them on the stairs for the umpteenth time, after a few days, I caved, and opened the small pack.

She made good on her threat.

This was 40 years ago, and remains seared into my mind as a learning moment, and I have, since that moment, been absolutely ruled by delayed gratification - I never made that error again.

So, in short, merely administering the test likely influences the outcome, as humans have memories, and children learn from experience.

Many details of this particular experiment made me greatly reduce my confidence and interest in social science. I was trained up in quantitative biology- and when I look at studies like this, I see a long list of "things that could go wrong, leading the investigator to falsely conclude their hypothesis is true". But in this case, I think the investigator actually didn't care enough about doing high quality research- they simply started with a moral belief/value judgement and ran an experiment and chose to interpret the results to support their "hypothesis". And the nature of social science is such that it's really hard to truly run an "honest experiment".

You can read over a hundred years of extensive, exhaustive criticism of most social "sciences" for exactly that reason, and many of us grew up with a categorical understanding of a difference between "hard" material sciences like physics and chemistry and "soft" sciences like social sciences and many subdomains of biology and medicine.

But that distinction has largely fallen out of the zeitgeist and many people now just take anything ever published in a "scientific journal" as sound.

It represents a huge regression in scientific literacy among the public and sets us up for people becoming increasingly skeptical of "hard science" conclusions because so much of what they've incorrectly come to accept as science never really was.

True but there has been a movement towards replicating these high profile findings in the soft sciences. Hopefully that will gain more traction as a lot of the "newsworthy" studies are forced to get retracted after failing to replicate.

> people becoming increasingly skeptical of "hard science" conclusions

A big problem is that "hard science" conclusions often only apply to very specific circumstances, but scientists and the general public then extrapolate to more generic situations. The consequence is that a lot of things that are supposedly based on "hard science" aren't really proven at all, it's just someone making educated guesses.

PSA: There's also a severe replication crisis in the hard sciences. The high horse is not well warranted.

Completely false equivalence. The entire foundations of modern Psychology are wobbling. In order for the same to be true for the hard sciences we would need to be failing to replicate experiments which hinge on germ theory, atomic theory, the standard model, etc.

Nothing like that is happening. This false equivalence originates from several types of people:

1. Journalists that want/need to foment the largest possible catastrophe.

2. Political pundits which want/need to discredit some field.

3. Social scientists playing defense.

What exactly do you think are the foundations of modern psychology? Serious question.

There are tons of non-replicable findings way, way further down the stack than psychology, and those tend to have a lot more relying on them than psychology/sociology studies. If you're upset about scientific validity, consider directing your ire to where problems are more likely to actually hurt people -- the "hard sciences."

Nice ad hominem but I'm none of those things. I work in clinical trials, one of the few areas where we actually do have to know things, and a very good empirical demonstration of exactly how incredibly difficult that is.

I'm curious to hear your perspective on the validity of psychology / psychiatry / sociology as someone adjacent to the field.

I am a hard science maths / data science guy, but unlike a lot of my peers I have a great interest in softer reasoning (philosophy, ethics, political science etc). But I am constantly disappointed by how tainted by ideology psychology and psychiatry feel (and economics, but this is a different discussion).

Do you think that psychology and psychiatry are held to the same rigour as harder sciences and should be considered as valid?

All fields of inquiry are tainted by ideology. Read the history of literally any scientific field ever. The entire system is designed to accept this as fact, because science is done by humans, and to still arrive at the truth nonetheless.

If we take those two observations: a) science is done by humans and b) humans have motivations, obviously the way to arrive at truth is to allow for most things to be wrong most of the time. This is the process by which we've learned every single thing about the universe.

I don't know what you mean by "held to the same rigor." I don't think any psychologist on the planet would tell you we understand psychology as well as we understand basic chemical reactions.

> The entire foundations of modern Psychology are wobbling.

What do you mean by this? the field of psychology is perfectly capable of policing itself, and it's rejected much of the conclusions of its historical predecessors.

> In order for the same to be true for the hard sciences we would need to be failing to replicate experiments which hinge on germ theory, atomic theory, the standard model, etc.

"hard sciences" also fail to produce results relevant to most people. Sure, they can maybe make better batteries, but how can they explain how dysfunctional society is?

EDIT: We can also directly blame the poor communicational skills of "hard sciences" for diet culture. "hard sciences" have failed in their efforts to produce a population that can reason about nutrition in an evidence-backed manner, and this can be traced directly to how scientists choose to present their data.

You don’t need science to explain how dysfunctional society is. The problem with “soft sciences” is that they can’t produce provably correct information. Incorrect information is worse than no information.

> You don’t need science to explain how dysfunctional society is.

You absolutely do lol, or you're just straight wrong. Take your pick.

> The problem with “soft sciences” is that they can’t produce provably correct information.

Neither can hard sciences. Proofs are incompatible with empiricism. Abductive proofs (which is what the scientific process offers) are necessarily bounded by limited certainty. All you can do is progressively improve certainty approaching 100%, but reaching it is necessarily impossible. This is just basic Hume. You can never be 100% sure the sun will rise tomorrow or that the fundamental laws of physics won't arbitrarily change.

But, the same applies to soft sciences. We can and do increase our certainty continually. This is absolutely worthwhile and is probably far more valuable to humanity than merely modelling physical phenomena.

> You absolutely do lol, or you’re just straight wrong.

This is false, an obvious counter-example is an explanation that happens to be coincidentally correct (without being derived from any science)

This seems like an attack on the notion of objective truth which while interesting in the abstract, is not particularly interesting to me personally.

> an obvious counter-example is an explanation that happens to be coincidentally correct

How can you establish "correctness" without something like the scientific method? How do you even bind loosely-defined english to verifiable claims to real-life referents without agreeing with others on terms? No, you're chosen simple, comforting delusion over anything resembling objective truth.

> This seems like an attack on the notion of objective truth

Yes, truth is an apriori concept; objective truth is a silly delusion. Coherence is generally a much stronger concept anyway.

Correctness (of a given assertion) doesn’t need to be established to exist

> "hard sciences" also fail to produce results relevant to most people.

I don't see how you can say this, would you prefer to live 200 years ago before hard sciences had started changing peoples lives? Almost every convenience you see around you exists thanks to hard sciences.

That's an awful lot to put on hard science rather than, say, industrialization—the result of soft sciences just as much or even more so than hard sciences. Plus if you insist on only evaluating material concerns we still need to grapple with soft sciences to figure out why society is so horrible at distributing material goods and services in a rational manner.

My point was not to reject hard sciences so much as to emphasize you can't easily extract the consequences of them in isolation. It's nearly futile to even try. My apologies for poorly articulating this.

To be fair all of that was up in question a century ago in the hard sciences. People used to believe in the plum pudding model, or doubt what component in the cell contained genetic information. The only thing that changed that was incremental experimental evidence. Social sciences are evidently going through a similar transition but we shouldn’t use that to discredit the field alone, work is still able to stand for itself if it is done well.

The level of ignorance of this basic, basic historical fact is just completely astounding on this website sometimes.

It was called natural philosophy for Christ's sake.

Modern medicine's forefather was heroic medicine, based on modulating the 4 humors (blood, phlegm, yellow bile, black bile).

We didn't zap into existence with the hard sciences being hard. We made them that way through centuries of intellectual effort, almost all of which turned out to be wrong!

True, but I remember reading the replication failure rates were twice as much in the social science than in medicine.

We constantly see small medical studies (<100 participants) posted here on HN that produce exciting results, which then disappear from view and/or fall apart when replicated with larger cohorts.

How is this different from science working?

When it happens in the hard sciences its science working, when it happens in the soft sciences its a replication crisis.

Haha that seems to be the case!

If that didn't happen, it would mean that math was broken. That's how statistics is supposed to work.

Depends what you count as hard science. The replication rate in high energy particle physics is near 100%? When the LHC started up they were able to measure nearly all of the particle resonances found in the 20th century. It's not like they suddenly disproved the existence of electrons or something.

Sure, further down the stack of chaos that is the universe (physics -> chemistry -> biology -> psychology -> sociology), it's much easier to conduct controlled experiments.

That doesn't mean the people engaged in research at the bottom of the stack are good and the people at the top of the stack are bad. Nor does it mean we shouldn't be trying our best to understand things near the top of the stack.

> it's much easier to conduct controlled experiments.

Very true. But this means more statistics and controls are necessary to get solid result from a social science experiment then a particle physics experiment, no? Clearly, this is practically impossible, but there you go.

> Clearly, this is practically impossible

No it's not? You put more money into the studies and you can do bigger, better versions of them.

A major obstacle to putting more money into studies: people jerking themselves off about how soft sciences are a joke and hard sciences are Super Serious Business.

But why do those fields deserve more money, when at least a large part of the problem is cultural.

One example is the famous reluctance to publish negative results in psychology. Nearly all published results in (collider) particle physics are negative.

If senior faculty prefer to only hire people with a string of published postive findings, you are literally encouraging p-hacking. Again, they are not "bad" people, it is just that the system the senior people have setup in that field is not conducive to doing good science.

> But why do those fields deserve more money

Because it'd be good to understand what makes people happy, for example. Or what enables relationships to thrive. Or when different forms of government are suitable or unsuitable to solve a set of problems, etc.

Sorry to break it to the hard-sciencers, but the vast majority of opportunities left in the western world to improve people's lives is not particle accelerators, it's answering questions like: "what actually helps people feel satisfied in life, loved in their relationships, and belonging in their community?"

> At least a large part of the problem is cultural

Is it? Why so?

Negative results aren't published in almost any field, and that's actually a good on ramp to the discussion we should be having, which is about the broken incentives of science and scientific publishing specifically. The broken incentive model isn't special to softer sciences and it has far more dire consequences in other domains.

You can't possibly think that soft sciences are the only ones hiring people with a string of positive results... right?

I agree that studying psychology better could be beneficial. Is it possible? Or more to the point, is it merely a matter of money, as you said?

I said a large part of the problem is cultural, I did not say that psychology is the only field with cultural problems. I'm not sure how you got that idea.

No no, what you said is that it's "clearly practically impossible" to have more statistics, more controls, etc. to get higher powered studies of high-chaos questions like the ones asked in the soft sciences.

I said, to that point: no it's not. You just do bigger, longer term, more complete studies. The limiting factor on this -- right now -- is typically money. Perhaps you can pour infinite money and problems with e.g. recruitment or monitoring will still prevent us from getting to statistical power, but maybe not.

That is not the only problem social sciences face, but most of the problems they face are not exclusive to social sciences whatsoever, which then prompts the obvious question of why they get so much flak.

I never said anything about practicality, you are confusing my replies with someone else's. I said this

"But why do those fields deserve more money, when at least a large part of the problem is cultural."

More money won't fix the cultural problems with the field. Maybe the lack of a universal quantitative framework makes issues that are slightly bad in some fields much worse in the social sciences. I don't know. But it's naive to just say other fields have the same cultural problems so we can ignore it for MY field.

There's a famous anecdote where Dyson came to Fermi with a theory for weak decay (I think). There were a handful of datapoints that matched experimental results.

However, Fermi threw out Dyson's theory, despite the empirical agreement, because it had 5 free parameters ("with 3 parameters I can fit an elephant, and with 4, I can make it waggle its trunk is the quote" IIRC). This is difficult, but essential criticism all fields need.

AND all this is apart from the fundamental question of whether important generic scientific truths can really be gleaned from the social sciences.

> Sorry to break it to the hard-sciencers

Believe me, you aren't "breaking" anything to anyone. If you could solve the secret of happiness (your example), no amount of money would be too small.

The issue isn't whether social science would be good to figure out. Definitely it would, to the extent there is actually a "thing" to figure out, which may be true and may not; i.e., "what makes people happy" may be so contingent and/or so ineluctably open to interpretation that it makes no sense as a rigorizable concept. (There is nothing wrong with unrigorous concepts, btw, these have been fruitfully explored by the poets and philosophers and therapists.)

Ok, so even granting that there is a stable, rigorizable "truth" for the social sciences to discover, the issue is whether the methods and analyses as they have been practiced are effective or even could be tweaked to be effective. Clearly, they aren't. And not just a few bad apple studies, but seemingly the whole darn lot.

> Clearly, they aren’t

Arguing is easy when you just assert your conclusion eh?

The question of "do those fields deserve resources" is answered as follows: are there interesting questions in those fields that we should ask and have answered (well)? I think the parent poster is saying: yes, there are.

This question is orthogonal to the question of whether the organizations currently conducting research in those areas are well-organized. You could fund them well and also demand re-organization as a condition. You could even find other scientists to do this work. But if you don't think the work is important, none of this matters.

That is fair. Is what you suggest possible though?

As my sister who is studying one of the soft sciences put it to me when i pointed out the lack of rigor compared to the hard. "sure we could make psychology a hard science but pesky ethics boards wont approve me raising batches of several hundred human clones in controlled environment for each test"

I mean, a psych experiment will never have an N comparable to a particle physics experiment or be able to reach the 5-sigma threshold for discovery that now prevails in physics. On the other hand, the object of study for psych is intrinsically interesting since we are people and if something reliable can be gleaned then it's certainly worth money. My concern is that "bigger, better" (as you say) would have to include millions of people across cultures and times, tracking longitudinally, with randomization and controls. (Again, more complexity requires more statistics, not less.). Is this practical? Maybe ...

I didn't claim that it means they are "bad" whatever that means.

I am saying that we should take those claims less seriously, especially if the results from that domain don't replicate, as in the case of psychology and other social sciences.

Maybe there is little we can conclusively say about those domains.

Yes we're in agreement on this.

There was little we could conclusively say about any domain until long term, concerted effort was made to understand each of them.

Anecdotally, I have a friend that's doing bio-chem doctorate and she has said that replication rate on that is abysmal for biology, chemistry and consequently bio-chem.

I'm sure some areas of physics have near 100% and some simply don't.

Did the public ever read published papers until recently? I can’t imagine most people having access to any publications until the Internet and late 90s at minimum.

My local library did not exactly have access to journals either.

I don’t think scientific literacy has ever been high. Society relied on other publications and the government to interpret the information for us. For better or worse.

University libraries still get hardcopies of monthly journals. Maybe back in the day a downtown central branch might have as well.

> You can read over a hundred years of extensive, exhaustive criticism of most social "sciences" for exactly that reason

There's nothing essentially non-scientific about the fields; it's just harder to control variables. The entire "hard science" vs "soft science" beef is a little silly when "hard science" isn't equipped to reason about most human concepts. Try not to chuck the baby out with the bathwater. I'd prefer to stop differentiating between the two ends of the spectrum as if they're inherently different.

I also find that people who poo-poo "soft sciences" still have strong beliefs about humans, society, etc, they just don't even bother trying to ground them in evidence.

The non-scientific aspect is that conclusions are drawn even when the variables haven't been controlled (because as you say most of the time it's basically impossible).

I imagine it must be really frustrating for people actually trying to learn facts when the famous results are dominated by nonsense like this, power poses, hungry judges, etc.

It's not just in research either. E.g. in the UK they did a regression of missed school vs exam results, noticed that they were correlated and now it's a criminal offence to take your children out of school for holidays, even for a single day.

> The non-scientific aspect is that conclusions are drawn even when the variables haven't been controlled

Yeah, it is so strange when people say we should listen to the scientists, we shouldn't since scientists make all these much stronger claims than the data supports.

Instead we should listen to the data, see what it says, rather than the scientists who did the studies. And it is really important to note that the data doesn't necessarily say what the study author says it does.

Edit: Above is for social sciences, since they lack robust results. In the hard sciences were we have a lot more robust results it makes more sense to listen to those who know those results, that doesn't have to be scientists though could be experts like engineers etc.

There's no rubrik for grading scientific correctness. Just because controlling variables is more difficult doesn't mean we should abandon the scientific method and just talk out of our asses.

The reality is social sciences are inherently more complex because they are higher-level problem spaces. Biology in merely a stepping stone to psychology, like assembly to modern Java. Assembly is simple, understanding how the JVM works is hard.

Psychology intertwines just about every field you can imagine. Anthropology, neurology, biology, chemistry, you name it. They're fundamentally working on a level much, much higher than the hard sciences. So the sheer amount of variables is absurd, but also how they work together.

Hell, even the "Dunning Kruger effect" is a misapplication of statistics.

The effect shows up even with randomly generated samples. Because there are floors and ceilings to the data. If you're low, you can only guess so much further down, so you're likely to overestimate your ability. If you're high, you can only guess so much further up, so you're likely to underestimate your ability.

Yes, the devil always seems to be in the details in psychology experiments. Were the experimenters giving subtle cues to the child, and was this simply a test at how deftly the child picked up these cues? What was the exact wording of the “deal” offered to the child and would a wording change alter the results? Was the experiment conducted at a time of food scarcity or abundance? What were the prevailing cultural norms of how a child “ought” to behave? Would the results change if average child age was 6 months older or younger when experiment was conducted? What was in the drinking water and the air and the paint at the testing site? (With strong claims in the literature that all these are correlated with measures such as average population IQ!)

In the face of all these potential confounders, more statistics and controls seem necessary than, say, in a physics collider experiment on electrons (each electron possessing exactly two characteristics, location and spin, and all such electrons behaving identically regardless of location or time). Yet, even in this setting of simplicity and reproducibility, physicists have still found it necessary to establish a stringent, five-sigma threshold for discovery — 3 sigma anomalies come and go. Such a stringent threshold is unthinkable in psychology due to practical considerations. Ergo, it’s hard for to see how psychology can become a reliable empirical science.

I'm not really disagreeing, but the 5-sigma rule is there because the hypothesis is not formulated before you run the experiment.

If you make the hypothesis first, 3-sigma is quite enough. Many physics experiments do exactly that, but famous high-energy ones don't.

(That said, not having an hypothesis beforehand was very common in psychology before the 21st century.)

This is one of those rules of thumb that don’t make any sense to me as someone who works with data in the field. You conduct the same exact experiment with the same conditions and same data and get the same result. But whether you speculate on a hypothesis before or after suddenly changes the significance threshold with no actual change in the underlying data or method? Did you cast some ancient spell when you came up with the hypothesis or something?

> with no actual change in the underlying data or method

Proving a known hypothesis or deciding what you want to know after the fact are completely different methods.

> Did you cast some ancient spell when you came up with the hypothesis or something?

"I'll know it when I see it" is an incredibly vague way of doing science that requires extra rigor somewhere else to compensate.

Or, in a maybe better explanation, testing for multiple hypothesis is subject to this kind of failure:

https://xkcd.com/882/

So you need more data confirming your theory.

But if you state your single hypothesis beforehand, you are at the situation of the top square, with a high-confidence result.

I mean let’s get real. This particular experiment is pretty irrelevant to how you feel.

There’s nothing that could be done in social science that you wouldn’t be skeptical of and want to dislike.

I have read from BOTH psychologists and sociologists criticizing this concrete experiment for years. It was popular among internet crowd of fancying themselves as "cool nerd kids" who play experts on everything. But if you read more boring write ups by actual scientists, they complained for years.

> Many details of this particular experiment made me greatly reduce my confidence and interest in social science.

There is a reason why many scientists diplomatically classify social "science" as a soft science. Less diplomatically minded scientists like Feynmann call it pseudoscience.

It's funny. When you first do work, you want the experiment to satisfy your hypothesis. When you are building on work, you also want the replication to succeed. But when it is a famous result like this, you actually want it to fail so people talk about your result. There are uncountable ways that these experiments can be unconsciously and subtly affected by the desire of the experimenter.

As an aside, I believe one interesting confounder in the marshmallow test is that it tests more (or at least as much) the subject's trust that the eventual reward will actually be given as it does the subject's ability to wait for the reward. So if you live in an unpredictable environment, it's better to just eat it.

I recall that your confounder was used as an explanation to wave away the air reproducibility of the marshmallow test, but I do not recall anyone actually ever testing that.

This recent article seems to indicate that it's all just horse feathers and so you can make up any confounder you want to explain it away...

As someone who lived in “unpredictable environment “ in the 90s in Russia the correct statement would be: So if you live in an unpredictable environment, it's better to just move to a predictable environment.

I saw a lot more people saving for the future rather than spending it all, which I surprisingly found the other way around in Canada, which is a predictable environment.

> As someone who lived in “unpredictable environment “ in the 90s in Russia the correct statement would be: So if you live in an unpredictable environment, it's better to just move to a predictable environment.

I think _everyone_ would take that opportunity if it was presented. Or at least most people.

Your environment was predictable, in that savings up enough would likely give you the option to move out. There are places where saving up enough money to leave your unpredictable environment unpredictably ends with you drowned in the Mediterranean, or in limbo short of your intended destination, or shipped back to where you spent so long trying to leave, in addition to the "success" case.

I guess it depends on the meaning of predictable? We couldn’t predict 1 usd = 5rub would turn into 1usd =25rub almost overnight. My parents had friends in business killed (sadly not predictably) so it was unpredictable when it would be their turn.

Kind of interesting, because what happened in Russia in the 90s specifically with privatisation of UDSSR assets through vouchers is almost comparable to the marshmallow test on a superficial level..

> The vouchers, each corresponding to a share in the national wealth, were distributed equally among the population, including minors. They could be exchanged for shares in the enterprises to be privatized. Because most people were not well-informed about the nature of the program or were very poor, they were quick to sell their vouchers for money, unprepared or unwilling to invest. [1]

[1] https://en.m.wikipedia.org/wiki/Privatization_in_Russia

Definitely happens from time to time.

When I took a look at a frequently cited paper 'disproving' Dunning-Kreuger, I was surprised by just how god awful the methodology actually was:

https://www.lesswrong.com/posts/6Tqm8Jet9mzo6buj9/the-dunnin...

>But when it is a famous result like this, you actually want it to fail so people talk about your result.

There are finite resources for replication and so those resources must be allocated. High-profile results tend to attract good and skeptical replication attempts. This has always seemed like a pretty good approach to me. But replication takes time, and some people think it's a catastrophe that "bad" results don't immediately get corrected.

Virtually every study I read about in AP Psych in HS failed to replicate - including this one. That whole class in hindsight was at best a waste of time and at worst provided bad info to make life decisions on

The reputations of these authors need to be dragged through the mud.

Daniel Kahneman's Wiki page doesnt make him look out to be a fraudster, despite him confidently mentioning studies that never replicated, despite him signing off on fake data from other fraudsters.

Thinking Fast and Slow was blowing my mind until I started running into more and more studies that I knew didn't replicate. It took on a Freakonomics/Gladwell vibe after that

Apologies for ignorance, can you tell me more about Gladwell issues?

Gladwell is known for citing junk science and twisting reality by adding his own unfounded interpretations to research he's basing his theories on. There's a host of criticisms of his work. [Wikipedia](https://en.m.wikipedia.org/wiki/Malcolm_Gladwell#Reception) is a good starting point.

Great, thanks for this link, will dig in more.

This one is tough to know where to start. To hear him cover an issue near and dear to your heart, is infuriating. In fact, I think almost all of his podcasts ideas from the first season were terrible.

Mostly, he gets so into a story it's about the narrative itself. He sounds like a crazy child with a made up theory trying to force it to work.

It's hard to say from memory, but it was multiple episodes where he defended corporations and authority figures from legit criticisms, based on nonsense.

Just to be clear he doesn't literally say "and there, so I proved them innocent", but he might as well and these just weird one sided crusades (about David and defending the Goliath as misunderstood).

He's really not far from saying because people sometimes remember things poorly, Brian Williams definitely didn't lie (in a situation where many people lie).

He basically gave the auto industry a free pass, because it's challenging to prove an exact field issue.

That's not how science works. If you doubt the result, do your own experiment and publish it. The reputations will take care of themselves.

Obviously signing off on known-fake data is straight up lying, which must remain in a different category than simply doing a study that doesn't replicate.

A lot of psychology is vaguely science flavoured. You have them making a bunch of surveys and the making super broad claims based on this.

I think you’ll find that if you actually read the papers themselves, they don’t make nearly as broad of claims as they’re purported to in media or the public consciousness.

Psychology and psychiatry, like osteopathy and phrenology, are parts of the pseudoscience branches of philosophy, not science. This makes them more like religious cults reading tea leaves rather than able to perform controlled and replicable experiments, make clinical diagnoses based on evidence, or measure or examine the organ they're supposedly treating.

An alternative interpretation of the Marshmallow Test is that it is a measurement of trust as much as it is of self control. If you don't believe that the researchers are going to give you the two marshmallows, then you're not going to wait.

Or, that it's complete nonsense with no predictability whatsoever so you can make of it what you want

I suspect (or hope) that many professional psychologists are beginning to doubt that data acquired in these contrived laboratory settings can provide a window into actual human behavior at all.

Could be completely random how the kid feels that day and how much they like marshmallows in the first place.

And low trust in researchers would explain low incentive for academic achievements. ;)

Seems like this needs repeated test. Where for a few first rounds there is no reward for later...

or a second marshmallow isn't all that appealing. if offer one now or two later i might pick one now because i don't want two.

Children that trust the adult making the promise tend to be able to delay their own gratification:

https://pubmed.ncbi.nlm.nih.gov/26799458/

From a game theoretic point of view it makes sense:

If your internal model of adults suggests, that you should put a gausian prior on the waiting time until they keep their promise, i.e. most adults in you life tend to keep their word, waiting makes sense.

If however your experience tells you to assume a power law as prior, cutting you losses after a time is perfectly rational.

This has a certain beauty, since it would mean that success in life correlates with dependable parents and given the temporal component I actually would assume causality.

Have any famous psychological tests replicated?

https://www.bps.org.uk/research-digest/ten-famous-psychology...

Tricky subject by all accounts

You can't answer "Have any famous tests replicated?" with "out of 100 studies from the year 2008, 36% replicated" unless one of those studies was actually famous.

> You can't answer

Only if you're a frequentist. A Bayesian would see evidence that studies in general fail to replicate, and thus have a better prior for famous ones than 50:50.

As an aside, such lines of argument regarding credence are not in any way incompatible with frequentism. Few frequentists deny the correctness of Bayes’ theorem.

True, I'm not going to read through the list and decide if any of them are famous for him.

I am pretty sure the test of "will you publish nonsense as if it were true for fame or money" has been replicated multiple times in many different fields.

The Asch conformity experiment seems to replicate: https://journals.plos.org/plosone/article?id=10.1371/journal...

As an aside, psychological experiments tend to become famous by being controversial, which in turn probably constitutes a bias against replicatability. There might be a lot of boring psychological experiments with unsurprising results that replicate without issue.

The Bystander Apathy Experiment has been replicated numerous times as has the Milgram experiment.

anedotally this has held up in my social group. the people that i grew up with and went to school with...the ones that could delay instant gratification and had long term goals ended up doing pretty well in life. the ones that didnt have any plans and just went with the flow did poorly and just getting by.

also in my life i notice a big difference in performance from when i had goals/vision for my life vs. going through the motions.

IMO i think you need to have goals/vision/standards for all the important areas in your life (hobbies,partner,career,family,relationships)

Could go either way in my social group. Some folks hit ivy and then ended up at mediocre tech jobs anyway, others hit ivy and struggling to find work. Others went with the flow and still made it to the same mediocre tech jobs. And the ones who failed through school and barely made it through community college have successful small businesses because they were charting their own path the whole time.

> And the ones who failed through school and barely made it through community college have successful small businesses because they were charting their own path the whole time.

By definition, it sounds like these folk were able to delay gratification quite well.

Maybe it depends on how you look at it? If gratification is "working on my side project instead of finishing homework due tomorrow" then it wasn't delayed much, they were gratified the whole dang time!

But what were they doing when they failed school? I feel like there are the Bill Gates, skipping school kids. And the ones I went to school with who just smoked, drank and hung out at the park.

I suspect the outcomes were fairly different although might both fit under your same category.

>Bill Gates, skipping school kids

Bill Gates could drop out of college and skip school because he had wealthy family that would have supported him if things went poorly. Poor people do not have that option, so when they skip school, they instead get labeled truants and harassed by the state.

IMO you do not. I know many people "doing pretty well in life" who are opportunistic rather than goal-driven, and having goals for your partner/family/relationships sounds to me like a recipe for disaster

In regards to the first part of your post, being opportunistic and goal-driven are not necessarily opposites. A person who is both has a plan that they follow by default, but the flexibility to turn on a dime if a better choice opens up.

The second part I partially agree with. But establishing a routine like meeting some friend every Thursday evening, that can be good.

lets have kids and raise them well is a pretty common shared goal for parents/family

That this personality trait, if it exists, is important for success is pretty obviously true.

If you can measure this trait by putting marshmallows in front of 4½ olds is a whole other question.

... and I guess another question is, how stable is this trait?

E.g. if we got really used to telling 12 year olds that the marshmallow test finding indicates that the ability to put of immediate rewards for larger later rewards is really important, could you effectively get (slightly older) kids to learn to delay gratification more, such that their performance as small children matters less?

Or (more likely) if you raise a generation with more distracting technology, can you destroy a whole generation's ability to patiently wait for a larger reward?

let's ask China

Many years ago, I recall reading in _Columbia History of the World_ that the ability to live in cities, that is civilization, began when people preserved their seed corn so that they could have multiple harvests during the growing seasons.

What I remember is that they summarize this as "Delayed gratification is the root of civilization."

And while this is pretty early in the history of the world book, I read no further because I doubted I would find anything more insightful in the subsequent hundreds of pages.

...

Years later I tried to find that quote and I could not. I still believe it is a valuable insight though even if I hallucinated it.

Why don't squirrels live in cities then?

they do, along with a large number of rats and other vermin

...Fair play.

Did you perform the marshmallow test on your friend as children? If not, I don't even know if you're really talking about the same thing, to be honest. The original study is such a weird and specific phenomenon to which a heroic effort of extrapolation was applied.

"Doing well in life," "delaying gratification," and "long-term goals" are about as far from concretely measurable traits as you can get.

What about a person who always waits to buy games on sale, but has experienced food insecurity and won't pass on free food, even if it's unhealthy? I could go on... there are countless variables when trying to evaluate those traits. What this study is saying is that extrapolating such broad strokes from small indicators is probably not a smart move.

Life can be a lot like a hologram, where the little things show the whole picture.

The marshmallow test is not really testing hunger or self control. It tests how willing people are to align with authority/the bigger picture.

The ideal participant isn't someone doing the calculus that 2 > 1. It's someone who recognizes that they are being tested, and cares about that more than any number of marshmallows.

The question isn't "how hungry am I?", but "what does adult attention mean to me?".

And that's why all of this stuff will stop replicating eventually, why new psychotherapies revert to the mean - it doesn't have the same amount of meaning for the test-givers after decades of trials.

> The marshmallow test is not really testing hunger or self control. It tests how willing people are to align with authority/the bigger picture.

I feel you're making the exact same mistake as the original researchers.

The marshmallow test is a proxy, but it's impossible to say what it's a proxy for in any given individual. One kid will wait because they're scared the researcher will be angry if they don't. Another kid will wait because they recently learned what marshmallows are, and they actually really want to eat two. A third will not wait, because they've never seen a marshmallow before and would rather try one first before getting two.

Agreed. For me the real question isn't whether being capable of delaying instant gratification leads to better outcomes, it's if the marshmallow test accurately measures susceptibility to pursuing instant gratification in the cases that matter.

Like, I've never liked marshmallows. A second marshmallow would have been uninteresting to me. And even if it were I could totally see a kid going "eh, it's just a marshmallow, I'm going to just eat it now and then go think about something else".

Being able to delay instant gratification for greater rewards is only valuable in cases where you actually care about the reward. Someone who applies it everywhere regardless of interest level is just min-maxing life, and it wouldn't surprise me if obsessively min-maxing even little details doesn't correlate with better outcomes.

the confirmation bias is biasing

That's why these bunk psychology studies are so insidious. It might in fact be a real effect! But maybe not at the level of babies and marshmallows.

Some fields of study will always be art, not science.

Literature, art, human psychology. A good writer, artist, or therapist can make a truly great contribution. But they cannot conduct disciplined experiments and establish truth numerically.

And that is OK.

What is not OK is the cabal of academic psychologists who don’t even know that they’re full of shit because they aren’t trained in any of the numerical / “hard” disciplines. (Hard as in well-defined, not difficult).

I remember growing up getting into scenarios like the Marshmellow Test, but I didn't learn to delay gratification; what I learned was I'm a sucker if I wait or make sacrifices. Often "you'll get two later if you surrender this one now" became "there is no second marshmellow and you're not getting the first one back." How many times do other kids have to experience this before they learn not to delay gratification? and thus get accused of "poor impulse control" when I'd call it "learning from experience" ?

Food scarcity will teach you that quick.

Almost the entirety of the lower bracket of employment works this way

“Do customer service for this big tech company and you could get into a corporate role”

Except that the number of people who have every successfully done that is close to 0% of hires

One of the things I love about Reddit is the visibility it has given to the “promises that no one will ever keep” system that runs most retail/service jobs

I've always suspected the marshmallow test measures desire to please the researcher more than anything else.

I'm supposed to sit here and stare at this marshmallow for some indeterminate amount of time, just to get one more marshmallow? Offer me a whole bag and we'll talk. Otherwise, you're wasting my time. My marshmallow would be gone before they could finish explaining the task.

I've eaten a bag of marshmallows today while coding till 4am on a Friday night.

Your proxy tests for self control have no power here.

My parents bringing me in, saying researcher is from a big name university and is very smart and do what he says, is going to have a much different effect on me than my step dad coming late to pick me up because my mom dumped me on his lap last minute because she has a new boyfriend and doesn't have time for me now and also my step dad says university is stupid and for geeks and don't believe a word the stupid college girl tells me to, is going to have a biasing effect on the kid!

When Dieter Uchtdorf was in the presidency of the LDS branch of Mormonism he gave a talk to the entire church about this study. It’s since basically become doctrine in the LDS church. Funny how far and widespread these inaccurate studies can become. And the large majority of the people who hear the original study will never hear that it wasn’t reliable.

Thank god, as I love marshmallows and instant gratification.

I submit to you that marshmallows are incompatible with instant gratification, because they're only good when you slowly toast them over a fire until browned on all sides. Cold marshmallows (or marshmallows shoved in the fire and then blown out) are just sad and not worth the calories.

If you added that (offer them marshmallows and a way to toast them), all you've done with that test is identify the foodies :-) (I 100% agree with you about them being toasted)

They are also good baked with cereal to produce a sweet bar.

Touche. That is a good use for them as well.

Found this article whilst looking for more details, the same results seem to have been reported for several years, including following the subjects into middle age:

https://anderson-review.ucla.edu/new-study-disavows-marshmal...

> As the researchers predicted, the study finds only a tiny correlation between marshmallow test times and midlife capital formation. A graduate’s score on the self-regulation index was, however, modestly predictive of their middle-age capital formation, the study finds.

I expect a contributing factor to this is natural inference. Delayed gratification is just fine for certain windows and in certain domains. But continuous delay (or non-gratifying outcomes) are surefire ends to update priors. Anecdotally, I'm very much in that position. I've delayed and subsequently mistimed my whole life and let's just say my expectations are totally unmet and I'm quickly unraveling into a hedonist plus flagellant.

I'd expect a decay of delayed gratification in aggregate. And this will vary from individual to individual dependant on their expectations/(negative realizations - positive realizations) or similar, and negative realizations are supposedly weighted higher than positive by a factor of 3-5. This exacerbates the rapidity of decay.

I'd posit, then, that delayed gratification can predict within a window; that window may be a "critical window" which leads to enhanced success. Failing to obtain that success then predicts regular decrements to delayed gratification metrics.

And delayed gratification isn't beneficial in all scenarios anyways. Sometimes the payoff is in immediate and remorseless action.

Maybe tempting children with marshmallows is a bad proxy for testing delayed gratification, but the thesis about being able to delay gratification leading to success seems to be true as far as I can tell. Anecdotally, all the people I know who can't delay gratification are just scraping by (this includes another SWE who earns a decent amount but is rather impulsive). All those I know who can delay it are doing great.

Maybe when they grow up some of them learned to steer away from instant gratification. Or maybe you need to account for how big luck is in the success in life

Think of the marshmallow test as a short story by a famous author. It rings with truth, but it's not "science".

"Ringing true" is in the telling. For example, you could imagine a long Paul Graham post framing the 1-'shmallow child as shrewd and decisive, grabbing a quick snack and keeping moving, whereas the 2-'shmallow child is passive and entitled.

It is interesting to read something like this and then go back to YouTube where there are 100s of videos pointing to this test as one of our most important ways to understand psychology and success. I suppose we all parrot things that have little basis in reality and we have not verified for ourselves

YouTube videos are driven by how engaging they are, not by how truthful.

Value of one extra marshmallow: 4 cents.

Time it takes to earn an extra marshmallow: 20 minutes.

Hourly earned value (assuming you like marshmallows): 12 cents.

Reasons not to like marshmallows: The principal ingredient is gelatin, a protein obtained by boiling skin, tendons, ligaments, and/or bones with water. And they don't really taste that great.

It has always seemed to me that the best strategy in this situation is to eat the marshmallow right away in the hopes that the psychologists will let you out of the room early. A better strategy might be to refuse to stay in the room for 20 minutes.

I think all of these studies fail to account for the credit component. i.e. I can see that this man I just met has one cookie, but now he's promising me another cookie (which I may or may not be able to see). And then if I do what he says, he'll give me two cookies. What probability do I assign to the chances he can deliver on his promises? Maybe he's a liar. Maybe before I completed the assigned task, he came across a better deal and allocated all his cookies.

To your point, there are multiple assessments being made, many of which not being accounted for in the original.

Does the listener fully comprehend "the rules" as they're being laid out?

The listener is evaluating the trusthworthiness of the speaker?

The listener may evaluate their own skills in pulling off a deception by taking the marshmallow and lie about it. Due to "the rules" laid out by the speaker, does the listener consider they may change "the rules" (influenced by their historical experience with adults)?

Does the listener place any value on a 'marshmallow' at all, maybe a toy, or a type of item previously identified as having high value would lead to different results?

Adjusting for variables in the 'fuzzy' sciences can be difficult due to the innate subjectivity.

I don’t know how any experiment like this could be taken seriously. Your action could change 15 minutes apart if you feel like having sugar, if the guy felt intimidating or not, if the last thing he said seemed friendly or if his facial expression was off at the very end, etc.

There must be a better way of judging the validity of a social experiment using first principles. There’s a huge psychological side that people completely ignore.

I'd like to see the 2024 version where the kid who got two marshmallows is fat and the one who didn't want any marshmallows at all is skinny.

Funny interpretation. The single marshmallow kid derided for having no self control or ability to delay gratification was actually harnessing the offered deal as a form portion control to maintain a healthy weight.

The study found associations at r=0.17 for both BMI and educational attainment. Not a lot but not zero.

The marshmallow test deals with kids so it’s noisy by nature, that there are two mild associations is interesting. It has mild predictive value.

I think there’s a strong desire to have this test shown to be faulty. Perhaps because the test is so easy to do, parents do it on their own kids and don’t like the outcome.

> I think there’s a strong desire to have this test shown to be faulty.

Indeed, there's enough of a desire that you can P-hack into failing results.

Marshmallow tests are more a test of the child’s priors about adult reliability.

If the child has reliable parents they tend to pass the test. The children of reliable parents do better in life, which is obvious.

The test also fails to account for a temperate child that doesn’t actually want more than one in the first place and isn’t playing the researchers game.

I've never understood the "so what" of this study. Did people not think self control was a virtuous characteristic before? Will they stop trying to teach their kids to exercise self control now that it's been debunked?

Sometimes it feels like much of social psychology exists primarily to sell books and lecture series tickets.

(that said, i'd love to see a study to see if there is a correlation between ability to resist wading into contentious comment threads on hackernews can "reliably predict adult functioning")

So there are two options. You get a coefficient of 0.2 and a std error of 0.2 so you say it’s not significant but the reason is you don’t have enough data so st error is too large. Or you have a coefficient of 0.0001 and a st error of 0.01 so you are pretty sure there is no relationship.

Hidden Brain had an episode on the recent studies challenging the Marshmallow test https://hiddenbrain.org/podcast/when-to-eat-the-marshmallow/

Imagined childish reasoning: I could eat one marshmallow now, and hopefully finish this stupidboringweird test sooner and go home. Or I could be stuck here longer, for one crappy little marshmallow, showing that I know how to play a stupid suck-up little teacher's pet.

How is this childish reasoning?

No interest in gaming the test, to make himself look better to grown-ups.

Obviously short attention span. And no filter on his emotions.

(Which is not to say that it's wrong. Unless you're at serious risk of starvation, a marshmallow is only a feeble token reward.)

By this point all the normal people have started ignoring this type of “science”, many of us were ignoring it from the very beginning. Too bad that this quackery has already made its way into many States’ apparatuses, see the obsession about the nudge thing, for example.

I've always thought this was stupid and obviously not real.

What if the child was being playful by not following the obvious "correct" path? Wouldn't that point to someone who is social and humorous and happy? Isn't that an advantage?

when marshmallow tested, i spit on my marshmallow, when asked why i explained "now noone will want that one", "now i get two marshmallows because i waited" , "and also a third one cause only i will want it"

I think eventually the only famous Psych. experiment which will remain undebunked is the Milgram Experiment, not because it is better than the rest, but because nobody would be allowed to try to reproduce it today.

It's almost like Psych and other social 'sciences' as a whole are unscientific and should be considered liberal arts fields.

The Milgram Experiment is debunked simply by reading about what actually happened in the experiment vs. the narrative people are given after the fact.

The results of this study have always bothered me.

I don’t like marshmallows. Never have. If I was run through the marshmallow test I would have done whatever it takes to get out of there quickly and not have to eat marshmallows.

No wonder, generally speaking. Human nature is way deeper and more complex, more fluid than any artificial model/framework imposed on it.

Psychology is not a reproducible science strictly speaking for that reason.

So in a nutshell, one of the greatest failings of science in history comes down to, researchers were under pressure so they caved and compromise their ethics and morals.

Even worse, the replication crisis is only one reason that the public has continued to lose faith in science in the post truth era.

It’s also the disinformation campaigns that set out to attack whatever’s in a groups interest whether it be politics or the environment.

Maybe the coup de grâce will be social media which encapsulates people into bubbles seemingly impenetrable to the truth.

This result could happen without any intentional conduct, so ironically you may have made a similar error as the original researchers.

While there are very real issues about reproducibility and motivation, rarely do studies actually claim what pop science puts in the headlines.

Popper has a better approach with the idea that evidence cannot establish a scientific hypothesis, it can only “falsify” it.

It is actually how we write computer programs in the modern era too.

The Scientific realism camp is committed to a literal interpretation of scientific claims about the world, but others like myself consider it confusing the map for the territory.

But that is the realm of philosophy and not science.

While the time scale and wasted effort from the flawed original paper is regrettable, this is the process working in the long run.

This paper's falsification is the process working, irrespective of some claims of 'ethics and morals'

Studies about humans will always be subject to problems, exactly because of ethics and morals, e.g the tuskegee experiments.

Of course it’s not all intentional. And it may not even be unethical by their standards as far as certain kinds of p-value hacking are allowed and journals are sometimes are hesitant to call out problems.

But there’s been a huge amount of questionable behavior and there has to be personal responsibility with that. It’s not an overstatement to call this part of one of the biggest failures of science in history, and you can’t just sweep that under the rug as unintentional.

As far as pop science I’m not addressing that but those sins don’t exonerate everything else.

I don’t get what you mean about Popper either, he likely would’ve been all over the reproduceability crisis and calling out integrity as a key issue.

Yes, science is self-correcting and things have definitely started to improve after learning from all of this. But the damage has been done. at the time when we need science, the most it’s been discarded by a significant part of the population.

I can speak to this test a bit from experience: as a very young child, I was in a pilot study used to design a large longitudinal study, and my younger sibling was in that large longitudinal study.

At about age 4, I ended up literally maxing out the delayed gratification test and being sent home with a ridiculously large bag of M&M's, much to this dismay of my mom.

With that as context, I wonder whether some of the changes/lack of reproducibility are actually measures of decreasing economic mobility and economic agency within the US.

Early studies on ability to delay gratification were done during the favorable economic conditions baby boomers grew up in. More recent studies were done in eras with far less economic mobility.

It's quite likely you'd see a smaller effect today, not because the impact isn't there, but because it's so much harder today to make a significant upward change in your economic status.

Good points.

Calling it a test is almost certainly an exaggeration at this point.

Perhaps we could call it "The Marshmallow Trick" now?

I was hoping (based on my initial reading of the HN title) that they tried the Marshmallow test on adults.

I'm confused. How do you access the full text of the article? Why is it behind a $15 charge?

why do you expect it for free? what is a reasonable charge, in your eyes?why is that charge reasonable and $15 isn't?

I don't know. Usually I see arxiv posted a lot here, and I can access those without issue.

If I do pay, do the authors of the paper get my money?

> If I do pay, do the authors of the paper get my money?

In general, no.

Because there is usually an effort in HN to post non-paywalled links.

Sadly Shy-Blub is cucked too, doesn't have it...

Once we control for X

You can control away anything the whole idea of isolation is bunk

>dynamite psychology result with far reaching conclusions fails to replicate

No way?

Try it on shareholders.

This is actually a really great point. If successful people can delay gratification, how do we explain enshittification generally? Running a good company into the ground while enjoying temporary gains is not delayed gratification, and yet the people that are in charge of such things are successful by other metrics (Ivy League, ceo, etc). It’s the marshmallow writ large which is pretty funny. Looks like we all delay gratification if and only if it actually serves us, otherwise snatch greedily at whatever is in reach

Wut? I gave up all those marshmallows for nothing?

If you will allow me to riff off of your jest...

I literally thought that to myself a little over 3 years ago. Yes, literally thought about the marshmallow experiment in the context of my life.

You see, I came within a gnat's eyelash of having the classic widow-maker heart attack -- on the doctor's treadmill. (I didn't. I have no heart damage -- I'm fine, and lucky...) But I did spend a 3 day weekend in the cardiac unit getting a stent put in, and had the opportunity to think about 50 years of life choices. I spent half a century being a 3-sigma gratification delayer. Now, that has had a lot of positive impact on my life, but I also came to the conclusion that it was time that I started eating more marshmallows. (Metaphorically -- literally I eat fewer marshmallows because I am much more careful about my diet...)

So I would not go so far as to say: "Life is short, eat desert first.", but... I will say: "Life is short, don't forget to eat desert in moderation as you go along."

From abstract (article is paywalled)

>Although modest bivariate associations were detected with educational attainment (r = .17) and body mass index (r = −.17), almost all regression-adjusted coefficients were nonsignificant. No clear pattern of moderation was detected between delay of gratification and either socioeconomic status or sex. Results indicate that Marshmallow Test performance does not reliably predict adult outcomes.

I guess the question is whether the covariates that were adjusted for in the regression are true confounders and not, say, something caused by ability to delay gratification.

That’s what I thought too. For example if they “control” for factors like IQ or social economic status, then the correlations will be reduced.

This isn’t a surprise unless you think the delay of gratification is itself the cause of success (seems like a straw man so they can claim to “challenge” the original study)

There is more info from one of the authors here which includes the preregistration document: https://x.com/jess_sperber/status/1818100487964496119

Edit: Also, I think the associations of 0.17 prove the title is false

Yeah, a fucking marshmallow won’t do much to predict you future. Family wealth does.

I don't like marshmellows.

Zira : Because I loathe bananas!

https://inv.tux.pizza/watch?v=G3Phb8AyrxE

That test was already broken a decade ago by Kidd. The socioeconomic part of it is BS and has been known to be for a while.

I was talking about this a few weeks ago and realized I would eat the damn marshmallow. Researchers do not act in good faith. Maybe they're testing me for delayed gratification. Maybe they're measuring my anxiety levels as I wait for someone to come back with a promised reward. Maybe they want to know how angry I'd get if they come back and said they were out of marshmallows - or come back and flat out ate the marshmallow in front of me. A lot of researchers would happily trick me into thinking I was killing someone if they thought they could get away with it.

Its the truth that demolishes all the hand-waving about the marshmallow test - it relies on the subject's trust of the person running the experiment. I wouldn't trust them, why should anyone else?

When evaluated that way - particularly when testing on children - the outcome is painfully predictable.

- Children who have adults in life that they trust have better outcomes.

- Children who do not have adults in their lives who they trust have worse outcomes.

I was a subject in a college psychology experiment when I was an undergrad. The researcher said I would get some amount of $ for each new word in a sequence that I could correctly remember and repeat without error. I mentally made up a story and added each new word to the story. At the end they said that I remembered too many words, more than they had ever anticipated, it was too much $ for them to pay and gave me $5. Later I wondered whether the real experiment was about reacting to being tricked.

Just remember that the reason why Ted Kazinsky said he bombed the federal government was because he was subjected to MK Ultra experiments in college.

I'd say you got a good deal getting two dollars instead of life long trauma.

This has been theorized by his brother and others, but I'm not sure he ever claimed that. He wrote a lot of words in a pretty famous document as to why he bombed his targets, none of which were federal government targets.

Ted Kaczynski didn't bomb the federal government though.

I'd also be curious about a citation for his motivation being the MK Ultra experiments, it's news to me that he ever explicitly called those as a motive.

i think verbal contract law would apply here.

Having been in the former camp to such a heavy degree, I wouldn't have even thought of this dimension as a confounding variable, despite always trying to see that sort of thing.

Thank you for the insight.

Indeed, I couldn't really participate in psychology research because there is almost always an element of deception and I couldn't help but look for it.

One extreme example arguably created the Unabomber.[0]

[0]https://en.wikipedia.org/wiki/Ted_Kaczynski#Psychological_st...

Yet another study that "explains it" turns out to be false. Good.

Failure to replicate could happen for any number of reasons. The sample populations might not enjoy marshmallows the same way!

But, yes, good to be aware of the possibility of both false positives and false negatives.

In general the original study felt like a more widely accepted Myers-Briggs of sorts. But as always happens with people and personality related theories the reality is either "more complicated" at best or the theory is outright false.

I don't think that it's a good thing if a study which seemed promising turned out to be false. The goal is to have explanations of the world, after all. It's better to have learned that something is false than to go on believing the falsehood, but better still is to have something true which explains things.

> don't think that it's a good thing if a study which seemed promising turned out to be false.

It is definitely a good thing.

It is good that the conclusions of a study that was demonstrated to be unsound have been replaced by those of a better study. If some even better study comes along later and replaces this one, that'll be good too. We now know more. It's not fun or convenient, but is generally aligned with the direction science should go.

If people who've made decisions based on their understanding of the results of this study, it's good that they'll no longer labor under a delusion, and can potentially make better decisions.

Good in the sense that hard things which make us incrementally better are good.

hahahaha!

They only measured the subjects' "adult" life outcomes at age 26. Perhaps the researchers were rushing to publish and unwilling to wait long enough for the effect to replicate.

Or perhaps there is no effect.

Or perhaps economic mobility has stagnated and external factors dominate.

fixing the structural problems in the economy is so boring, let's blame the marshmallows

Reply on news.ycombinator.com