Undisclosed tinkering in Excel behind economics paper
retractionwatch.com
retractionwatch.com
(Edit: I wrote “community college” because I though that was what the grand-parent had said - but it was just “college”.)
In the United States, community colleges are primarily two-year public institutions of tertiary education. Community colleges offer undergraduate education in the form of an associate degree. [In addition community colleges also offer remedial education, GEDs, high school diplomas, technical diplomas and academic certificates, and in rare cases, a limited number of 4-year bachelor's degrees.]
https://en.wikipedia.org/wiki/Associate_degree
An associate degree or associate's degree is an undergraduate degree awarded after a course of post-secondary study lasting two to three years. It is a level of academic qualification above a high school diploma and below a bachelor's degree.
The latter offer recreational courses, like painting or language learning or Indian cooking, usually one hour per week for a quarter year or so. The courses are held by laypeople.
"In most [UK] universities, professorships are reserved for only the most senior academic staff, and other academics are generally known as 'lecturers', 'senior lecturers' and 'readers'."
https://en.wikipedia.org/wiki/Academic_ranks_in_the_United_K...
This is not true. The law defines which schools are allowed to call themselves "hogeschool", and also that the English translation of that word is "university of applied sciences" or, if limited to a specific job profile, "university of <said profile>".
See https://zoek.officielebekendmakingen.nl/stb-2017-97.html
> Het voeren van de naam hogeschool is voorbehouden aan de instellingen voor hoger onderwijs die zijn opgenomen in de bijlage van deze wet onder g, en aan rechtspersonen voor hoger onderwijs. Onder het voeren van de naam hogeschool wordt tevens het voeren van deze naam in samenstellingen verstaan, alsmede het voeren van de naam hogeschool in vertalingen. In afwijking van artikel 1.22, eerste lid, wordt de naam hogeschool in het Engels aangeduid met «university of applied sciences» dan wel, bij hogescholen die opleiden tot een bepaald beroepsprofiel, «university» met daarachter het vakgebied.
But that‘s intentional. Former FHs are practically identical in law to universities. They all have the right to promotion, their masters and bachelors are legally identical to universities‘ masters and bachelors.
There's a very clear distinction between FHs and Universities. Not necessarily in reputation or level of education, but they are separated entities. And you cannot get a PhD from an FH.
In short, bachelors and masters are indeed legally identical to the ones acquired at universities, but most do not have the right to promotion. This mostly depends on the state they're in. And they are not allowed to call themselves "Universität" (though some have rebranded to "Hochschule").
All FHs that I know do have an option for getting a Phd through them, but they themselves can't grant them. Instead, an FH can cooperate with an Universität through a sort of loophole. The Phd students spend their time at the FH, but they have two supervisors and get their Phd from the "Universität".
But it seems like it is only a matter of time until difference, too, is no more.
The RWTH in Germany still uses Hochschule, but does add University now to their name.
And if you don't trust Wikipedia, the RWTH refers to itself as an "Universität", see here for example: https://www.rwth-aachen.de/cms/root/Die-RWTH/Aktuell/Hochsch... Doing so would be illegal if they weren't one, so I don't see where the claim that they're not comes from.
(Slide 9, for example).
Edit: you are right that the difference might not always apply, but they are not always identical by default as far as I understand it.
Universities of applied sciences are governed under the "Fachhochschulgesetz"[1] (Universities of Applied Sciences Law) while universities are governed under the "Universitätsgesetz"[2] (Universities Law). This carries loads of differences with it, i.e. when it comes to academic freedom and right to promotion. Professors at universities of applied sciences are much more limited in their academic freedom and these institutions do not possess the right to promotion at all.
Not sure where you got your information as it's wildly untrue in some respects.
[1] https://ris.bka.gv.at/GeltendeFassung.wxe?Abfrage=Bundesnorm...
[2] https://ris.bka.gv.at/GeltendeFassung.wxe?Abfrage=Bundesnorm...
Enough harassment for tonight?
So "Technische Universiteit Delft" is not an advanced institution?
In either case, in Sweden "högskola" covers everything from small rinky-dink schools up to KTH (Kungliga Tekniska Högskola) in Stockholm and CTH (Chalmers Tekniska Högskola) which are among the top engineering schools in Sweden.
Both award doctoral degrees. I have not heard many people in Sweden argue that KTH/CTH are not as advanced as the top universities. The difference is just that the technical schools do not have departments for the humanities.
It is, and it is also one of the 14 Dutch institutions that are allowed to use the Dutch "universiteit" name. But the English term "university" seems to be unprotected in the Netherlands.
(And then, of course, there’s also the term ‘volksuniversiteit’…)
It still isn't. Unlike KTH and Chalmers, "Jönköping University" is listed as a university college, not a university: https://www.uka.se/swedish-higher-education-authority/about-...
These are the checks and balances that are in place, and unfortunately looking up from the faded old daguerreotypes of Ådalen 1931 and addressing issues from this century does not seem to be at all a priority.
You can get some action if you manage to dress the problem up as as a J.P. Morgan type industrialist opressing the workers (as happened with the Tesla blockade), but if it doesn't fit that narrative you're generally SOL.
> To put it very simply, the difference between universities (universitet) and university colleges (högskolor) is that universities can award degrees at the doctoral (third cycle) level because they have general permission to do so.
The text wrongly implies that "högskolor" cannot award doctoral degrees. This is not true. It's just that they don't have a built-in general permission to do so.
In practice, KTH/CTH etc have a large number of doctoral students and award many doctoral degrees every year.
And as was written above, some establishments, as they grew from institutes to universities, decided to keep the old name, MIT and CalTech included.
The institutes of technology have been reorganised lately and are now called “technical universities”.
The degree you get is generally much the same from any of them - but for some insane reason graduates of the institutes of technology/technical universities don’t get to vote in the national elections for the upper house (senate). Only graduates of the more prestigious ones get that right.
The whole bloody upper house is insane in Ireland, but it's convenient for the legislators so it doesn't get reformed.
It was deeply funny listening to the sitting senators tell us how important the Seanad is, because for all of us - the Seanad was some useless shit we couldn’t vote for or influence in any real way.
I have no idea if it's useless shit and I admit to learning about the Seanad 15 minutes ago from Wikipedia but it seems to me that it's supposed to be a weak technocratic advisory chamber insulated from popular will. At least it seems to make more sense than the House of Lords.
In 1979, there was a referendum in which 92% of the voters were in favor of extending Seanad representation to graduates of other higher education institutions. This changed the constitution in a way that allowed laws to be passed to implement that, but no such laws have yet been passed. There was a ruling by the Supreme Court of Ireland in 2023 requiring a law to actually be passed, but the ruling is currently suspended until 26 May 2025.
https://en.wikipedia.org/wiki/Seventh_Amendment_of_the_Const...
As a graduate of an institute of technology, having transferred from a university, I don’t get to vote in the Seanad (senate) elections at all.
To clarify: most upper house seats are elected by members of parliament. There’s a pool of seats reserved for graduates of a subset of universities to vote on.
It’s terribly undemocratic and needs to be reformed or abolished.
But the Irish Senate has a relatively unusual composition and voting system anyway: https://en.wikipedia.org/wiki/Seanad_%C3%89ireann#Compositio...
The difference here is that you are allowed to wear the title of MSc either way and both programmes do confer ECTS points - only the latter does not qualify you for a PhD.
nothing else.
usually just adding a business major to a place with medicine, engineering and law is enough to cross the threshold and jump from college to uni.
To circumvent this, private non-state-certified schools deliver things called "Bachelor", "MSc", "MBA" or "Mastère" that walk and quack like regular diplomas but since they're using those specific names, they're not actually state-recognized diplomas. Which means that if a company is hiring "people with a Licence" they can reasonably deny someone with a "Bachelor" because even though it's also a 3-year degree, there is no guarantee whatsoever that it's actually worth anything. Anybody can create a company and hand out sheets of paper with "MSc in Psychoceramics" written on it.
This is a big problem for high schoolers who usually don't know the difference and end up paying thousands of euros for private schools that deliver close-to-worthless degrees (not all are worthless, but their worth is mostly correlated to how well-known the school is, whereas if you get a real state-certified degree from a random university anywhere in France it'll be recognized by any company).
I have seen this with my own eyes. A bit of excel drama is usually behind their published papers. Also seen that with my own eyes.
The field has trouble collecting data too because it’s expensive and very subjective. The worst one I saw was somehow turning 7 data points via imputation and interpolation into a hundred or so. The principal researcher on that I actually had to teach BIDMAS to and her calculations were completely whacked out (excel again)
Try marine biology, they are always looking for decent papers.
Over 50% of social science studies do replicate.
Sorry that isn't high enough. We should definitely just give up on understanding ourselves.
Or, maybe fix the financial incentives.
Any industry across all fields, if the incentives are miss-matched, will produce crap.
There is some tasteful irony here. Could you link to a study on this claim? And ideally a few other studies that support its claim? Thank you.
>Study replication rates were 23% for the Journal of Personality and Social Psychology, 48% for Journal of Experimental Psychology: Learning, Memory, and Cognition, and 38% for Psychological Science. Studies in the field of cognitive psychology had a higher replication rate (50%) than studies in the field of social psychology (25%).
https://en.wikipedia.org/wiki/Replication_crisis
Definitely better than medicine!
>Of 49 medical studies from 1990 to 2003 with more than 1000 citations, 92% found that the studied therapies were effective. Of these studies, 16% were contradicted by subsequent studies, 16% had found stronger effects than did subsequent studies, 44% were replicated, and 24% remained largely unchallenged.[76] A 2011 analysis by researchers with pharmaceutical company Bayer found that, at most, a quarter of Bayer's in-house findings replicated the original results.[77] But the analysis of Bayer's results found that the results that did replicate could often be successfully used for clinical applications.[78]
25% of experiments replicating in-house is crazy low, but explainable due to very different incentives (the researchers need to get the paper out; the Bayer scientists need to have a functioning product)
I'd like to see this across all, including STEM.
Note: I'm using 'social science' colloquially. On HN, seems like Psychology is a 'social science'. Anything dealing with people is deemed a 'social science'. Even though that isn't technically correct, they are separate fields. There is a 'Social Science' field and a 'Psychology' field.
The Quarterly Journal of Economics, Oxford University Press (colloquial known as "the QUGE")
Econometrica, Econometric Society
Journal of Economic Literature, American Economic Association
Journal of Economic Growth, Springer
Journal of Financial Economics, Elsevier
Maybe swap in the Journal of Political Economy if you're looking for more political leaning research. Swap in the American Economic Review if you're interested in more American-centric research.
- Econometrica
- American Economic Review
- Quarterly Journal of Economics
- Journal of Political Economy
- Review of Economic Studies
[1] https://newsroom.iza.org/en/archive/research/the-tyranny-of-...
The suggestion above that JEL is a top five is bizarre. The CS equivalent is saying that the book “learn python in 21 days” is a top outlet for CS research.
The authors of this paper should be deeply embarrassed about the way they handled that data, but I’m slightly relieved to see that this garbage was published in a tenth-rate journal no one has ever heard of. (Though to be clear there are definitely problems with what gets through in top journals too!)
> In email correspondence seen by Retraction Watch and a follow-up Zoom call, Heshmati told the student he had used Excel’s autofill function to mend the data.
The most charitable interpretation is that the professor has no idea what autofill is or isn't capable of doing, so he misused it.
"Um, both countries start with the letter N"
When he dragged down a selection he _saw_ the values being copied or a trend being extrapolated.
But neither these tools nor an LLM has real understanding, they're not going out into the real world and collecting data. The results need to be verified. Automatically filling an index column with 1,2,3... is one thing, automatically filling a data column with guesses from pattern matching is different. This problem is only going to get worse as LLMs proliferate and can more reliably but still imperfectly and still opaquely fill in more complicated data.
It sounds like he didn't realize or notice that autofill was using adjacent cells (from other countries, in this instance). That behavior would shock me as well. I thought autofill only took columns of data into account.
No, it doesn’t sound like that: “in several instances where there were no observations to use for the autofill operation, the professor had taken the values from an adjacent country in the spreadsheet.”
It was not Excel’s fault.
If they'd been using R/Matlab/Python written by a copilot it would have been just as much gibberish.
The headline isn't blaming Excel. "Undisclosed tinkering in Excel" is not far from the accurate headline: "Misuse of commonly-used Excel tool..."
The interesting thing is that a supposedly sophisticated person was using an unsophisticated tool and then relying on the data it produced. Failing to mention "Excel" in this case is doing a disservice to the thrust of the problem.
The tools have nothing to do with it, and the same could have been done with paper and pencil. Autocomplete is the problem, it just repeats a single number or a series, and when using it you see exactly what the results were.
This was not done by mistake or in ignorance or because the tool encouraged them, it was done to knowingly falsify incomplete data so that they could finish their paper and publish.
To compare OECD countries.
What’s the point of even doing the research?
In combination with the (reportedly)blatant lack of any reference to the methodology in his paper this feels like wilful gross misconduct in professor-land.
IMO this sort of thing rightly undermines confidence and credibility of research in a much broader context and should be a sackable matter if shown to be true as asserted.
That isn't to diminish when Excel's help has disastrous consequences, but rather to say it shouldn't be unexpected. If your data requires precision, there are better tools than Excel for working with your data and there should be steps taken by more experienced members of the team to ensure the team doesn't get bit by Excel's features. Excel's behaviors are well known and I'd have to imagine at least one member of any research team has been bitten by Excel's assistance features before during research. I would be surprised if any serious educational institution that teaches about ethics and common pitfalls in research *didn't* caution about how Excel might corrupt research [0].
0 - Okay, maybe not _that_ surprised, but I would be disappointed and think less of the institution/instructor.
Someone double checked that and found that there was an error in the equation is correction show that austerity programs were mildly worse for recovering from economic disasters.
--
On an editorial note, it seems to me that economics is just highfalutin astrology and this does nothing to convince me otherwise
> In email correspondence seen by Retraction Watch and a follow-up Zoom call, Heshmati told the student he had used Excel’s autofill function to mend the data. He had marked anywhere from two to four observations before or after the missing values and dragged the selected cells down or up, depending on the case. The program then filled in the blanks. If the new numbers turned negative, Heshmati replaced them with the last positive value Excel had spit out.
This quote seems unintentionally telling:
> "If we do not use imputation, such data is almost useless,” Heshmati said. He added that the description of the data in the paper as “balanced” referred to “the final data” – that is, the mended dataset.
Lets not forget that people/politicians also will ignore errors if they support their point of view. This can lead to policy decisions that impact the global economy.
The US 'Right' believe strongly in biblical/moral individual responsibility, so any study that supports that gets a pass.
This lead to 'austerity' measures after 2008. Because "everyone should just be more responsible in their spending".
""They actually found, using a different method, that the economic growth would be around 2.2 percent. Reinhart and Rogoff admitted that their spreadsheet was accidentally omitting 5 rows in an Average formula, and claimed that the corrected result would be a positive growth but of just 0.2 percent.""
https://www.powerusersoftwares.com/post/2016/08/11/the-excel....
https://www.theguardian.com/politics/2013/apr/18/uncovered-e...
https://www.washingtonpost.com/news/wonk/wp/2013/04/16/is-th...
But if you knew anything the history of science, you'd know that a sloppy scientist with good tools gets you sloppy science, and a diligent scientist with bad tools might still get you as-good-as-possible science.
Improvements in tooling raise the baseline universally, making the "sloppy" scientists malfeasance obvious and helping the "diligent" scientist avoid inadvertent mistakes.
Moralizing about humans being humans and wishing we were better gets us nowhere, but technology carries us into the future.
https://www.youtube.com/watch?v=WXAGhtqI5xw
Is is jaw-dropping that a professor would use Excel auto-fill.
Admittedly this isn’t my field. But that sounds a lot like “I only show my work to people who will validate it,” does it not?
It doesn't put him in a much better light, though. It's like he's so incompetent that he didn't even realise that there was something that needed covering up.
It's likely that this was omitted from the paper as an oversight; such imputation likely wouldn't have affected the outcome of peer review.
Only if you did that with this paper, you’d have bounds so wide that no useful conclusion could be drawn, as indicated by the author’s statement that without imputation such data is useless.