Mathiness | Hacker News Reader

Mathiness

votito.com

89 pointsadzicg52 comments

I think Paul Romer, economics Nobel laureate, coined this term in 2015

https://www.aeaweb.org/articles?id=10.1257/aer.p20151066

“ Mathiness lets academic politics masquerade as science. Like mathematical theory, mathiness uses a mixture of words and symbols, but instead of making tight links, it leaves ample room for slippage between statements in the languages of words as opposed to symbols, and between statements with theoretical as opposed to empirical content. Because it is difficult to distinguish machines from mathematical theory, the market for lemons tells us that the market for mathematical theory might collapse, leaving only machines as entertainment that is worth little but cheap to produce.”

Following from "truthiness", coined by Stephen Colbert in 2005 to describe the state of politics.

https://en.m.wikipedia.org/wiki/Truthiness

Romer's examples aren't quite the same as what this page talks about. He describes papers where the maths is valid - the conclusions follow from the premises - but it's not tightly tied to the real world via words.

Contrast to "only with mathematics can the scientist say precisely as little as they mean to say." (Bertrand Russell)

Just in case people think I’m bad at reading with this comment, the original article didn’t cite romer.

I’m glad it’s updated since it’s a good improvement to the article but this makes my comment a bit less coherent.

> In Calling Bullshit, the authors give the example of the Virginia Mason Quality Equation Q = [Ax(O+S)÷W] (Quality equals Appropriateness times the sum of Outcomes and Service divided by Waste), a formula used for improving operations management in healthcare.

I think it's best to think of this as an overextended metaphor, rather than a clumsy attempt to provide a rigorous relationship. The message to take away from this 'equation' is:

* more appropriateness means more quality; * better outcomes means more quality; * better service means more quality; * less waste means more quality.

It's a small minority of people who give a shit about mathematics. The children who used to whine about, 'When are we every going to use this?' grow up to be adults who complain about not having learnt about how to complete their tax returns at school.

The equation above is an aide de memoire for people managing healthcare facilities. It's irrelevant to people who understand that it's an abuse of mathematical ideas.

The 'Fixing mathiness' section, however, is very good. This article is from a survey company, so this is about increasing the validity of the results you get from their product, but it speaks to the problem of inappropriate statistical methods being used to manufacture signals from noise.

That equation predicts that great service can outweigh net negative outcomes.

See you wrote it in a clearly understandable way without abusing mathematics or giving any credence to mathematics being involved in the concept.

Only if they are measured on a scale allowing negative values for outcomes and sufficiently high values for service. It would be unsurprising to have outcomes measured from zero and service topping out at 100%. The equation doesn't say anything remotely precise, it's just a bundle of vibes.

> It would be unsurprising to have outcomes measured from zero

Assuming that you can never make anybody worse off than they were before you showed up would be a much worse abuse.

Or you're using a log scale because your effects span many orders of magnitude, or whatever. The specifics are not particularly central to my point beyond that it isn't too hard to wrangle up some scenario where the equation says something completely different.

I'd read it as a symptom of something going pretty badly wrong. For example, imagine a funny scenario where a business decides that it needs to use computers to improve efficiency. Some desktop towers are bought. The front-line employees use them as the physical foundation for a new table that they needed and report "mission accomplished" up the chain. The problem in this scenario is that while technically yes, real problems are being solved with the suggested tool someone here is badly, badly out of touch with what needs to be happening to see real improvement.

Similarly here, if there is a formula that stupid being used, even as a memory aid, it suggests to me that real lives could be saved if they embed a few real Ops Research specialists somewhere important instead of whatever they are doing.

Using an equation to represent this is dishonest. It assumes linearity and proportionality between variables which may not be the case. Also none of the terms are really measurable. You might as well write statements instead.

I mean try defining waste and quality.

Fundamentally, and to use a non-mathematical term appropriately in context, it is a load of bollocks. It is used to make simple ideas look like they are rigorously defined to people without the tools to interpret them. And that is dishonest.

As for inappropriate statistical methods, survey companies are a breeding ground for providing tools which the results of are not interpreted with any statistical rigour or language.

Source: annoyed mathematician.

+1.

One of the few comments on HN i fully agree with without reservation :-)

While folks may use symbols instead of long-winded statements they need to make it clear that it is merely a shorthand and no mathematical rigour is implied.

>I think it's best to think of this as an overextended metaphor

It may be best, I agree, but it's not what people do. I've seen enough people with advanced degrees take things like FMEA risk priority numbers or adding up the numbers in a Pugh matrix very literally, not as a basis for discussion and insight.

>The children who used to whine about, 'When are we every going to use this?' grow up to be adults who complain about not having learnt about how to complete their tax returns at school.

I've seen my former classmates complain on Facebook about not having been taught things ("there should be a class in high school!") that were absolutely taught in the dumb state required classes.

I'm very skeptical of this view. Additive relationships and multiplicative relationships differ dramatically; the difference is related to linear Vs non linear phenomena. If all you mean is "positively correlated" and "negatively correlated" then you should just say that (or words to that affect).

Exactly this. The equation also fails simple dimensional analysis -- it's nonsense.

I could think of a bunch of simpler ways to state this equation without the baloney false math (similar to the paraphrasing you did).

No, the purpose of putting this into an "equation" is to imply false rigor, and simultaneously to imply that you should hire our people to implement our "healthcare quality framework". I mean, would people really think this is any sort of non-obvious advice if you simply stated, "To improve quality, you should improve appropriateness of treatments, improve outcomes and improve service, and reduce waste." I mean, no shit Sherlock. But once you put it into an "equation", you're deliberately, and falsely, implying "Oooh, science!"

so all of economics, basically.

a mitigating argument for mathiness is that we use math to describe shapes and relationships we can't physically see, and how do you contruct an analogy for a dynamic between factors (or narrative elements) without using changing quantities?

Is the analogy a useful abstraction, or does it provide consistency with lower or higher levels of abstraction, or have external consistency with the rest of maths? Probably not, but as an application that is sufficient for its purposes, some mathiness enables people to separate the things they talk about from just their personal animal interests.

Sure, some people want more from the math, and economics is a great gateway to math because it's probably one of the most sophiticated systems of bullshit ouside string theory, and it provokes the desire for rigour.

Math isn't evidence, it's the lens, and you can reject mathiness in anything by just declining to accept that lens.

Well put! I’m personally a big Marx fan so I won’t give you all of economics, but let’s say “lots” ;)

I feel strongly that math is a set of intellectual tools for dealing with quantities rigorously, so “lens” seems absolutely correct to me. As they say in formal logic: an argument can be sound (well-constructed, uses its tools properly) without being valid (accurate to the actual world) if it employs some bad premises. Which new economics does in spades by constructing inaccurate metrics to do their math with.

Which, hey, I don’t blame em. As the other top HackerNews thread rn on Efficiency and Metrics teaches us: when you’re dealing with the actual world, you can basically never have a flawless virtual metric.

what makes him so evil to me is that where we start with math or geometry and a notion of perfection that we can extrapolate from- one which humans are uniquely imbued to appreciate and emulate, he eschewed the quantiative and replaced it with viral language and sold it as somehow more material and "real."

with any contemplation at all the existence of geometry has necessary and unavoidable moral implications about the possible intent of a creator, and it's hard to see how constructing an ideology to dissolve our connection to those is anything other than a poison. he was a snake and a pimp who wrote a mind virus for self-enslavement as a way to separate people from the dignity of their own humanity. math is not a mystery cult either, and I sympathize with professionals seeing their discipline used for bullshit, but gatekeeping its tools only to initiates should be treated as suspect. to scholars i would say either enlighten people or fuck off, as they aren't equipped for what real gatekeeper personalities are capable of, and a little knowledge won't save you from them.

I always often feel this way about folks who give probability based estimates of their certainty, instead of just describing their level of certainty with human language. Some caveats for things like 99%, which can be said in a way that makes it clear that it is actually just being used as and expression.

A funny way to describe 95% certainty, among a certain type of nerd at least, is a “critical failure.” IMO it is a nice way of expressing the fact that you’d be very surprised to be wrong, but then, D&D has a whole mechanic about 1-in-20 events happening occasionally. All without any numbers.

95% certainty is also, for a normal distribution, two standard deviations. Two sigma events happen very regularly.

https://en.m.wikipedia.org/wiki/68%E2%80%9395%E2%80%9399.7_r...

Yeah, I'd say they happen about 5 percent of the time.

T-Shirt sizes?

> Without a numerical value, such estimates cannot be misused easily in mathematical formulas.

Those units will still be misused and even abused. "What does an M mean in days?", you'll be asked. And voila, there's a number. And, if there's a number there's some equation in which it'll be used.

Agree with the underlying thesis of the post, but wonder if their criteria suffers the same issues as the thing they are being critical of.

I wonder what the author would think of the Drake Equation or quantum mechanic's superposition, as they seem to check a good deal of these boxes but are widely regarded.

The Drake equation is well regarded by people who are terrible at math.

I don’t think I’d call the Drake Equation widely regarded. Personally I can’t stand it, one of the most quintessential examples of “mathiness”, though at least it’s dimensionally sound xD (it’s all probabilities).

I don’t know what equation you mean by “quantum mechanics’s superposition”, but most equations used in quantum mechanics actually are meaningful and represent some physical phenomenon to a high degree of accuracy though (I.e. they make predictions which can be verified through observation and measurement).

The first factor of the Drake Equation (the average rate of star formation in our Galaxy) has dimension 1/time. The last factor (the length of time for which such civilizations release detectable signals into space) has dimension time. The other factors are unitless ratios.

So yes, the Drake Equation is dimensionally sound.

It is also a true equation. Not observed to be true, but reasoned to be true. The right hand side is the left hand side multiplied by some non-zero factors, and divided by the same factors, and rearranged (into the multiplication of a time rate, 5 ratios, and a length of time).

There’s still a lot of uncertainty about the numerical value of some of the factors — but the math is sound.

Therefor I wouldn’t put the Drake Equation in the same category “mathiness” that the original article is talking about.

This makes me wonder if there is some useful way to apply lattice theory to software development estimation. Essentially, take the tasks and compose a lattice using a “definitely harder than” relation. While we are observably bad at predicting just how hard a task is, I believe we can do a much better job of answering is A definitely harder than B with a yes or no. And if we can’t then we provide no answer while building the lattice.

And then, well I have no idea. I’d have to build some examples and play with them.

Yes you can do pairwise comparison and ranking of subjective things (how long will this task take) to get a less arbitrary measure of it.

In the past I've done this for consumer product testing - if you ask someone to rate hair straightness out of 10... it's just really difficult and inconsistent. But you can show them two photos they can usually pick the best, even if it feels pretty arbitrary.

Bradley-Terry is the standard method, it's extremely easy to use.

I have never seen anyone try it for bug estimation though. I think it would be too much effort and tbh the industry hasn't even figured out that you need to be able to write down confidence for estimates yet. There's a long way to go before we start doing things properly.

> In Calling Bullshit, the authors give the example of the Virginia Mason Quality Equation Q = [Ax(O+S)÷W]

I read somewhere (maybe in Thinking, Fast and Slow?), that formulas like this can actually be surprisingly effective, even though the units don’t make sense, because they encode an intuition but prevent you from putting your finger on the scale when applying it, by mixing in other biases. IIRC, the studies on this found that trying to tune these formulas by adding in weights tended to make them worse, which is especially surprising.

I’ll have to see if I can dig up the reference on this.

This post essentially described Pincipal Component Analysis in its entirety.

Trying to find eigencomponents in a mixing of incompatible units means that your result is completely arbitrary to the scale factor of your units.

And "fixing" the dimensionality by choosing an arbitrary normalization factor only further increases its mathiness factor.

1. If I wanted to use PCA, is there a better approach than normalization? I am asking because you seem better at math than I (based on your comment history).

2. Not that PCA is my favorite technique, but you can apply PCA to data that are expressed in the same units.

3. Do you oppose most clustering techniques since one can apply them to mixed units data?

1. Yes, take the logarithm, especially if your data domain is a constitutive quantity (i.e. there exists an unequovical "absolute zero"). It converts multiplicative scale factors into translational offsets. If your data domain have negatives, consider whether if it's just an artifact of your relative measurement and try to re-offset your data based on reasoning. If your data is a positional quantity, consider if there exists a corresponding metric for which your data is an exponential measurement of (e.g. acidity <-> concentration), and re-shape your log base appropriately.

2. Almost yes, but you also need to make sure that there can exist a rotational symmetry in the measurement metrics, unless it is precisely the asymmetry which you are measuring (but then that would mean you have determined a priori that it's axis-aligned?)

3. Depends, I cannot say I am encyclopedically well-versed to give a judgement on the proportion of how many techniques are well-founded and how many are not. However, I can give you an example of one which is rigorous and spectacularly elegant -- see Michael F. Ashby's work.

Woah that’s an unexpected and very hot take! What application of PCA are you familiar with that’s flawed? I learned it in the context of data science, where a) parameter normalization is a very important setup step, and b) it has directly observable empirical success.

Like, I believe you that it’s used for Mathiness bs by some people, but disregarding it in its entirety seems like disregarding linear regression, or, idk, long division. I didn’t realize that was an option!

I guess parent is talking about using PCA for doing something akin to Factor Analysis (and making interpretations over the components and the loadings / contribution from each variable towards each component). I have at least seen that in psychology, for example the Big 5 personality traits were extracted and then analyzed/discussed in that way, using FA though.

I also don't think this is about disregarding the methods usefulness, but rather being very careful with assumptions and use critical thinking when interpretating results in studies that use a lot of math.

In the case of DS/ML for prediction, it doesn't matter at all. As long as it empirically works, you're good. Unless you want very explainable/understandable models that is.

PS: I don't like using wikipedia as a source, but you can take a look here as an example of what parent is talking about: https://en.m.wikipedia.org/wiki/Big_Five_personality_traits#... . As I said, in this case is for FA, but a lot of people use PCA for making the same kind of studies, specially in less rigorous data-science for business settings.

Adding quantities of wildly different units is totally unsound [1].

______________

[1] except in machine learning

OLS legitimately incorporates all unit conversions in the betas. That's gross but sound.

If at the end of the day you have to compare wildly different things, you have to compare wildly different things. Usually this will involve making a judgment call about how to weigh these things, or in other words, selecting units for the purpose of adding quantities of wildly different units.

Sophisticated marketing annd political campaigns include everything from bikini clad women to academic papers.

Using mathematical language is always a rhetorical choice to suggest precision and authority.

so much of economics is just mathiness

The term was introduced by Romer with respect to economics and predates the book referenced in the article: https://en.m.wikipedia.org/wiki/Mathiness

E = mc²+AI

https://x.com/karpathy/status/1663393508240261122

Management's favourite equation: Power = Work Over Time

Mathiness + AI

I partially attribute this issue to this paper from 2012:

"The abstract that included the meaningless mathematics tended to be judged of higher quality. However, this “nonsense math effect” was not found among participants with degrees in mathematics, science, technology or medicine."

https://www.cambridge.org/core/journals/judgment-and-decisio...

unblock everything

What is bullshit is the attempt to appropriate a useful neutral adjective to mean more than it does. Mathy is just mathy. Physics is more mathy than psychology. Too mathy can be bad, sure.

You may not be familiar with the "truthiness" term/meme it is a reference to.

https://en.m.wikipedia.org/wiki/Truthiness

H/t Colbert.

I see it as attempting to provide a patina of quantifiability for an essentially qualitative or incalculable result. If it is actually used as a measure, then it will lead to horrible distortion. The "next article" on the site is about OKRs (objectives and key results) and how they can be used to motivate and also destroy motivation, inspire and also make people afraid, by how faithfully you stick to the measurement and how much you assume the Os and KRs are actually tied to business results. The example of the Virginia Mason Quality Equation is somewhat egregious because it is presented as an equation but nothing in it is properly quantifiable, the units are meaningless, and the authors say it's not to be calculated.

However, there is value in putting everything in a mnemonic so that you remember its parts, and what adds and subtracts from the results. Amazon shipping and FedEx Express have different types of quality. Amazon is next-day or better for virtually everything and unreliable. It comes with a guarantee to refund the cost of shipping, which is 0 in most cases. FedEx Express has varying service levels at increasing costs for performance and includes meaninful guarantees and insurance tied to the performance goal. If Amazon fails to deliver on time, it is frustrating but probably not typically enough to make the user quit using the service or attempt to sue, as they didn't pay for specific service and they are satisfied with the overall expediency of the service. If FedEx Express had Amazon levels of performance, they would lose to competitors who are willing to provide the requested service, regardless of whether they refund people's delivery charges. Putting it back in the equation, the Appropriateness of the service dominates the difference between Amazon Shipping and FedEx Express, where Amazon can tolerate more bad outcomes in their model and FedEx can tolerate more waste (higher fees). Amazon can impress you with same-day delivery you didn't ask for while FedEx has no need to do so, people want merely on-time delivery at the price level they requested.

In the past century Psychology transformed from speculative writing combined with clinical observation, to sophisticated statistical analysis and attempt to follow scientific methods, because many audiences including yourself believe that makes it more true.

Reply on news.ycombinator.com