Statistician Proves Gaussian Correlation Inequality | Hacker News Reader

Statistician Proves Gaussian Correlation Inequality

quantamagazine.org

316 pointstambourine_man71 comments

> Not knowing LaTeX, the word processer of choice in mathematics, he typed up his calculations in Microsoft Word

...

> He opted instead for quick publication in the Far East Journal of Theoretical Statistics, a periodical based in Allahabad, India, that was largely unknown to experts and which, on its website, rather suspiciously listed Royen as an editor.

I couldn't help but laugh at this part. He proved a classic theorem in statistics and then precision engineered the actual submission to be roundly ignored.

Makes me wonder if there's a proof of P=NP with ClipArt illustrations that's buried in the South Asian Journal of Mathematics.

While it's hard to believe, there are entire fields where top journals can take 2 years to publish (no early publications, no culture of preprints, etc.) For example, in one subfield of EE where I once worked in mid oughts, the choice ws between an IEEE journal that took 2.5-4 yrs to publish or lesser known IET and online journals that published faster.

Such fields are few in 2017. But it's not implausible that in 2014 the only two quick forums in statistics he was aware of were Arxiv and lesser known journals.

The great thing about the arxiv, though, is that you can submit the preprint early while the paper goes through review at a 'real' journal.

Review takes time... Unless you submit to your own journal... I'm surprised good friend didn't help with a better choice of journal, even a non top tier one.

There's entire trails of papers that cite each other on arXiv while they wait for a 1-2 year review for journal publication. It's crazy — the journal has in some regards become a formality necessary for career advancement, while the actual research in a field progresses in the form of postings on a preprint server.

> the journal has in some regards become a formality necessary for career advancement,

I think this is ever so slowly going away - the entire format of the journal is still a hangover from a print-only era and really needs to go away.

There are some good news: the NIH (medicine's US funding body) now allows preprints to be cited in grant proposals, one nail in the coffin of journals. See http://www.sciencemag.org/news/2017/03/nih-enables-investiga...

Are you saying that journals are going away in favor of unpeer reviewed preprints? That seems like a big step backwards.

It is a step forwards. The important thing is to get rid of the journals, then new evaluation systems can rise from the ashes. The peer review is letting more and more crap through anyway.

Peer review mostly guarantees that the paper is not ridiculously wrong. It's a pretty small hurdle to clear and whether or not a paper passes peer review depends mostly on the mood of the reviewers. I have received completely contradictory reviews for the same paper.

Depends on the venue you publish in. One side effect of getting rid of peer reviews is that you need another indicator if a paper is worth reading, most of the time that's going to be author reputation. This means it's difficult for work from lesser known authors to get noticed (similarly to what happened here).

The willingness to cite a preprint would seem in and of itself to be a form of peer review, no?

> I couldn't help but laugh at this part. He proved a classic theorem in statistics and then precision engineered the actual submission to be roundly ignored.

He simply did not care:

“I am used to being frequently ignored by scientists from [top-tier] German universities,” he wrote in an email. “I am not so talented for ‘networking’ and many contacts. I do not need these things for the quality of my life.”

Sounds like a bullshit excuse. I don't understand why anyone would even argue that they intentionally do a poor job presenting their work because they somehow allege they are ignored.

The only reason why anyone would go through such pains to get ignored is if they intentionally want to be ignored.

And this does not make any sense, particularly in a "publish or perish" environment.

It is clearly stated in the article that Thomas Royen just wanted to get published quickly, as he isn't an academic.

I on the contrary find it perfectly reasonable. Academic publishing has its own share of bullshit and I can well understand someone, specially a retired person, wanting to give up on that kind of prestige in order to save time and energy for actual science. For example, as far as I know, Grigori Perelman has not published the work that won him an offer of the Fields Medal in any journal.

I sympathize, after getting my PhD I went into the private sector. I often have side projects that could turn into publications, but massaging something through the publications process is a part-time job itself, and there's definitely quite a bit of who-you-know that can muddy things up.

If he really didn't care, wouldn't it have been easier and more effective to only publish on the arxiv, and perhaps to tell people working on the problem about his solution?

If you're old and don't know LaTeX? Probably not, especially since he was an editor with the journal he published in.

He did publish it on arXiv

> Not knowing LaTeX, the word processer of choice in mathematics, he typed up his calculations in Microsoft Word, and the following month he posted his paper to the academic preprint site arxiv.org.

> Makes me wonder if there's a proof of P=NP with ClipArt illustrations that's buried in the South Asian Journal of Mathematics.

For some reason, the gem implausibly hidden in an obscure Indian publication reminded me of Borges' The Secret Miracle [1], where God is hidden in a single letter of a single book in the Clementinum library. It also makes me wonder, how many talents are forgotten, how many revolutions hidden away collecting dust?

---

[1] https://en.wikipedia.org/wiki/The_Secret_Miracle

When I worked at the public library as a teenager, I proposed a "hundred dollar bill contest," where we would announce that there was a hundred dollar bill in one of the books. The head librarian got a horrified expression when she realized what I was proposing, and threatened to fire me on the spot.

So I wouldn't mess around with God. ;-)

Maybe this explains the time I found a $20 bill in a rather obscure library book.

The university library I worked in as an undergraduate had a $100 note inside a hard copy of someone's PhD thesis, put there by the author.

It was still there when I left, four years later. The secret of its existence was amazingly well kept by the staff - and nobody read the thesis.

Should have used a cheque

Why?

Presumably in order to notice when/if it's cashed?

I think it really takes some of the magic away for the recipient, and the wonder away from the author don't you think?

Sometimes it's nice to just put a bit of magic out in the world and let it be without running tracking on it.

I seem to recall that this has been tried and then hordes of people stormed the library, went furiously through the books and basically threw them on the ground, making a huge mess until the bill was found. Not exactly what the librarians had in mind.

The thing that's interesting to me is that he knew that this was a famous unsolved problem, knew that he was correct, wanted to take SOME amount of time to publish it, and then didn't care to get it out there at all.

I feel like my course of action would to be to send it to the guy at PSU he contacted and say "if you want to handle the publication bureaucracy, you can be second author."

In biology, the reason why there was a thirty-five-year gap between the publication of Gregor Mendel's classic paper on peas and the birth of genetics was that his 1865 paper was published in an obscure journal.

Great example.

It also seems to have been the case that when he became Abbot of his monastery in 1867 he ceased research.

A simple proof of the Gaussian correlation conjecture extended to multivariate gamma distributions by Thomas Royen: https://arxiv.org/pdf/1408.1028.pdf

Mathematics has this funny habit of throwing around "it is easy to say that ...", or in this case, calling a proof that eluded mathematicians for decades "simple."

Simple (easy to say) is not the same as being easy to deduce. A needle is a simple object, but finding one in a pile of hay takes a lot of work. Likewise, it is hard to pick out the features that make a problem simple when they are embedded in a sea of complexity. It becomes easy to say once you are no longer faced with the difficulty of knowing the right thing to say.

> A needle is a simple object, but finding one in a pile of hay takes a lot of work.

1. Gather all the hay in multiple trash bags (this assumes that the needle remains with the hay and doesn't fall out)

2. After gather the hay, use a roller-magnet pickup tool over the area the hay was at to find the needle if it fell out of the hay (assumes needle is made of ferrous magnetic material)

3. Place each bag, one by one, inside the torus of a CT scanner. Turn on CT scanner.

4. Remove bag of hay. Check inside of CT scanner torus for needle.

5. Repeat as needed with other bags.

Alternatively, one could set the hay pile on fire, then run over the ashes with the magnetic pickup tool.

Alternatively I could step on the hay with a bare foot, because if the needle is in there, I'll step on it for sure.

Or you could just burn the hay.

>Simple to say is not simple to deduce.

Case in point: Fermat's Last Theorem, or Collatz Conjecture

Well, no, the point isn't that simple propositions may have complex proofs, the point is that a proof may itself be simple, though hard to discover in the first place.

You just need to understand the language.

"Trivial" = "regularly covered in undergraduate courses"

"Easy" = "a good topic for an undergraduate honour's thesis"

"Non-trivial" = "a good PhD thesis topic"

"Distinctly non-trivial" = "a groundbreaking result which will establish a professor's reputation in the field"

I had a professor who liked to talk about how "elementary" did not mean easy, it just means "uses only the foundations."

This proof is a perfect example of that statement. Formulating the problem the right way -- which is most often the hardest part -- was the real challenge here, not the mechanisms needed to do the formulation or the proof.

Not sure why, but this reminds me how we had a professor that would say

  Let y(x) = a*x^4 + b*x^3 + c*x^2 + d*x + e,
  whereby e is not _necessarily_ the base of 
  natural logarithm

My favourite along those lines was "let epsilon be a small number which is not necessarily greater than zero". Everybody who spent the preceding year on epsilon-delta proofs did a double-take at that.

There's a wonderful book, by the way, Proofs from THE BOOK, of, well, simple beautiful proofs. The book is named after

> mathematician Paul Erdős, who often referred to "The Book" in which God keeps the most elegant proof of each mathematical theorem. During a lecture in 1985, Erdős said, "You don't have to believe in God, but you should believe in The Book."

https://en.wikipedia.org/wiki/Proofs_from_THE_BOOK

This made my day!

And of course, those change over time.

I had an undergraduate course my freshman year where we went through a circular proof of the equivalence of twelve or thirteen formulations of the axiom of choice. A hundred years ago, proving many of the steps of that proof might well have been non-trivial, perhaps even distinctly so.

Easy to prove =/= Easy to discover (+).

It is simple to say E=MC^2.

That doesn't take away from the fact that it was very hard for it to be discovered.

And it's a good thing. It's a good thing that things which were hard to discover can be communicated easily. Otherwise we wouldn't be able to compress thousands of lifetimes worth of discoveries into a one-semester lecture.

---

(+) footnote: well, unless P=NP.

Abstract link: https://arxiv.org/abs/1408.1028

The proof is hard to read for non-statistician. Is anyone willing to give a short human-readable summary of how it works?

Also, the conjecture seems fairly simple, so it'd be great to understand what made it hard to prove.

Here is a blog post that I think clarifies why this conjecture had remained unsolved: https://almostsure.wordpress.com/2009/09/27/the-gaussian-cor...

There is also a 'companion' article on arxiv that provides a clear presentation of Royen's proof: https://arxiv.org/abs/1512.08776

terry tao has a great discussion of the entropy power inequality

http://mathoverflow.net/questions/167951/entropy-proof-of-br...

From reading the article, it seems that his proof is straight from The Book.

> Imagine two convex polygons, such as a rectangle and a circle

Hrm....

You can think of a circle as an infinity-gon.

I like "infini-gon" :D

Terminology pedantry: a disk (not a circle, which is far from being even convex, being just the boundary of a disk) is indeed not a (convex) polytope, although it is a convex body.

I love stories like this one. Reminds me of the the twin prime story from a little while ago. Well written article as well!

As someone who has been funded by them, I think that I am nonetheless not being biased when I say that the Simons Foundation does a huge amount of good for the sciences, not least by funding this kind of consistently quality science journalism.

I'm not funded by them, and I generally think the Simons Foundation does great work.

Here's another data point in one of HN's favorite topics, how academia and publishing are broken.

> “It is like a kind of grace,” he said. “We can work for a long time on a problem and suddenly an angel — [which] stands here poetically for the mysteries of our neurons — brings a good idea.”

Had he learned LaTeX, I wonder if it would be a matter of justice.

Could anyone explain why it is impossible for a Gaussian distribution to be anti-correlated with a group of N other Gaussians? This proof clearly implies that fact, but none of the discussion seems to remark on it. I can imagine a distribution such that the "random seed" puts things closer to the tails based on the measured variance of a group of other normal distributions. Intuitively it seems like that ought to be able to be a normal distribution itself; I don't follow why that is not true.

I seriously can't believe that a problem that has evaded mathematicians for decades could turn out to have a 9 page elementary proof!

I don't know whether to be excited that, maybe, other hard problems in mathematics could have such elementary proofs, or depressed that mathematicians took so long to solve a problem with such a simple answer :)

Or it was ignored because it didn't have an author from Harvard or Princeton on it.

This conjecture seems to be easily seen as true by intuition. It's surprising that it took so long to find a proof.

Can you explain to me how this conjecture is intuitively true?

Let's say A has a 0.7 probability, and B has 0.6, and are independent. The compound probabilities are gotten by multiplying the individual ones:

Neither: 0.3 * 0.4 = .12 A, not B: 0.7 * 0.4 = 0.28 B, not A: 0.3 * 0.6 = 0.18 Both: 0.7 * 0.6 = 0.42

Now let's say that they're not independent; in fact, B absolutely requires A. You'd get something like:

Neither: 0.3 (this is reduced to the chance of not A) B, not A: 0 (B can't happen without A) Both: 0.6 (only other probability with B, has to make up the 0.6) A, not B: 0.1 (simply what's left to sum to 1.0)

Concrete examples of independent events: (fair) coin tosses -- the chance to come up heads is always the same, no matter the prior result. Related events: being dealt a face card in blackjack, and winning the hand -- whatever your normal chance of winning a hand, the odds of winning that particular hand just went up.

Calling this a "famous long-standing mathematical conjecture" seems a bit of a stretch. It gives the impression that this would be comparable to Fermat's Last Theorem, the ABC conjecture, the Riemann hypothesis, etc. However, this conjecture doesn't seem to have a Wikipedia page. The discoverer, Thomas Royen, does not have a Wikipedia page either. The Loren Pitt paper establishing a special case of the conjecture was cited only 50 times.

The only justification for the importance of the problem that I see in the original article is that it was open since 1972 and someone is quoted as having worked 30 years on it and knowing other people who have worked long on it. That's something, of course, but it's not so much -- there are lots of problems in mathematics that remain unsolved after some decades despite serious efforts, and not all of them are famous ; it depends on how much attention they have attracted.

Citing the Wikipedia articles' existence is a curious argument to make when an article titled "40% of Wikipedia is under threat from deletionists" [1] was just on the front page.

[1]: http://boingboing.net/2017/02/16/40-of-wikipedia-is-under-th...

Obviously obscure content will not survive unless it somehow gains fans willing to guard the page from these people, who pretend wikipedia is paper encyclopedia with limited budget for "notable" topics. They have to defend citations, perceived bias/neutrality and fight merge/deletion votes, gain allies with admins once it reaches arbitration and explain the article "notability" each time its questioned(and articles can lose "notability status" once one of these pedantic autists start questioning every line and word, deleting everything not following exact rules). In fact an isolated article, with new content or very few backlinks will not even be accepted and get speedily deleted before it gets worked into something acceptable to wikipedians. There has to be an alternative to this.

Despite deletionists' efforts, I doubt that any long-standing mathematical conjecture which can reasonably be called "famous" (to a general, non-specialist audience) does not currently have a Wikipedia page.

My point about the article is just that I found it annoying to be made to believe that this was about a very major result and find out later that the claims of fame had been exaggerated.

The paper has only seven pages, but the actual proof is even shorter --- it spans from p.3 to p.5. So, it's a three-page proof. Really well done.

Reply on news.ycombinator.com