Hacker News Reader

Comment by dealforager | Hacker News Reader

Man, this is an example of how difficult it is to know what is BS or not if you're not an expert on the subject. On one hand, this article was published in Nature, which I thought was trustworthy. On the other, there's this comment on a social media platform that links to a blog that also seems legit. No wonder misinformation spreads so fast. Even after reading both, I don't know what to make of it. The reaction and comments here just confuse me more.

View on HN

Nature has published some very questionable papers in AI/ML that are filled with malpractices. Another bogus paper that comes to mind was predicting earthquakes with a deep(read huge) neural network that appears to have information leakage and was fuelled with the hype of DL when a simple logistic regression (i.e. single neuron) could perform just as well [1,2,3].

[1] https://www.reddit.com/r/MachineLearning/comments/c4ylga/d_m...

[2] https://www.reddit.com/r/MachineLearning/comments/c8zf14/d_w...

[3] https://www.nature.com/articles/s41586-019-1582-8 / https://arxiv.org/pdf/1904.01983.pdf

This is a frighteningly common practice in DL research. Baselines are rarely taken with resect to alternate techniques, largely due to publication bias.

On one hand papers about DL applications are of interest to the DL community, and useful to see if there is promise in the technique. On the other hand, they may not be particularly useful to industry, or to forwarding broader research goals.

A good rule of thumb is to be slightly more suspicious of "DL for X" unless X was part of the AI/ML umbrella in the 2000s. If no one was publishing about X in AAAI/NIPS/ICML before 2013 or so then there's a pretty good chance that "DL for X" is ignoring 30+ years of work on X. This is becoming less true if one of the paper's senior author comes from the field where "X" is traditionally studied.

Another good rule of thumb is that physicists writing DL papers about "DL for X" where X is not physics are especially terrible about arrogantly ignoring 30+ years of deeply related research. I don't quite understand why, but there's an epidemic of physicists dabbling in CS/AI and hyping it way the hell up.

Anecdotally, having come from a physics background myself - DL is more similar to the math that physicists are used to than traditional ML techniques or even standard comp-sci approaches are. In combination with the universal approximation proofs of DL, it's easy to get carried away and think that DL should be the supervised ML technique.

Curiously, having also spent heavy time on traditional data-structures and algorithms gave me an appreciation for how stupendously inefficient a neural net is and part of me cringes whenever I see a one-hot encoding starting point...

Re: similar to the math they know, this makes sense.

I don't understand why over-hyping and over-selling is so common with AI/ML/DL work (to be fair, over-hyping is more related to AI than physicists in particular. But people from non-CS fields get themselves into extra trouble perhaps because they don't realize there are old-ish subfields dedicated to very similar problems to the ones they're working on.)

I think the answer is related to the startup culture, and that is to gather funding.

That's the confusing thing. Appealing to rich twitter users/journalists/the public doesn't strike me as a particularly good strategy for raising research funds!

Random rich people rarely fund individual researchers. More common for them to fund an institute (perhaps even by starting a new one). The institute then awards grants based on recommendations from a panel of experts. This was true before Epstein scandals, and now I cannot imagine a decent university signing off on random one-off funding streams from rich people.

All gov funding goes through panels of experts.

Listening to random rich people or journalists or the public just isn't how those panels of experts work. Over-hyping work by eg tweeting at rich/famous people or getting a bunch of news articles published is in fact a good way to turn off exactly the people who decide which scientists get money.

Maybe a particularly clueless/hapless PR person at the relevant university (or at Nature) is creating a mess for the authors?

>Random rich people rarely fund individual researchers.

Yes and no. There are private foundations that, if someone donates a reasonably large amount, say at least the amount of their typical grant, they will match the donor with a particular researcher, and the researcher will have lunch, give a tour, and send them a letter later about about the conclusions (more research is needed).

That doesn't mean the donor gets input into which proposals are accepted; that is indeed done by a panel of experts as far as I know. It's more of a thing to keep them engaged and relating to where the money goes when there are emotional reasons for supporting e.g. medical research.

Even less than that now. The ability of specific donors to direct funds to specific academic groups for specific research is WAY more constrained now than it was pre-Epstein. Institutions want that that extra layer of indirection.

While I agree with you, the nature paper that I linked above was published by the folks at google of all places. I think a valid hypothesis is that work done during internships (or even residencies) may not be on-par with what NeurIPS/ICLR/etc require but they give publicity and thus the PR teams push for that kind of papers.

However, it still does note explain why this kind of sloppy work done and published by publicly funded research labs, except perhaps as a form of advertisement.

Well, yeah, corporate research is what it is. A lot of the value add is marketing.

I agree with the sibling comment that whenever ML has not been used before on in a field and DL and especially DRL (deep reinforcement learning) are used, it is likely that the authors are ignoring decades of good research in ML.

After a very theoretical grad course in ML, I have come to appreciate other tools that come with many theoretical guarantees and even built-in regularization that are less Grad Student Descent and more understanding the field.

I think that the hype that was used to gather funding in DL is getting projected onto other fields, if only to gather more funding.

The articles don't contradict each other when it comes to cited facts - you can believe both!

I suppose its all in the implications though, which are contradicting as the nature article implies it is a big deal. The nature article doesn't give any examples of interesting conjectures, or examples of interesting consequences if any of the conjectures should be true. They talk a lot about alternate formulae to calculate things we already know how to calculate. Why would we care? Do they have a smaller big-oh? Nature references the theory of links between other areas of math, if true that's great, but if its true surely they would have mentioned an example of such a link? Anyways I lean towards this not being that interesting, even if you base that just on what the nature article said.

Re why would we care: this is a search algorithm for numerical coincidences. Most numerical coincidences are trivial, for example can be derived from hypergeometric function relation which was known to Gauss. In fact it would be interesting to automatically filter formulae which can be derived from hypergeometric function relation... On the other hand, numerical coincidences can lead to deep theory, monstrous moonshine is a prime example. Hope is that by searching for numerical coincidences, we can discover one leading to deep theory without already knowing that deep theory. This seems reasonable.

That's a very good point - and it really motivates these kinds of computer searches.

The Nature paper has quite a lot of detail in its supplementary

https://static-content.springer.com/esm/art%3A10.1038%2Fs415...

Table 3 inside also shows new conjectures for constants such as Catalan's and zeta(3). These results do not seem to trivially arise from known knowledge.

FWIW that blog is written by one of the top leading number theorists in the US today. Of course, his opinions are his and you’re free to form your own, but just wanted to clarify that the blog is very much legit.

It seems like Calegari is chasing after PR and may be angry at computer scientists getting into his field.

His criticism was discussed and found incorrect by the peer review process:

https://static-content.springer.com/esm/art%3A10.1038%2Fs415...

This is an incredibly strange slate of reviewers. Only the first seems to really understand the mathematical context. It's odd to appeal to the peer review process when the "peers" are not suited to complete the review.

I assure you that Calegari knows more about number theory than any of those referees, and the reasons why the paper is bad are well-explained on his blog (cf. the two links above) and by referee #1. Speaking of "peer review," look at how all the excellent mathematicians commenting on that blog agree with him!

I agree. Without being an expert in the field, the 2nd and 3rd reviews "smell funny"; they clearly lack depth, are very enthusiastic, and don't seem to make a good and comprehensive point as to _why_ this paper is supposed to be as great as they claim it is. In a strong community, any editor should consider these reviews disappointing, and any author should at least have mixed feelings about them.

Calegari gets to cherry-pick comments he approves or rejects on his blog, so calling it "peer review" is taking the concept out of context =).

True! I tried several times to comment on his blog - but Calegari didn't confirm my comments

It's hypocritical to criticize but to avoid criticism ...

The authors tweeted at Elon Musk and Yuri Milner, so it's obvious who is chasing PR (and "dumb money")

Meanwhile, the blog author congratulated Mathematica for being for being good at solving continued fractions

I'd ask you where the criticism was "found to be incorrect", but I know that's absurd (aka, not even wrong), as peer review comments are not in the business of "finding criticism to be incorrect".

Assuming that it's the PI/senior author doing all of this shameless promotional work, I feel really REALLY bad for the grad student(s) on this paper... what a way to start your academic career :(

The paper is actually really nice work, but holy jesus someone on that author list is making a complete ass out of themselves.

Academia isn't startup world. The community is small, people have long memories, and I've rarely seen the strategy being deployed here work out. It does work sometimes, but more often it backfires. Especially for folks who aren't yet on a tenure track.

Science/Nature are prestigious, but the quality of their articles are often questionable. Part of the problem is the short format, which makes it difficult to include a lot of context and sanity-checking. Another issue is that they prioritize the “sexiness” of the research over pretty much everything else.

I'll never understand why Science/Nature carry any currency in CS and Math. TBH I consider them negative signals in these fields, and I encourage others to do the same when hiring -- the same way that a prestigious newspaper or magazine would treat someone with a bunch of (non-news) Buzzfeed bylines.

There are some exceptions. E.g., a Science/Nature paper summarizing several years worth of papers published in "real" venues. Truly novel work that's reported on for the first time in Nature/Science is almost universally garbage. At least in CS/Math.

In my experience, at least in pure math, publishing in Nature/Science doesn’t carry any weight. The most prestigious journals for a given subfield usually specialize in that subfield (with a name like Journal of Geometric Topology), with a few exceptions like Annals and JAMS. Even those are still focused heavily on pure math; I can’t think of any which are cross-disciplinary outside of math.

Nature in particular seems vulnerable to the academic equivalent of click-bait articles. I think the top journals within a specific field are more reliable.

This is why I have empathy for conspiracy believers. From their perspective, their understanding of the world is accurate.

This is also why I see the inevitable failure of social media platforms in regulating truth-vs-non-truth.

As a thorough non-expert, I don't take headlines in the style of The Register seriously, even if the article is in Nature.

Although, if it was really from The Register it probably would have said "boffins" rather than "humans".

Sometimes popular science is itself the misinformation. The authors stretch the findings to land in prestigious journals. The news stretches the findings further to sell clicks (c.f. Gell-Mann Amnesia effect). The people on the internet selectively quote articles and selectively ignore others. The algorithm tries to only show you content that you like.

The truth doesn't have a chance.

This phenomenon has a name: epistemic learned helplessness.

https://slatestarcodex.com/2019/06/03/repost-epistemic-learn...

Reply on news.ycombinator.com