Hacker News Reader

Comment by Azrael3000 | Hacker News Reader

Apparently a new paper is supposed to come out from the original authors about the production method [0]. So it really is unclear whether everybody generates the same material.

It's a shame that these papers were published before they were polished by the authors.

[0] https://forums.spacebattles.com/threads/claims-of-room-tempe...

View on HN

Seems the fastest way to clear this up would just be for the original team to send a sample to some other scientists for testing. That would solve any issues regarding other teams reproducing the material incorrectly, and at a first pass, what everyone cares about is whether its a superconductor at room temp, not how to make it.

> Once the findings are published in a peer-reviewed journal, which Kim says is in the works, he will support anyone who wants to create and test LK-99 for themselves. [1]

Sounds like that’s the plan once they’ve secured the Nature paper.

[1]: https://www.newscientist.com/article/2384782-room-temperatur...

Sounds like they have it the wrong way round. Considering the uncertainty surrounding all of this, and the initial problems with replication, why would Nature publish this before it has been independently replicated?

Not sure what they're worried about here. They already have priority, and if this replicates, every journal on Earth will be fighting over the privilege to publish it.

Because that’s the scientific method? It would be entirely infeasible for them to require someone else to test the sample.

I don’t think you’ve been in academia?

> Because that’s the scientific method?

The world's leading science journal publishing grand claims without verification, in the face of mounting evidence that the claims might be incorrect, is most certainly not "the scientific method".

> It would be entirely infeasible for them to require someone else to test the sample.

What nonsense. Why would that be infeasible?

Actually this is the scientific method. Anything else just makes publishing groundbraking results harder and depends on some people’s (faulty) judgements.

Publishing incorrect results should be hard, especially if they would be ground breaking if true.

Nature basically publishes the sensational claims papers. That's their model, anything "high impact" goes there. It's then on other scientists to publish work refuting it or proving it's not correct. It's an issue that the refutations tend to go in lower impact journals, but it's the way it's works.

Extraordinary claims require extraordinary evidence.

But seriously, why can't they ship their sample somewhere?

They are busy trying to characterise it properly and making sure they understand what they have created. At least that's what they ought to be doing if this isn't a big scam.

> why would Nature publish this before it has been independently replicated?

Because it's Nature?

We're not talking about Organic Syntheses [0], which has replicated every paper before publishing since 1921.

[0] https://www.orgsyn.org/

I guess don’t hate the player. They’ve got a floating rock. What would materially affect conclusions?

Pride. They don’t want to be remembered for some incomplete Arxiv preprint. They want to be remembered for the finished paper in a top level journal.

Most likely the peer review process will include replication.

Normally (at least in my field), that’s beyond the scope of peer review in the traditional sense. Peer review typically just involves review of just the material submitted to the journal (manuscript, data, maybe code depending on its size and complexity). Depending on the experiment, reproducing a result may require resources, equipment, or skills unavailable to the reviewer or that cannot be acquired in a reasonable amount of time needed for the review.

Peer review serves basically as a smell test where capable experts certify that the paper is novel, interesting, properly justifies its claims, and contains enough detail to reproduce the results. It is not uncommon for only a few labs to contain all of the expensive, specialized equipment to reproduce an experiment, and getting that experiment working (if it is reproducible) can take a significant time commitment from multiple individuals in a lab. Also keep in mind that reviewers are normally not paid for their services, journals typically want at least two independent reviewers (who they pick, sometimes with input from the author to narrow down who has the background to perform the review), and going from submission to publication is typically a many-month process without the high standard of reproduction.

Reproduction happens in the manner that is going on right now. If your publication makes waves, people will want to build on your result and either reproduce it as a first step or conduct a related experiment that adds evidence to support the hypothesis. This way is far more practical, but the devious thing is that, if you lie and manipulate your data, some poor grad student might waste two years of their PhD trying to get your thing to work.

Having journals or some independent body fund reproduction themselves runs into both capital challenges and the problem of attracting a large enough number of capable scientists to cover every discipline who would only want to work on reproduction full time. As nice as that would be, I’m not sure such a venture would work in the real world.

That's true but if a reputable journal publishes this before that verification is in and it ends up being mistaken or even a hoax or fraud then their reputation will be dinged so with the stakes this high given the material and the visibility there is a fair chance that they will indeed wait for independent replication before committing to publishing it. But I don't see it happening during the peer review process itself, it will happen simply because another lab tries the same and gets lucky (or not, in which case the whole thing will blow over).

The journals reputation will be dinged if it comes to light that they have not adequately performed their due diligence. Which, as has been explained above, does not include replication.

If you require replication for publication practically nothing will get published ever. Once published, others will attempt to replicate, and it is the authors' reputation on the line if nobody manages to replicate it following the paper and consulting with the authors.

They are not going to replicate this themselves, obviously. But extraordinary claims etc, and this one is about as extraordinary as they come. Papers that are non-controversial even if they might turn out to be wrong get published all the time. But papers where 95% of the scientists out there are going to be super skeptical will find it much harder to get published. Both the authors and the publishers have a reputation to protect. Keep in mind that the peer-reviewers will have to sign off on the publication as well.

How can you replicate it before the paper with the stuff to replicate is published?

It would be a variation on what's already out there right now. And the whole problem is that the recipe is vague, in other words even if they have a working sample they don't know exactly what goes into making it.

Which is why it in arxiv and not some journal.

Seems like somebody pulled the trigger on publishing too early.

Exactly, there is some talk about the team not being in sync on this.

Why would Nature have published the production method of a random material if they didn't have experimental evidence of novel behavior?

They do that every now and then. Peer review rarely, if ever, involves any sort of replication.

I’m asking on a legitimate basis. I am not involved in the science world and only know about Nature as a publication from high school and college. The way Nature was described to gave me the sense that you weren’t going to have a paper published unless you had some surprising results, and that didn’t include the null hypothesis.

If that’s not the case for this publication then I would appreciate if someone showed me how they changed in the past decade

They publish surprising and high-profile results, yes.

However, there's also an informal crazy claims filter. If your paper is so surprising they think it might be made up or wrong, they might ask for extensive revisions prior to publication.

So they won't publish my paper saying the moon is made of cheese no matter how surprising such a finding might be.

Easy to say, but this is hard. "I got a box of dust from you". Now was this the lab sending dust because they know it doesn't work, and want to hide their tracks; the receiving lab destroying the sample because they want their own discovery to win a Nobel prize; or just regular managed in shipping? You should be able to think of other reasons that an unethical lab on either end might want to ensure the samples are destroyed instead of tested.

The above can be managed, but it can't be done over a weekend. The lab has stated they are willing to send their working samples out to other labs. However to ensure it arrives safely (avoiding issues like the above) means that you shouldn't even be concerned until two months have passed, and accept reassurance for a few months after that.

This assumes the chemical structure is stable, which may not be the case. If the material is truly not superconductive at room temperature, it will take a long time to gather enough diverse experimental evidence to build a scientific consensus. A good rule of thumb is probably 3 months to reproduce, 6 months to discredit.

why is it slower to discredit than to reproduce?

If I can replicate a result, that's fairly straightforward: I replicated it!

If I can't, that doesn't mean the result was necessarily wrong; maybe I just messed up, or got unlucky and the result only happens 20% of the time. You might need several independent failed replications to start to be confident that the original result was definitely wrong.

It's like saying: there's a buried treasure in this acre. If I find it on my first pass through with a metal detector, job done. If I don't find it on my first pass, I'll probably need to make several more passes, maybe bring in fancier equipment and so on, before I could be pretty confident that it's not there.

Someone claims that a human can run a 5 minute mile.

To prove it true, you just need to show a human who can run that fast. To provide it fast, you have to accumulate incontrovertible evidence that it's simply beyond us...

For those firing up Google: The mile run record has been sitting at 3:43 for over two decades now.

>It's a shame that these papers were published before they were polished by the authors.

Saying nothing about these specific papers, in general I disagree strongly.

I find the peer review process to be extremely flawed. If you get a bunch of interest from experts in the field on the internet who comment and do work to understand, replicate and analyse (and without hiding behind fake-anonymity) that is a million times better in every dimension than anonymous "peer review." That is, it works better as a BS detector, it works better identifying issues in the data analysis, methodology and so on.

Anonymous peer review has been great compared to alternatives for hundreds of years, just now, our alternatives are frequently better than that.

For these papers in particular, it seems like peer review was badly needed before publication. Sure, it's a meme that Reviewer 2 is always going to complain that could you please run like, 50 more experiments, and for something as potentially groundbreaking as room-temp superconductor, even more so.

But one of the main issues with the papers right now is their chemistry is deficient. The key chemical reactions don't balance, and there's also an unexpected and unexplained oxidation state change in the copper. This is the kind of issue that would be flagged quickly in peer review, and it's the sort of catastrophic paper issue that makes replication problematic--how can you be sure that you're comparing against what the original sample was?

Sure and all of that is coming out in public with people's names and reputations attached to it. So we're certainly not behind on any of those issues. I say there are very real benefits too.

> It's a shame that these papers were published before they were polished by the authors.

While that's true, sometimes having some time pressure is useful to make sure a release even happens.

They've apparently been working on this for nearly a quarter of a century already... (!)

> It's a shame that these papers were published before they were polished by the authors.

We've seen no evidence that they were.

> It's a shame that these papers were published before they were polished by the authors.

Indeed. It's also a shame that they chose to use language like:

"We believe that our new development will be a brand-new historical event that opens a new era for humankind."

When you make grandiose boasts like these, you better be fucking certain you are correct.

I think a huge part of that is translation: imagine what you might sound like if you wrote a press release for or tried to explain to a layman some project you're involved in, and then had it translated to Korean.

Some benign description of why room temp atmospheric pressure superconductors are interesting becomes 'a brand-new historical event that opens a new era for humankind'.

> imagine what you might sound like if you wrote a press release for or tried to explain to a layman some project you're involved in, and then had it translated to Korean.

My understanding is that this was supposed to be a (draft of a) research paper, not a "press release". The article certainly doesn't read like laymen are the intended audience.

> Some benign description of why room temp atmospheric pressure superconductors are interesting becomes 'a brand-new historical event that opens a new era for humankind'.

Although I don't speak Korean, I strongly doubt that.

Wasn’t the paper leaked by the industry guy? Didn’t he edit it to include himself?

He might have jazzed up some other wording as well.

It’s worth giving the benefit of the doubt to the final product. Let’s get some replication (or not) and go from there.

When you're claiming you have a room temperature superconductor, it would be more suspicious if you weren't claiming that. Because if LK99 works, then that's exactly what it is. It'll be material of the century.

Look at some of the most ground-breaking research papers in the history of science, such as the Watson/Crick DNA structure papers. They literally discovered the core of life itself, which is infinitely more important than a room-temperature superconductor. Yet they are incredibly modest about it, and don't waste a single sentence waxing philosophical about what all this means etc.

In a similar vein, Andrew Wiles announced his proof of Fermat's Last Theorem in a lecture modestly called "Modular Forms, Elliptic Curves and Galois Representations" – not "The Historic Proof of Fermat's Last Theorem, with Endless Implications for the Future of Mathematics".

This isn't how science is typically done.

And the unsanctioned paper is titled: The first room-temperature ambient-pressure superconductor. And the response paper is titled: Superconductor Pb10−xCux(PO4)6O showing levitation at room temperature and atmospheric pressure and mechanism.

Notably not “We’re fucking changing history”. What are you griping about? That the paper admits that this would be a significant finding with wide-ranging applications? Because that’s true…

Nice to be modest after building off of Rosalind Franklin's unaccredited work.

But she is brought up constantly and annoyingly in this regard. Why be a negative Nancy? There’s books from not long after the discovery that tell her story. You don’t think there are people who are legitimately forgotten in the process of every discovery?

I'm perfectly willing to forgive some hubris if their claims turn out to be accurate. Watson and Crick are not a great choice of role model for good character.

I read The Double Helix, Watson's autobiography (which I enjoyed), what are you referring to that he failed to mention?

Rosalind Franklin's work paved the way for Watson and and Crick. She and her PhD student took the photo that showed the double helix. The scientist leading the team at the lab, Maurice Wilkins, showed the photo to Watson and Crick. He and Franklin did not work together, and she was hired while he was not there, under the impression she would be working alone. And so she did. Wilkins was not involved in the taking of the photo, yet he shared her private research with Watson and Crick.

This photo was the missing piece for them to understand the structure of DNA and Franklin was not credited for it at the time. Wilkins, meanwhile, received the Nobel prize with Watson and Crick despite the single biggest contribution that came from the lab being Franklin's photograph.

It's nearly impossible to overstate just how pivotal and important this photograph was. There was no theory or model at the time that would have gotten them to understanding DNA. This x-ray crystallography photograph was THE missing piece.

Watson, Crick, and Wilkins could have properly credited her at any time. They included her photograph in the paper, explained how important it was. She wasn't even mentioned. They effectively plagiarized her work and took the credit.

You read a whole book, but can't be bothered to do a google search titled "watson and crick problematic"

Weird

The commenter made a point to mention this, so I asked a follow-up question to hear their thoughts.

It's called conversation and it's why online forums like HN exist.

You'd think they'd find a new word

"superpressreleasing"

Reply on news.ycombinator.com