Why Meta’s latest large language model survived only three days online

technologyreview.com

19 points·p0pcult··7 comments

“A fundamental problem with Galactica is that it is not able to distinguish truth from falsehood, a basic requirement for a language model designed to generate scientific text. People found that it made up fake papers (sometimes attributing them to real authors), and generated wiki articles about the history of bears in space as readily as ones about protein complexes and the speed of light.”

How would anyone think releasing this was a good idea?

People have been expressing surprise about this issue with computing machinery for 200 years.

1. There is a large corpus of scientific publications.

2. Thanks to scihub et al, it is possible to download them en masse.

3. It is now feasible to train good language models on a large text corpus.

4. Wouldn't it be interesting to see what a model trained on this data comes up with? Maybe even new connections in existing knowledge?

> How would anyone think releasing this was a good idea?

Why would it not be? A model built on any consistent large data set could be interesting I’d expect.

But just because the data set has some semantic property, that doesn’t mean that a model built from it will necessarily generate text that had the same property (in this case, “being scientifically correct”). The real question is why would anyone think that?

Good points on both sides. I think the answer to your second question is, "first mover advantage, even if the product is bad"

Humans do the dame thing.

Stories like these are the reason why I immediately download and backup software if I even get a tiny inkling that it may be „too free“.

Reply on news.ycombinator.com