Training of 1T Parameter Scientific AI Begins | Hacker News Reader

Training of 1T Parameter Scientific AI Begins

hpcwire.com

59 pointsrbanffy26 comments

I’m somehow really skeptical—has there been a postmortem of the general failure of BLOOM? That model was unreasonably ineffective. I expect the same of government and academic models, but I don’t have a good rationale.

We've gone full circle https://thatsmathematics.com/mathgen/

Argonne National Laboratory (ANL) creates a 1T parameter model reading papers

Quite interesting who is funding such a huge endeavor?

I’m gonna guess who typically funds national labs, the DoE

You mean the American taxpayer?

Who funds the American taxpayer? Follow the money.

I wonder how many of the papers it will be trained on have been replicated.

This is exactly the kind of question an LLM like this might help with.

Someone has to ask it how to prevent the heat death of the universe.

THERE IS AS YET INSUFFICIENT DATA FOR A MEANINGFUL ANSWER.

more likely: what a great question! sure, it is possible to prevent the heat death of the universe. here are the steps:

1. ...

LET THERE BE LIGHT!

We are getting a bit ahead of ourselves here.

For the uninitiated: https://users.ece.cmu.edu/~gamvrosi/thelastq.html

It's my favorite short story.

Here's a TL;DR for people short on time:

"The Last Question" by Isaac Asimov is a seminal science fiction short story that explores humanity's evolution and the quest to overcome entropy over trillions of years. It starts in 2061 with two technicians, Alexander Adell and Bertram Lupov, operating Multivac, a powerful computer. They discuss using solar energy and the finite nature of the sun, leading to the titular question: Can the increase in entropy in the universe be reversed?

The story jumps through various epochs, where different iterations of computers (Microvac, Galactic AC, Universal AC, and finally Cosmic AC) are repeatedly confronted with this question but always respond with "Insufficient data for a meaningful answer." Humanity evolves, colonizes the universe, achieves immortality, and ultimately merges with the Cosmic AC.

At the universe's end, when only the Cosmic AC exists and time, space, and matter cease to exist, the AC finally finds an answer to the question. The story concludes with the words: "LET THERE BE LIGHT! And there was light --" suggesting the cycle starts anew, possibly with a new universe. Asimov's tale is famed for its profound philosophical inquiry and visionary depiction of humanity's future and technology.

This one, and Clarke's The Star.

A possible contender for the top spot: The Eyes Have It by Philip K. Dick

https://www.gutenberg.org/files/31516/31516-h/31516-h.htm

Gpt:“This is just your speculation”

I would like a GPT capable of making significant advances in string theory. It is extremely hypothetical by nature, so I think it should be a good fit for a very strong scientific GPT to check many different hypotheses (e.g. supersymmetric theories), assess their likelihood, and maybe even come up with new ideas that have value for the scientific community.

"With a budget of €7.5 billion, the LHC is one of the most expensive scientific instruments[1] ever built" - Wikipedia

Perhaps we should invest the next 7.5 billion in an exceptionally large GPT model.

I'm pretty sure that's the point of this 1 trillion parameter model.

Why would you want to do that?

Just let the Big Bang happen again to start a fresh game.

Tens of billions of years is enough for one round.

>> "Why would you want to do that?"

Because doing the thing they do in sci-fi when science starts working on the real thing is a time-honored tradition.

> It combines all the text, codes, specific scientific results, papers, into the model that science can use to speed up research

why is this article so bad? sounds fake

This is a related official press release (I can not find the actual source for this article): https://www.anl.gov/article/new-international-consortium-for...

> The Trillion Parameter Consortium (TPC) brings together teams of researchers engaged in creating large-scale generative AI models to address key challenges in advancing AI for science. These challenges include developing scalable model architectures and training strategies, organizing, and curating scientific data for training models; optimizing AI libraries for current and future exascale computing platforms; and developing deep evaluation platforms to assess progress on scientific task learning and reliability and trust.

I don't think it's fake, but it's just a "language" model.

This could be tremendously helpful based on the combined text & data it is trained on, and who knows, it might contribute to a 10X or more performance for more than just a small percentage of a scientist's needs.

I would think it depends on the type of scientist quite a bit.

Reply on news.ycombinator.com