Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling

arxiv.org

68 points·iyaja··7 comments

A high energy physics simulation library

I asked ChatGPT what a top quark can decay into. Among other things it suggested a top quark could decay into a photon. ChatGPT seems not to have learned about baryon number conservation...

On a more serious note, Pythia has been around for 40 years... surely they could have picked a better name...

I asked ChatGPT (GPT-4) model about the top quark decay channels and it was mostly correct about top decay to W and b quark. It then started listing possibilities of further W decays which were mostly correct.

On your same serious note, particle physicists are the worst in naming things, let's not forget that people came up with a very unique name of "ROOT".

I'm ok with this as long as it's not the n-th software project called Spark.

I've been happily using Pythia in Acronymy Assistant: https://github.com/dwrensha/acronymy-assistant

It's quite convenient to have a continuous range of model sizes. Usually I want "the largest model that fits on my GPU", but sometimes I want to trade between quality and performance, and Pythia makes that easy.

Is this the link?

https://engineering.fb.com/2019/05/21/ai-research/pythia/

or maybe this? https://github.com/EleutherAI/pythia

I'm confused.

No, the title is linked. The title refers to Pythia by EleutherAI for evaluating LLMs.

You linked something that is a framework for multimodal models by facebook.

Reply on news.ycombinator.com