Hacker News Reader

Comment by mlsu | Hacker News Reader

Far fewer than 600,000 of those are pretrained. Most are finetuned which is much easier. You can finetune a 7B model on gamer cards.

There is basically the big guys that everyone's heard of (google, meta, microsoft/openAI, and anthropic) and then a handful of smaller players who are training foundation models mostly so that they can prove to VCs that they are capable of doing so -- to acquire more funding/access to compute so that they may eventually dethrone openAI and take a piece of the multi-billion dollar "enterprise AI" market for themselves.

Below that, there is a frothing ocean of mostly 7B finetunes created mostly by individuals who want to jailbreak base models for... reasons, plus the occasional research group.

The most oddball one I have seen is the databricks LLM which seems to have been an exercise of pure marketing. Those I suspect will disappear when the bubble deflates a bit.

View on HN

You've nerdsniped me so hard that I had to make an account.

There are DOZENS of orgs releasing foundational models, not "a handful."

Salesforce, EleuthierAI, NVIDIA, Amazon, Stanford, RedPajama, Cohere, Mistral, MosaicML, Yandex, Huawei StabilityLM, ...

https://docs.google.com/spreadsheets/d/1kT4or6b0Fedd-W_jMwYp...

It's completely bonkers and a huge waste of resources. Most of them will see barely any use at all.

Competition isn't a waste of resources, it's the best mechanism we have to ensure quality.

Furthermore, I'm happy to be in a golden age with lots of orgs trying things and many options. It's going to suck once the market eventually consolidates us and we have to take whatever enshittified thing the ologopolists feed us.

Gods if this isn't exactly how it'll turn out.

It's a wast if they are mostly all trying the SAME things. Which is mostly what is happening.

I want someone to spend a million on a Chess LLM so we can get a sense of how sophisticated they can get at non-linguistic pattern matching.

I want someone to spend a million on an LLM trained on Python program traces so we can try to teach it cause and effect and "debugging". Maybe it will emulate a Python interpreter and get highly reliable at predicting the outcome of Python code.

etc.

Interesting! That is more than I thought. Honored to have caused a nerdsnipe.

In the grand scheme of things, though, most of these are quite small -- 7b range. A 7b model is nothing to sneeze at but it's not megacorp resources either. It's in the range of "VC check" size.

The "big boys" who are training 70b plus are FAANG or government-scale entities. Microsoft, Google, and Meta have multiple entries on that "big" LLM foundation list -- it's because the GPUs are already bought, have to train something to keep utilization up. Also bear in mind that training of these things is still something closer to an art than a science; you put terabytes of data into the cauldron, let it brew, and only after it's done can you taste what you've made. Makes sense that some of these models will be junk.

It’s like cryptocurrency hashing but, now, all the players are large extremely rich corporations. It is gonna be the funniest historical rhyme ever.

Very nice! This list is super convenient for LLM “connoisseurs”(?) like me.

Did you have a script to generate it or was it manually done?

Just spotted this link. Just to clarify, I (not the original poster, although everyone's welcome to share this link, it's a public doc) maintain this list (and the rest of the sheet) manually. While I keep the foundation models that I'm interested in fairly up to date, obviously there are too many fine-tunes/datasets to track now. I started this when LLaMA was first released and I was getting myself up to speed on the LLM landscape.

A group at the CRFM maintains a bigger list of models (their goal is stated for cataloguing foundation models, but it looks like they have some tunes mixed in these days): https://crfm.stanford.edu/ecosystem-graphs/

This site also seems to keep track of models, with more closed/announced models that I don't bother to track: https://lifearchitect.ai/models-table/

Very useful info. Thank you!

Thanks for this spreadsheet! It's amazing!

Yep, seems like every company is taking a longshot on a AI project. Even companies like Databricks (MosaicML) and Vercel (v0 and ai.sdk) are seeing if they can take a piece of this every growing pie.

Snowflake and the like are training and releasing new models because they intend to integrate the AI into their existing product down the line. Why not use and fine-tune an existing model? Their in-grown model maybe better suited for their product. This can also fail like Bloomberg's financial model being inferior to GPT-4, but these companies have to try.

> Why not use and fine-tune an existing model?

Not all of them have permissive licenses for whatever the companies may want (or their clients want). Kind of a funny situation where everyone would benefit, but no one wants to burn their money for the greater good.

Their biggest competitor release a model. They must follow suit.

It isn't like they could have started this after the release and been done by now.

> an exercise of pure marketing

Yes. Great choice of words. A lot of non-frontier models look like "an exercise of pure marketing" to me.

Still, I fail to see the rationale for telling the world, "Look at us! We can do it too!"

Mid-level managers at a lot of companies still have no clue what LLMs are or how they work. These companies (like databricks) want to have their salespeople upsell such companies on "business AI." They have the base model in their back pocket just in case one of the customers in the room has heard the name Andrej Karpathy before and starts asking questions about how good their AI solution is... they can point to their model and its benchmarks to say "we know what we are doing with this AI stuff." It's just standard marketing stuff which works right now because of how difficult it is to actually objectively benchmark LLMs.

> oddball one I have seen is the databricks LLM

Interesting you'd say that in a discussion on Snowflake's LLM, no less. As someone who has a good opinion of Databricks, genuinely curious what made you arrive at such a damning conclusion.

600k?

Reply on news.ycombinator.com