Hacker News Reader

Comment by GreedClarifies | Hacker News Reader

This is amazing work and bravo on to the people working on redpajama.

This is fantastic for the world, this means LLMs will not be controlled by a couple of companies with the associated rents.

Yes, private LLMs will likely be a couple of years ahead of 'free' alternatives, but that's OK, we want to incentivize for profit research so long as the services are low priced in time (and in this case in short order).

AMAZING WORK.

View on HN

My first reaction was, "why is replit building LLMs," but I guess it fits their needs to have one optimized for their use. But I wonder, is this the beginning of another wave of "every company is an AI company?" Are we going to see a spike in tech hiring around AI/LLM, money starting to flow again, etc? And how many years until it all blows up and the layoffs start?

Finetuning models and LLMs (and any model) is going to a be common practice . Each company is its own domain, which domain knowledge and data to specialize open sourced models or used other models to distill/teach their own proprietary model (home grown or modify someone else's).

Have you even tried it? It’s pretty bad

But that's fine it can be a year or two behind the state of the art. That's not the point.

The point is that there will be alternatives and that will reduce the price in time further increasing the impact of the technology.

There was a possible future where only MSFT and maybe GOOG and maybe one or two other companies had this technology and extracted massive rents.

to be clear this work is not based on redpajama - though we did discuss that in the previous episode https://twitter.com/swyx/status/1648080532734087168?s=46&t=9...

Oh my bad!

I thought I read that, is it based upon:

https://arxiv.org/abs/2211.15533 (The Stack) ?

partially. Reza discussed their data pipeline in the blogpost that we reference in the show notes

Reply on news.ycombinator.com