Meta unveils Llama 2 Long AI beats GPT-3.5 Turbo and Claude 2 on some tasks | Hacker News Reader

Meta unveils Llama 2 Long AI beats GPT-3.5 Turbo and Claude 2 on some tasks

venturebeat.com

39 pointsnickthegreek12 comments

The fact that Metas models have managed to be less accessible than fully closed, API-only offerings makes me really struggle to see the value here. If you’re willing to jump through corporate hoops, OpenAI can have you up and running in a few minutes, with few relevant restrictions for most businesses. If you’re willing to trade performance and ease of use for control, then there are plenty of open source models with decent performance you can fine-tune. Metas models are like the worst of both worlds - subject to restrictive licensing and approval with limited control (at least legally), but requiring a fair bit of elbow grease to get working compared to just hitting an API.

I thought llama is open weights and open to commercial with reasonable restrictions. I haven't heard practitioners who fine tune llama having trouble with it

? They aren't subject to restrictive licensing. It's about a 10 liner to run behind an API

There are two clauses in the llama license that I’m aware of that are restrictive:

1) if you have more than 700M daily active users you can’t use it.

2) you can’t use the output of this model to train other models.

Are you aware of others, cause I’m not, and those aren’t unreasonable.

The TOS also outlines a number of restricted use cases, which are broader than other models. For example, Llama prohibits any ITAR related use, while Anthropic limits specific usage related to defense and weapons. (There are many peaceful technologies that are covered by ITAR).

Actually - fair enough, I think I was overestimating how much people consider the ability to use the output to train your own models important. I guess for me these felt really restrictive since that always felt like the end game - collect a bunch of data, then train a smaller, cheaper to run model for a specific use case - but I’m clearly an outlier!

I think you are misunderstanding just a bit. You can certainly fine-tune/train LLAMA for a specific use case. Just grab LLAMA 7B and start training, that will fit on plenty of devices and you can even quantize the weights and make it run faster (and on even smaller devices).

What you cannot do is use LLAMA outputs to train your ChatGPT weights.

What on earth are you talking about. Metas models are the foundations for almost all the popular open source LLMs.

In what sense are they foundations? Are they using the literal code and models? Or just using the architecture and reimplementing and retraining the new models?

Anyone know if it will be open sourced?

Of course not.

At most, it'll be like the previous one. But not open source.

As usual, it will not be open source. But be sure to see Lecun et al. on tour promoting their "open source approach"....cant make this up!

Reply on news.ycombinator.com