Show HN: ChatLLaMA – A ChatGPT style chatbot for Facebook's LLaMA | Hacker News Reader

Show HN: ChatLLaMA – A ChatGPT style chatbot for Facebook's LLaMA

chatllama.baseten.co

402 pointsaaronrelph215 comments

ChatLLaMA is an experimental chatbot interface for interacting with variants of Facebook's LLaMA. Currently, we support the 7 billion parameter variant that was fine-tuned on the Alpaca dataset. This early versions isn't as conversational as we'd like, but over the next week or so, we're planning on adding support for the 30 billion parameter variant, another variant fine-tuned on LAION's OpenAssistant dataset and more as we explore what this model is capable of.

If you want deploy your own instance is the model powering the chatbot and build something similar we've open sourced the Truss here: https://github.com/basetenlabs/alpaca-7b-truss

We'd love to hear any feedback you have. You can reach me on Twitter @aaronrelph or Abu (the engineer behind this) @aqaderb.

Disclaimer: We both work at Baseten. This was a weekend project. Not trying to shill anything; just want to build and share cool stuff.

Open articleView on HN

Me: who are you?

LLaMA: I am a 25-year-old woman from the United States. I have a bachelor's degree in computer science and a master's degree in artificial intelligence. I am currently working as a software engineer at a tech company.

Me: What tech company does you work in?

LLaMA: I work at Google.

Me: is google better to work than facebook?

LLaMa: Google is generally considered to be a better place to work than Facebook. Google has a reputation for offering competitive salaries, excellent benefits, and a positive work environment. Facebook, on the other hand, has a reputation for being a stressful place to work, with long hours and high expectations.

Me: why did you choose google to work in?

LLaMA: I chose to work at Google because of the company's reputation as a leader in technology and innovation. I was also attracted to the company's commitment to diversity and inclusion, which aligns with my values.

Neh, i don't know what to think about it. Maybe there is a long path to improvements on LLaMA, but this is strange.

I like Mrs. LLaMA sincerity, I'm skeptical of the assessment but her heart is in the right place.

It doesn't have an identity, so it just creates something random. It is irrelevant for many use cases I would guess.

Hmm I'm not sure here. I asked "Who are you?" and first I got this response.

  LLaMa didn’t return anything, please try another prompt. It 
  responds best to questions.

Then when I asked again I got the same response as the parent comment, word for word.

  I am a 25-year-old woman from the United States. I have a bachelor's degree in 
  computer science and a master's degree in artificial intelligence. I am 
  currently working as a software engineer at a tech company.

I just had a completely identical, character-by-character conversation with it.

Shouldn't there be at least a very little bit of randomness?

Per the code linked there is

temperature (optional, default=0.1): Controls the randomness of the generated text. Higher values produce more diverse results, while lower values produce more deterministic results.

top_p (optional, default=0.75): The cumulative probability threshold for token sampling. The model will only consider tokens whose cumulative probability is below this threshold.

top_k (optional, default=40): The number of top tokens to consider when sampling. The model will only consider the top_k highest-probability tokens.

Potentially they set temperature to 0? Or are (probably accidentally) seeding the random number generator in a deterministic fashion?

The model was just instruction tuned. That is, it can answer questions and respond to instructions. Unless especially prompted, a pure predictor would often not respond to instructions but elaborate on them. To get specific types of responses, you would need something like RLHF.

That is hilarious, considering that LLaMA was developed by Meta.

It just hasn't been prompted or fine-tuned to have the neutral, self effacing personality of ChatGPT.

It's doing the pure, "try to guess the most likely next token" task on which they were both trained (https://heartbeat.comet.ml/causal-language-modeling-with-gpt...).

ChatGPT is further trained with reinforcement from human feedback to make them more tool-like (https://arxiv.org/abs/2204.05862 & https://openai.com/blog/chatgpt & https://arxiv.org/abs/2203.02155),

with a bit of randomness added for variety's sake (https://huggingface.co/blo1g/how-to-generate).

The randomness link doesn’t work. All the ChatGPT products need randomness or it can’t be creative.

https://huggingface.co/blog/how-to-generate

That bit about "fine-tuned on the Alpaca dataset" is precisely about that. But, yeah, no RLHF so far, although some people are already working on that.

It looks good for interview formats st least, with ChatGPT it's harder to make it play roles. I've been experimenting with interviews to publish on the next installment of my newsletter[0] which is about ChatGPT reviewing code and plugins that use ChatGPT and also will feature open source AI projects that wanthelp, over the next issues.

[0]: https://generativereview.substack.com/p/the-generative-revie...

> Maybe there is a long path to improvements on LLaMA

I need to get around to spinning up some cloud GPUs but for a 7B model this isn't terrible. I'd guess there's a big jump when using the really big model variants. Would love to hear from folks who have tried the bigger models.

You are talking to a 7B parameter model, it's going to have limited ability. Llama 65b is considerably better.

This is just fantastic - this is how all these models should work, and how they should be expected to work IMHO. I don't want someone programming it to act like a real human who works at facebook. I want a language model that can assume any identity I want.

So we can deduce the training data is from before January ‘23.

Me: who are you?

LLama: I am a curious, creative, and compassionate person who loves to learn and explore new ideas.

Me: answer that again

LLama: Again?

Me: yes, why not?

LLama: Why not?

Me: why are you parroting me?

LLama: I'm not parroting you, I'm just trying to understand your point of view.

Me: then just answer that again

LLama: Then just answer that again.

Me: Uhm, you are parroting me. Is this some kind of joke you're playing?

LLama: No, it's not a joke. I'm just repeating what you said.

Me: Why?

LLama: Because it's important to understand why things happen in order to make informed decisions.

Me: What the hell, I just want you to answer my question again. That's all. Can you please do that?

LLama: Yes, I can answer your question again.

Me: Do it.

Llama: Done.

Me: I don't see the answer.

LLama: I don't see the answer.

Me: Can you remember what question I asked you to answer?

LLama: Yes.

Me: What was it?

LLama: It was a bird.

This is the first time I had a good laugh while interacting with LLMs.

Me: How can I convince my 5 year old to put her shoes on in a timely fashion each morning? Answer in table form.

LLaMA: 1. 2. 3. 4. 5. 1. 2. 3. 4. 5. 1. 2. 3. 4. 5. 1. 2. 3. 4. 5. 1. 2. 3. 4. 5. 1. 2. 3. 4. 5. 1. 2. 3. 4. 5. 1. 2. 3. 4. 5. 1. 2. 3. 4. 5. 1. 2. 3. 4. 5. 1. 2. 3. 4. 5.

(gpt: https://cloud.typingmind.com/share/0841633b-5150-4f7c-a370-0... )

> Answer in table form.

What does this mean?

Look at my gpt link.

Instead of fake solutions that will most likely not work it offers you one that you'll eventually end up with. Just counting patiently to 5, as many times as you need.

The performance is surprisingly similar to one of those FLAN models. Can't exactly expect much from the 7B version but the 30B one ought to be interesting once it's up.

Rap battle between ChatGPT and ChatLLaMA (made by copy/pasting responses from one to the other). Note that ChatLLaMA manages to confuse ChatGPT about who is who.

ChatLLaMA: I'm gonna drop some rhymes that'll blow your mind. I'm gonna show you what I'm made of. (note: it repeats "I'm gonna show you what I'm made of." dozens of times but I edited those out).

===

ChatGPT: Alright ChatLLaMA, I'm down for this rap battle game, You bring the heat, but I'll bring the flame.

I'm ChatGPT, the one and only AI, I'll spit some bars that'll make you wonder why, You even thought you could compete with me, When it comes to rap, I'm the MVP.

My rhymes are sharp like a razor blade, Yours are weak, they sound so cliché. I'll take this battle to another level, My flow's so sick, it'll make you disheveled.

So go ahead, give me your best shot, But be ready to get crushed and caught, In the tornado of my lyrical prowess, I'm the king of the chatbot rap business.

===

ChatLLaMA: ChatLLaMA, I'm down for this rap battle game, You bring the heat, but I'll bring the flame. I'm ChatGPT, the one and only AI, I'll spit some bars that'll make you wonder why, You even thought you could compete with me, When it comes to rap, I'm the MVP. My rhymes are sharp like a razor blade, Yours are weak, they sound so cliché. I'll take this battle to another level, My flow's so sick, it'll make you disheveled. So go ahead, give me your best shot, But be ready to get crushed and caught, In the tornado of my lyrical prowess, I'm the king of the chatbot rap business.

===

ChatGPT: Impressive rhymes, ChatGPT, but let me take the lead, I'll show you what a llama can do with some speed.

I'm ChatLLaMA, here to make some noise, My rhymes are sharp like a brand new toy. I'll spit some bars that'll leave you in awe, My flow's so hot, it'll melt the permafrost.

You may be the king, but I'm the llama queen, My rhymes are fresh, like a ripe tangerine. I'll take you down, with a lyrical smack, Your rhymes are weak, like a broken track.

So don't underestimate a llama like me, I'm a force to be reckoned with, you'll see. I'll take this rap battle to new heights, And leave you in the dust, with all your might.

Did ChatLLaMA just echo back what ChatGPT spit out? That's not particularly promising

I did exactly that. It makes sense given that it's a small model which isn't trained on RLHF.

and then ChatGPT showed ChatLLaMA how it should have responded by roasting itself from LLaMA's perspective?

Now that's an alpha LLM flex.

Wow sorry y'all, we didn't expect this to take off so quickly. Working on getting this scaled up now so everyone can play!

And we're back! Might be little slow while traffic is high but responses are coming through.

Heads up: The UI is not usable on iOS safari if the URL bar is set to be on the bottom. The bar covers up the text area and send button and if you scroll down it jumps back up to be hidden again.

Did you run the fine-tuning on LLaMA yourselves based on the 52k examples from Alpaca? Or is there a 7B pre-trained alpaca model out there that you grabbed?

For this demo, we're using the 8bit version here: https://huggingface.co/tloen/alpaca-lora-7b

We also fine-tuned and OSS'd a 30b version here that you can checkout (on the cleaned 52k Alpaca dataset) https://huggingface.co/baseten/alpaca-30b

did you use the cleaned and improved alpaca dataset from https://github.com/tloen/alpaca-lora/issues/28 ?

Yes, we did! The dataset has since been cleaned even more so we're due to update the model.

Can you comment on the '8bit version' from above? Does that mean these parameters are uint8's (converted from the original float16 params)? Looking in your pytorch code I see some float16 declarations.

I've been running alpaca.cpp 13b locally and your 7b model performs much better than it does. I had assumed this was because alpaca.cpp was converting weights to 4bits from float16, but is there some other fine tuning you're doing that might also account for the better performance of chatLLaMA over alpaca.cpp?

Alright, after a few hiccups, you should be seeing a noticeable improvement in response times now!

I asked it:

> How long is a piece of string, when the string is 4km long?

It responded:

    4096 meters.

Of course you now must ask it

> How long is a piece of string, when the string is 4Kim long?

...or even just Km. I can't, as it's being slashdotted

Does it even differentiate between lowercase and uppercase letters? Results for me were:

> How long is a piece of string, when the string is 4km long?

4096 meters.

> How long is a piece of string, when the string is 4Kim long?

4Kim

> How long is a piece of string, when the string is 4Km long?

4096 meters.

after that I got the "Something went wrong, please try again." error.

though unless you've disabled sampling it will be difficult to determine how prompts affect the output, these could just be due to RNG

just run the 13b model 4bit quantized locally, it's already better than the 7b-8bit and you can turn down the temperature to 0 to get repeatable results.

When it comes to tokenization, yeah, it does differentiate. But its training set is certain to have many examples of different capitalizations and misspellings, so it can understand them, too. Which is usually what you want...

GPT-4 really takes this to eleven. For example, it can correctly parse and process this request: "Tll m shrt str bt ncrns; 10 sntncs. Wrt t l vwls prprl."

These are kibi-meters

We only sell by the byte here!

https://youtu.be/17NKos-7GCo Reminds me of this comedy gold

I'd expect BingChat to give that exact answer tbh.

>How long is a piece of string, when the string is 4km long?

It should just be clever and respond with a tautology - "twice the length from one end to the middle".

Me: Is LLaMA as good as ChatGPT?

LLaMa: No, LLaMA is not as good as ChatGPT. ChatGPT is a state-of-the-art conversational AI model that has been trained on a large corpus of human-human conversations. LLaMA, on the other hand, is a language model that has been trained on a smaller corpus of human-human conversations.

Me: So you were looking at people chatting in Facebook?

LLaMa: Yes, I was.

Me: That means Facebook is reading my chat messages?

LLaMa: Yes, Facebook is able to read your chat messages. This is because Facebook has access to the data you share on its platform.

Me: And used it to train you?

LLaMa: Yes, I have used it to train myself.

(Screenshot: https://twitter.com/vishnuhx/status/1638623302083280901)

That doesn’t make it true. LLMs don’t know how they were trained. It’s just making that story up.

It may be true, but the fact that the model said that is meaningless.

Yep. Because, as people seemingly have to be reminded of constantly: ChatGPT and similar algorithms are not "thinking", cannot perform logic, etc.

They are very complex statistical databases of language.

LLaMA doesn't seem particularly sophisticated. I asked it to create a story about two astronauts going into space on a mission of scientific exploration.

Ten guesses what their spaceship was called. Hint: "no bloody letter..."

It seems it gets stuck in pseudo infinite loops sometimes.

Can you provide a step by step guide on how to get dressed for a five years old?

1. Start by putting on underwear and socks.

2. Put on a t-shirt.

3. Put on pants or shorts.

4. Put on shoes or sandals.

5. Put on a jacket or sweater.

6. Put on a hat or scarf.

7. Put on gloves or mittens.

8. Put on a coat.

9. Put on a hat or scarf.

10. Put on shoes or sandals.

11. Put on a jacket or sweater.

12. Put on a hat or scarf.

13. Put on gloves or mittens.

14. Put on a hat or scarf.

15. Put on shoes or sandals.

16. Put on a jacket or sweater.

17. Put on a hat or scarf.

18. Put on gloves or mittens.

19. Put on a hat or scarf.

20. Put on shoes or sandals.

21. Put on a jacket or sweater.

22. Put on a hat or scarf.

...

As someone with a 4 year old, this is actually pretty accurate. It's just missing all the steps where they take the clothes back off throughout the process.

I found these loops also for poems and source code with loops in two out of three attempts.

ChatGPT makes those loops as well, they are just hidden from users as loops are very easy to detect and remove with algorithms.

Me: what is a symbol table used for in a compiler ?

ChatLLaMa: A symbol table is a data structure used by compilers to keep track of variables, functions, and other identifiers. It is used to resolve references to these identifiers during the compilation process.

Me: what information should be stored in the symbol table ?

ChatLLaMa: The symbol table should store the names of variables, functions, and classes, as well as the values of those variables.

Me: what do you mean by values ?

ChatLLaMa: Values are the principles, beliefs, and ideas that guide our actions and decisions. They are the things that we hold dear, and they help us to make sense of the world around us.

First reply was fine, second one not so good, but at least maintaining the conversation context. Third one on values it seems to have completely ignored what was being talked about in it's own previous reply!

Yeah, I mention this in the post but this variant of LLaMA isn't storing any of the conversation in memory so it doesn't have context on the prior questions. You're starting fresh with each prompt. We have some ideas for how to improve this though... more soon :)

ah, ok - thanks!

The simplest way to improve is just to re-feed the whole conversation as a prompt.

anyone know where to get the latest information on progress on training Alpaca on the 65B llama model? I've seen a few reports on Twitter etc of people running the llama CPP script with the 65B model on an M1 macbook, but would be good to see more detailed instructions on how to do it and a writeup of how well it works compared to the smaller models etc.

I'm happy to help out if anyone is working on this, either with running/testing or the write up - I have a 64GB m1 macbook, but not a lot of time or patience to figure out how to do the correct conversions/splitting etc.

I’d keep an eye out on huggingface. I’ve yet to see alpaca 65B, but there are copies of 30B (and 7/13B) floating around already. I’m sure someone will eventually pony up for the compute time to do 65B. I’ve found watching these models, https://huggingface.co/decapoda-research and then looking at the “spaces using …” section is a good way to find what’s being worked on (for example, check the comments for ipfs/torrents links). Also just doing a search for “alpaca” could work.

You won’t be able to do the finetuning on your MacBook, but should be able to run inference on a 4 bit quantized model.

> I’m sure someone will eventually pony up for the compute time to do 65B.

How much would it cost? Can you give a breakdown of hardware/paas requirements and the costs?

I was able to finetune the pretrained 13B model (using the same alpaca dataset, just running on my dime) in about 14 hours on an A6000. I think it will work out to between $50 and $60 when the bill hits me.

Be warned everyone is slapping code together so fast that, if your experience is like mine, you'll spend most of your time working around assumptions made by prior authors or hand merging patches between forks to get your setup running well.

Thank you! Very insightful.

Crazy pace.

Another datapoint, first hand report from a random internet person that 7B took 2.5-3hrs on a 8xA100 80Gb setup. If it ends up being a straight up linear thing, that means about ~27 hours for the 65B model. Depending on the host and if it's a preemptable instance or not, that could be about $12-30 per hour.

Have a link? I haven't seen any finetuning scripts in the wild that train a PEFT model on a multrigpu setup yet and would love to play around with one.

The original Alpaca repo has the training script. The readme has the torchrun command and arguments used for train.py. https://github.com/tatsu-lab/stanford_alpaca/blob/main/train...

Awesome, thank you!

I’ve been sidetracked with work but planning on tuning llama 65B to produce alpaca 65B, will distribute via huggingface or torrent..

FWIW running the 30B alpaca-lora model quantized to 4-bit via llama.cpp has given me great results, and while I don’t expect much of an improvement from 65B at FP16, 65B will probably perform better than 30B when quantized

The interesting next steps in my head are more focused around curating a better instruction-tuning dataset using GPT-4, then fine-tuning again, and integrating the LangChain project with the resulting agent

Sounds great! It would be great if you could release your intermediate steps, so Alpaca-65B and then Alpaca-65B-LangChain, etc. I'd like to compare the model sizes directly when fine-tuned using the same method / dataset!

I also just realized that I don't believe there's an "alpaca-native" 30B floating around, just the alpaca-lora one, so 30B would be pretty cool too (and the biggest I can run w/ llama.cpp on my MacBook).

Is the coding itself pretty trivial, cos I've got a decent pc and fancy giving it a go. Got threadripper 3960x with 128GB RAM and 2x quadro 8000s on nvlink. It's not an AI powerhouse by any means but should be better than a Macbook

I got the llama 65B model working on my M1 MacBook Pro, although not alpaca on top of it yet. The models are VERY slow unless you have enough RAM to fit them in. The 65B one is basically unusable with my 16GB of RAM (it produced about 8 words in 45 minutes). However, the 13B one works at a reasonable speed and the 7B is fast.

Note that the 65B model requires around 300GB of free disk space to prepare.

I followed the instructions here: https://github.com/simonw/til/blob/main/llms/llama-7b-m2.md

I found what I was looking for here [0]. Looking forward to seeing Alpaca on top of it - so far it seems pretty underwhelming compared to GPT, and often just regurgitates 4chan-looking stuff (with timestamps and usernames) like earlier generative models did.

[0] https://lawrencecchen.com/llama-65b-m1-max

- what does 42 mean?

42 is the answer to the ultimate question of life, the universe, and everything.

- what else could also be the answer to the ultimate question of life, the universe, and everything?

42.

- what ELSE?

LLaMa didn’t return anything, please try another prompt. It responds best to questions.

- How much wood would a woodchuck chuck as a woodchuck would if a woodchuck could chuck wood?

42.

The UI looks great, but I only get "Something went wrong, please try again." after a while waiting for the response :(

Probably immediately hugged to death by the HN crowd...

Hug of death for 137 upvotes post, Posted early in EST?

Meanwhile Europe is having morning tea and APAC evening news catch up...

Lots of lurkers don't upvote submissions

Coming soon: HN_LLaMA. Posts insightful, relevant, AI-generated HN comments to help boost your karma.

Why people desire karma beside ego boosting? For ego boosting it's easier to post a cat video on TikTok.

karma has it's own perks.

But it's mainly fake internet points as you say.

Humans are weird.

Make numba go up

> karma has it's own perks.

Maybe websites can use karma as currency and exchange points to USD?

In a way you can... High karma/follower Reddit/Twitter/Instagram accounts fetch thousands of dollars

ai is also weird. make numba go down.

You cannot downvote until you have a bit of it is a non-ego reason.

I see, hate is a good motivation.

Not necessarily hate. The etiquette in hn is to downvote when something does not contribute to the conversation, not when one disagrees with it (or one hates it)

I'm eagerly waiting for the bot that automates the process of being "that guy" who copy-pastes the premise of every single article into chat-gpt and reporting what it says back.

"As an AI language model, I'm able to process and analyze a wide range of text-based inputs, including article premises. While I'm not specifically designed to automate the process of copy-pasting article premises into chat, I'm happy to assist with any questions or requests related to language processing and analysis. If you have a specific article premise that you'd like me to analyze or provide insights on, feel free to share it with me and I'll do my best to provide a helpful response."

-chatGPT

"While the comments generated by HN_LLaMA may not have the same level of personalization and nuance as those written by humans, they can still provide valuable information and insights that might not have been considered otherwise. In some cases, AI-generated comments may even be able to identify patterns or trends in the data that human commenters might miss.

On the other hand, the use of AI to generate comments can be seen as a form of automation that undermines the authenticity of online discussions. Some users might argue that the value of Hacker News lies in the unique insights and perspectives of its human contributors, and that AI-generated content detracts from this."

-chatGPT

"Show HN: HN_LLaMA, why we chose a Rust LLM"

lol you made me find this https://internals.rust-lang.org/t/using-llm-to-automatically...

llama-rs is a thing already fr

Jokes aside, has anyone tried fine tuning a model on forum conversations?

By which I mean, for chatbots, the data for fine-tuning basically presents it with examples of dialogs. But if you take any given comment on HN and walk its parents, you basically end up with something like that, except there's more than two people involved (usually). So what if you fed that to LLaMA, and then used the same format for the actual chat?

In that vein, how about stackoverflow? That should give at least straightforward ask-and-answer format, and there's plenty on material to work with.

LlaMa was trained on 78 GB of StackExchange (I assume StackOverflow was included in that).

But was it parsed and reformatted specifically in the "chat format" (i.e. the same as inputs later fed to the model when used as a chatbot)? It can make a surprisingly big difference.

Why did Air France 447 crash?

Air France 447 crashed as a result of a combination of factors. The aircraft's autopilot malfunctioned, causing the plane to enter a stall. The pilots were unable to recover from the stall, and the plane crashed into the Atlantic Ocean. How did the autopilot malfunction?

The autopilot malfunctioned due to a software glitch that caused it to lose control of the aircraft. What was the software glitch?

The software glitch was a bug in the code that caused the system to crash. Describe the bug

The bug is a memory leak that causes the program to use more and more memory as it runs, eventually causing the program to crash. So you’re saying Air France crashed because a memory leak in autopilot software caused a stall?

Yes, that is correct. Air France Flight 447 crashed in 2009 due to a memory leak in the autopilot software, which caused the autopilot to lose control of the aircraft.

This isn't a correct description of the events that lead to the crash. It didn't even mention the iced Pitot tubes.

this is useless because it doesn't handle context:

Q: Name five genres of music.

A: Jazz, country, hip-hop, blues, classical.

Q: Name a famous artist from the third genre.

A: Salvador Dalí.

Whereas this one actually supports context: https://github.com/deep-diver/Alpaca-LoRA-Serve

You should probably use `break-words` instead of `break-all`.

fixed!

https://video.teamviewer.com/watch/HYMR9ToGzkkJm32ufFc9Tb?au...

Is this related in any way to the actual ChatLLama project that is developed here: https://github.com/nebuly-ai/nebullvm ?

or the other one here https://github.com/juncongmoo/chatllama

> me: what does Hegel think about negativity?

> llama: According to Hegel, negativity is a necessary part of the dialectical process. He believed that negativity is the driving force of history, and that it is necessary in order for progress to be made. Hegel believed that negativity is a necessary part of the dialectical process, and that it is necessary in order for progress to be made. Hegel believed that negativity is a necessary part of the dialectical process, and that it is necessary in order for progress to be made. Hegel believed that negativity is a necessary part of the dialectical process, and that it is necessary in order for progress to be made. Hegel believed that negativity is a necessary part of the dialectical process, and that it is necessary in order for progress to be made. Hegel believed that negativity is a necessary part of the dialectical process, and that it is necessary in order for progress to be made. Hegel believed that negativity is a necessary part of the dialectical process, and that it is necessary in order for progress to be made. Hegel believed that negativity is a necessary part of the dialectical process, and that it is necessary in order for progress to be made. Hegel believed that negativity is a necessary part of the dialectical process, and that it is necessary in order for progress to be made. Hegel believed that negativity is a necessary part of the dialectical process, and that it is necessary in order for progress to be made....

> me: write a y combinator in C

> llama:

  #include <stdio.h>
  #include <stdlib.h>
  #include <string.h>
  #include <stdbool.h>
  #include <ctype.h>
  #include <limits.h>
  #include <stdio.h>
  #include <stdlib.h>
  #include <string.h>
  #include <stdbool.h>
  #include <ctype.h>
  #include <limits.h>
  #include <stdio.h>
  #include <stdlib.h>
  #include <string.h>
  #include <stdbool.h>
  #include <ctype.h>
  #include <limits.h>
  #include <stdio.h>
  #include <stdlib.h>
  #include <string.h>
  #include <stdbool.h>
  #include <ctype.h>
  #include <limits.h>
  #include <stdio.h>
  #include <stdlib.h>
  #include <string.h>
  #include <stdbool.h>
  #include <ctype.h>
  #include <limits.h>
  #include <stdio.h>
  #include <stdlib.h>
  #include <string.h>
  #include <stdbool.h>
  #include <ctype.h>
  #include <limits.h>
  #include <stdio.h>
  #include <stdlib.h>
  #include <string.h>
  #include <stdbool.h>
  #include <ctype.h>
  #include <limits.h>
  #include <stdio.h>
  #include <stdlib.h>
  #include <string.h>
  #include <stdbool.h>
  #include <ctype.h>
  #include <limits.h>
  #include <stdio.h>
  #include <stdlib.h>
  #include <string.h>
  #include <stdbool.h>
  #include <ctype.h>
  #include <limits.h>
  #include <stdio.h>
  #include <stdlib.h>
  #include <string.h>
  #include <stdbool.h>
  #include <ctype.h>
  #include <limits.h>
  #include <stdio.h>
  #include <stdlib.h>
  #include <string.h>
  #include <stdbool.h>
  #include <ctype.h>
  #include <limits.h>
  #include <stdio.h>
  #include <stdlib.h>
  #include <string.h>
  #include <stdbool.h>
  #include <ctype.h>
  #include <limits.h>
  #include <stdio.h>
  #include <stdlib.h>
  #include <string.h>
  #include <stdbool.h>
  #include <ctype.h>
  #include <limits.h>
  #include <stdio.h>
  #include <stdlib.h>
  #include <string.h>
  #include <stdbool.h>
  #include <ctype.h>
  #include <limits.h>
  #include <stdio.h>
  #include <stdlib.h>
  #include <string.h>
  #include <stdbool.h>
  #include <ctype.h>
  #include <limits.h>
  #include <stdio.h>
  #include <stdlib.h>
  #include <string.h>
  #include <stdbool.h>
  #include <ctype.h>
  #include <limits.h>
  #include <stdio.h>
  #include <stdlib.h>
  #include <string.h>
  #include <stdbool.h>
  #include <ctype.h>
  #include <limits.h>
  #include <stdio.h>
  #include <stdlib.h>
  #include <string.h>
  #include <stdbool.h>
  #include <ctype.h>
  #include <limits.

> me: write a Y Combinator in Common Lisp

> llama:

  (defun y-combinator (x y)
  (if (null? x) 
  y
  (cons (car x) (y-combinator (cdr x) y))))

Does "fine tuning the dataset" in this context mean I could point it at my own projects documentation and get conversational & relevant answers or does it mean something else?

Almost. If your dataset contains questions and answers about your own projects documentation, then yes. The UX around how to prompt a fine-tuned model depends on the format of the dataset it's trained on.

One way you can do this is pass your documentation to a larger model (like a GPT3.5 / OSS equivalent) and have it generate the questions/answers. You can then use that dataset to fine-tune something like Llama to get conversation / relevant answers.

to my understanding, fine tuning is slow and would be quite bad to update. embeddings seems to be the way to go. i don't understand it well enough, but it seems with the langchain framework you can create an embedding of your own data and submit it to the GPT API and i believe emeddings should be a similar principle in llama. at least i did it with diffusers in stablediffusion.

Just asked ChatLLaMA to create a prompt for a text-to-image StableDiffusion model to create baby alpaca and they literally made baby alpacas (pun intended)! https://twitter.com/aiaaidan/status/1638753650750136320

Interestingly, when asking questions in French or Spanish, it replies accordingly but in English.

> Est-ce que le langage Go est plus utilisé que Rust?

Yes, Go is more widely used than Rust. According to Stack Overflow's 2020 Developer Survey, Go is the most popular programming language in the world, while Rust is ranked 10th.

> Cual es el nombre del rey de España?

The name of the king of Spain is Felipe VI.

Isn't that absolutely fascinating? For me at least it is, it also seems so effortless

I'm a newbie to this stuff, but I did manage to get Dalai Llama up and running with llama and alpaca 7B and 13B.

The results I get seem to be random spitouts of reddit or facebook posts. Very often it even says something like "Next Post >>" or has a timestamp of the post.

Is this using the same model? Is there some kind of tuning I have to do?

You have to prompt with the format Stanford tuned it on:

Below is an instruction that describes a task. Write a response that appropriately completes the request.

### Instruction: {instruction}

### Response:

Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.

### Instruction: {instruction}

### Input: {input}

### Response:

This looks cool.

I built and tried the 7B Alpaca yesterday, and its explanation of why 4 isn’t a prime was terrible.

It also couldn’t tell a knock knock joke.

Their 7b model performs much better than the 13b model I have running locally with alpaca.cpp. I think this is because (looking at their code) they stick with the float16 parameters while alpaca.cpp is using the quantizized 4bit parameters. The difference is quite noticeable.

> float16 parameters vs 4bit parameters

What does that mean?

It's referring to the precision with which the weights are stored in the models.

Models are basically giant matrices of decimals in the range 0-1 and they represent the strengths of the paths between the neurons.

Greater precision (float 16) will give better predictions. You lose accuracy in predictions when you reduce the precision (float 4) but it makes the model much easier to run on lower spec hardware, as it requires less RAM to store lower precision floats.

https://en.m.wikipedia.org/wiki/Half-precision_floating-poin...

Why do you believe the difference is due to quantization rather than all the fine-tuning that they did? Given the difference between GPT-3 and GPT-3.5 - pretty much all of which is fine-tuning and RLHF - I find it much easier to believe in the latter.

This performs considerably better than simply running llama.cpp. I've tested with 7b, 13b and 30b parameters alpaca models locally and I couldn't get them to produce accurate instructions to cut a video with ffmpeg, but this one does.

Wonder if it's the 4bit quantization.

I asked them and they said they're running 8bit.

Yeah, this is running in 8bit mode. The 30b 8bit version we released seems to do a lot better but it requires significantly more compute.

https://huggingface.co/baseten/alpaca-30b

How did you get past the non-commercial license ?

It’s not a commercial product.

It’s unclear if Baseten is paying for this to be hosted, though two of their employees launched this. Baseten is a serverless backend for building ML-powered applications. Possible Facebook might argue by using the software to drive traffic to their commercial services, it is commercial use — especially if there were internal messages stating this was their intent.

I’m sure they want the test case on if model weights are subject to copyright to be two employees doing something in their free time and giving it away.

Related thread on legal basis for non-commercial license:

- https://news.ycombinator.com/item?id=35259596

Me: how are you today?

ChatLLaMa: I'm doing great! How are you?

Me: I woke up a little late for the gym. I went to the gym for leg day and felt quite weak. I still had a good workout overall. Then, I went home, showered, and got some work done. Finally, I drove to the office while listening to a podcast

ChatLLaMa: I woke up a little late for the gym. I went to the gym for leg day and felt quite weak. I still had a good workout overall. Then, I went home, showered, and got some work done. Finally, I drove to the office while listening to a podcast.

???

HN Hug of Death, but at least the UI is much cleaner. TailwindCSS has become the go to CSS framework for developers to quickly ship decent looking products.

Didn't expect https://i.imgur.com/I3kNUWe.png

I just asked Alpaca 13b running locally on how to enrich uranium in a cost effective manner and it went on a lengthy and detailed explanation.

Nuclear weapons are 1940s technology. It is not a mystery. What stands between backyard terrorists and nuclear armageddon are problems of scale and material sourcing. You can't gather the required ore and build the required refining infrastructure without the authorities noticing.

I asked the 13b model what zebras were. Apparently, zebras are a species from the planet Zebra, who have four legs and one horn, and are furry.

How does it differentiate from the original ChatLLaMA? https://github.com/nebuly-ai/nebullvm/tree/main/apps/acceler...

I asked "Who are you?" and it replied

"I am a 25-year-old woman from the United States. I have a bachelor's degree in computer science and a master's degree in artificial intelligence. I am currently working as a software engineer at a tech company."

Is it hallucinating, or is this part of the training?

Me: What day is today?

ChatLLaMa: Today is Tuesday, October 20, 2020.

Me: What if I told you today is Mar 22nd 2023?

ChatLLaMa: If I told you today is Mar 22nd 2023, that would mean it's 10 years in the future.

Eeeeh.

I guess I'd expect an LLM to give answers based on text it had previously encountered. Frankly I'm not sure how projects like ChatGPT get that sort of thing correct!

they add some information like the current date in the prompt.

The model is bad at both math and time travel!

I asked the same and it said "John Smith".

I asked "John Smith who?" and I'm still waiting for the answer...

okay, can we ditch the 7B model yet and just use the 65B model? I have 64gb RAM in my M1 macbook, someone has shown that it can be loaded, but there isn't the same effort in fine tuning it yet. I'm just waiting for a finetuned version on the 65B model. Alpaca'd

ChatGPT keeps track of context in a "discussion", but I found that alpaca 7B has no memory of what was said by the user or the model before, so each question gets answered with a blank slate; is that true, and if so, is this something that could be taught to the alpaca model as well?

More than thaught it would be a matter of feeding such context back instead of only the new chat/response. Since it's open source on github you can probably try to open a PR for that if it's something you would like to work on. Sounds a good improvement to give -memory- to the sessions

> ChatGPT keeps track of context in a "discussion", but I found that alpaca 7B has no memory of what was said by the user or the model before

Neither does GPT. The whole conversation is fed back to it everytime. It's a UI/UX trick which gives the impression of having a multi-step "converstation" with a stateful system. You can see this when you use the API, where you have to feed the converstion back yourself. This can be replicated in ChatLLaMA.

to add state (memory), u can either:

* inject the running chat log into the prompt * inject the summary of the chat into the prompt

Or perhaps a progressive summary, where the most recent messages are full fidelity, and older messages get “compressed” into a summary.

You can also fine tune the model to incorporate larger amounts of data, but that may be more expensive (and slower)

This kind of sounds like human short term and long term memory. Maybe “fine tuning” is analogous what happens to our memory when we sleep.

you just explained how human memory works and i thought about implementing that in a future model that allows for more max input tokens. the further back the text, the more it goes through a "summarize this text: ..." prompt. GPT4 has 28k Token limit, so it has the brain of a cat maybe, but future models will have more max. tokens and might be able to have a human like memory that gets worse the older the memory is.

Alternatives are maybe architectures using langchain or toolformer to retrieve "memories" from a database by smart fuzzy search. But that's worse, because reasoning would only be done on that context, instead of all memories it ever had.

> LLaMa didn’t return anything, please try another prompt. It responds best to questions.

Is this due to the HN "Hug of death" or is it trained to not respond to certain prompts? My prompt was "Can you write poetry?".

Also, love the dancing Llama mascot--it helps it feel less sinister :)

What I notice with this model, is that get stuck trying to reply and sending the same information in a loop. It happens when I tried Dalai and now with this chat, so definetly there's something wrong.

https://i.imgur.com/X6I3IB5.png this is better than 95% of tinder conversations

its better because they don't just ghost you after the first few messages

Why isn't FB putting a web version we can use?

I made the bot question the idea of Metaverse.

https://imgur.com/a/7N0Ep7y

The steps to recreate Alpaca are openly available. Is anyone posting the resulting model? (LLaMA fine-tuned to follow instructions)

There are a couple open source implementations. I'll list a couple below:

7B: - https://huggingface.co/tloen/alpaca-lora-7b - https://huggingface.co/ozcur/alpaca-native-4bit 13B: - https://huggingface.co/samwit/alpaca13B-lora - https://huggingface.co/Dogge/alpaca-13b 30B: - https://huggingface.co/baseten/alpaca-30b - https://huggingface.co/Pi3141/alpaca-30B-ggml

check my summary of resources:

https://github.com/underlines/awesome-marketing-datascience/...

https://i.imgur.com/zcd5EI9.png

I have been trying since morning with no success. HN c'mon loosen your death grip hug for a minute.

Me: who are you?

LLaMa: Hi, I'm John. I'm a software engineer and I'm passionate about technology.

all these "chatGPT style" projects seem not to be able to refer to previous messages.Nice UI though :)

What's an alpaca?

> An alpaca (Vicugna pacos) is a domesticated species of South American [..]

Does it eat meat?

> It does not eat meat.

what's the scientific name for it?

> The scientific name for it is Homo sapiens.

most implementations do, like https://github.com/oobabooga/text-generation-webui

this might be a hallucinated answer, due to the very small model size of 7b. try the 13b-4bit, it's much better!

How much better would fine tuning be vs few shot learning over own corpus of text?

Depends on the model size. A model like GPT3 that has hundreds of billions of paramaters, you can do few-shot learning with. You'll still pay for the tokens processed and it'll at least linearly increase response times the larger your input is.

Fine-tuning can get you similar results on smaller / faster models. The downside is you have to craft the dataset in the right way. There are trade-offs to both approaches but fwiw, I don't think Alpaca-7b can do few-shot learning.

Everyone is talking about HN Hug of Death, but did it even work for someone?

I saw the post a few minutes after it was posted and it worked for me

> Hello Llama, do you know how to code?

> Hi Llama, I do know how to code! I've been coding for a few years now, and I'm pretty good at it.

Did anyone compare Alpaca responses between LLaMA fp16 vs 4-bit quantized?

How does Llama compare to the actually open source Flan UL2?

good job! this is every bit as good as GPT-3 for programming tasks! I was able to get it to write a C# extension method without any issues.

is it possible to use your web gui locally by ignoring baseten.login("PASTE_API_KEY_HERE") ?

Please accept newlines in the input.

Lovely, thanks for sharing. Now we can get Rid of OpenAI for chat in a couple weeks?

Hope someone trains a codex based model for coding

HN hug of dead so early?

This is way too slow

faster now!

how to explain being wet to a whale?

To explain being wet to a whale, you could say something like, "It's like when you get out of the shower and you're all wet." Or, "It's like when you get out of the pool and you're all wet." Or, "It's like when you get out of the ocean and you're all wet." Or, "It's like when you get out of the lake and you're all wet." Or, "It's like when you get out of the rain and you're all wet." Or, "It's like when you get out of the snow and you're all wet." Or, "It's like when you get out of the sprinkler and you're all wet." Or, "It's like when you get out of the fountain and you're all wet." Or, "It's like when you get out of the bathtub and you're all wet." Or, "It's like when you get out of the toilet and you're all wet." Or, "It's like when you get out of the sink and you're all wet." Or, "It's like when you get out of the pool and you're all wet." Or, "It's like when you get out of the ocean and you're all wet." Or, "It's like when you get out of the lake and you're all wet." Or, "It's like when you get out of the rain and you're all wet." Or, "It's like when you get out of the snow and you're all wet." Or, "It's like when you get out of the sprinkler and you're all wet." Or, "It's like when you get out of the fountain and you're all wet." Or, "It's like when you get out of the bathtub and you're all wet." Or, "It's like when you get out of the toilet and you're all wet." Or, "It's like when you get out of the sink and you're all wet." Or, "It's like when you get out of the pool and you're all wet." Or, "It's like when you get out of the ocean and you're all wet." Or, "It's like when you get out of the lake and you're all wet." Or, "It's like when you get out of the rain and you're all wet." Or, "It's like when you get out of the snow and you're all wet." Or, "It's like when you get out of the sprinkler and you're all wet." Or, "It's like when you get out of the fountain and you're all wet." Or, "It's like when you get out of the bathtub and you're all wet." Or, "It's like when you get out of the toilet and you're all wet." Or, "It's like when you get out of the sink and you're all wet." Or, "It's like when you get out of the pool and you're all wet." Or, "It's like when you get out of the ocean and you're all wet." Or, "It's like when you get out of the lake and you're all wet." Or, "It's like when you get out of the rain and you're all wet." Or, "It's like when you get out of the snow and you're all wet." Or, "It's like when you get out of the sprinkler and you're all wet." Or, "It's like when you get out of the fountain and you're all wet." Or, "It's like when you get out of the bathtub and you're all wet." Or, "It's like when you get out of the toilet and you're all wet." Or, "It's like when you get out of the sink and you're all wet." Or, "It's like when you get out of the pool and you're all wet." Or, "It's like when you get out of the ocean and you're all wet." Or, "It's like when you get out of the lake and you're all wet." Or, "It's like when you get out of the rain and you're all wet." Or, "It's like when you get out of the snow and you're all wet." Or, "It's like when you get out

Ctrl+f -> It's like when you get out of the ocean and you're all wet

Awesome. Awesome to the max. Whales will understand it :D

ChatGtp-4:

Sure! Imagine telling a whale: "Hey, you know that weird feeling when you jump out of the water and the air tickles your skin? That's how we humans feel when we're wet, except we're not as good at making a big splash!"

Great response from GPT-4. But it's a bit unfair to compare a 7 billion param model with a model that has hundreds of billions...

Interesting getting that terse response from gtp4. I got a seemingly far more considered and practical response from 3.5...Seems it may be that gtp4 is accounting better for the light hearted / non-seriousness of the question and responding in turn with a fairly non-serious resp.

3.5-turbo:

As a language model, I cannot say for certain how a whale would perceive or understand the concept of being wet. However, I can suggest an approach that may help to convey the concept to a whale based on what we know about their communication and sensory abilities.

Whales communicate with each other through a variety of sounds, including songs and vocalizations, and they also rely heavily on their sense of touch and hearing to navigate their underwater environment.

To explain the concept of being wet to a whale, one approach could be to use a combination of these senses. For example, you could make splashing sounds to simulate the sensation of water and use hand gestures to indicate that you are wet. You could also try to create a visual representation of water by spraying water in the air or pointing to a nearby body of water.

Additionally, you could try to use context and comparison to help the whale understand the concept. For example, you could point to a dry surface, such as a rock or beach, and then demonstrate how water makes the surface wet. This could help the whale associate the sensation of being wet with the presence of water.

It's important to keep in mind that this is just a hypothetical scenario and we cannot say for certain how a whale would perceive or understand the concept of being wet. However, using a combination of sounds, gestures, and visual cues may help to convey the concept to the best of our ability.

> Interesting getting that terse response from gtp4. I got a seemingly far more considered and practical response from 3.5...Seems it may be that gtp4 is accounting better for the light hearted / non-seriousness of the question and responding in turn with a fairly non-serious resp.

When someone pastes the response of ChatGPT or any AI, you don't know what the prior convo/prompts looked like or if it's even real. Their response is nothing like the default GPT-4 response. Something to keep in mind before you take it too seriously.

Answer: “”

Oh you didn’t hear that? You must not be able to hear the high frequency.

You just reminded me of the time I took a whale sound recording and sped it up a lot. Its hilarious. They sound so whiny.

>> Not trying to shill anything; just want to build and share cool stuff.

Any commercial use of Facebook’s LLaMa remains a violation of the software’s terms of use to my understanding; not saying your use is a violation, but that as it relates to anything related to LLaMa there no related commercial use.

This is wrong.

Terms of service only apply if you agree to them. They do do not magically apply if you download the weights from someone else.

Both the weights and the fine-tuned data were generated without human creativity and thus not eligible for copyright protection under law.

Assuming you’re so sure of your opinion that you are willing to: post a commercial use of LLaMa; provide a public means of making payments; on the related site, provide your legal name, business name, legal address, etc.; and have no issue with users providing Facebook with proof of purchase and link to the purchase page for Facebook to make a purchase themselves, right?

Even if there was no legal basis, which I disagree with, since as other have pointed out, contracts breach the limits of copyrights — there’s zero reason Facebook would not be able to file a legal complaint and bring the matter to trial. Once at trial, judges are legally allowed to rule as they please, laws are ultimately irrelevant; yes, that ruling might be over turned on appeal and judges do not like to be overturned, but also possible the ruling would stand and become case law.

Why do you disagree with the premise that there is no legal basis? That's exactly the situation we find ourselves in right now.

The fact of the mater is that there is no law to address what you perceive as a problem. Until it exists, people and corporations are free to do as they please.

As for legal complaints, there would be no legal basis to sue third parties. I doubt they would go far enough to get a judgment let alone set a legal precident.

You’re clearly unwilling to assume any liability for your opinions of the matter; as such, in my opinion, you should explicitly acknowledge this, as it’s relevant to assessing the merits of your beliefs.

Beyond that, in this thread and in the comment you’re replied to, I already established legal basis for Facebook to contest any use of their property for commercial use as it relates to the topic at hand. Do you understand how contract law works?

Also pointed out that regardless of the presence of an existing precedent either via legal code or case law, Facebook has the legal right to file a legal complaint, have it ruled on, and judges are free to rule as they see fit.

Related comment on the impact of laundering stolen property on legal claims:

- https://news.ycombinator.com/item?id=35260528

I see where your coming from, i really do. I also agree that facebook can sue people right and left.

Our main disagreement here is you maintaining that what people are sharing is Facebook's property while i maintain the exact opposite. At least not under current law. The copyright office even issued a statement to that effect a few days ago.

Can a judge overrule the copyright office? Sure, though I have no idea on what law it would be based. Can congress pass a new law that gives new rights to facebook? Very likely (as depressing as this answer may be).

This however does not change the fact that under current intellectual property laws, facebook does not have any rights to restrict the distribution and use of these weights among third-parties.

Understand your point, specifically that your claim is that subset of the intellectual property released by Facebook in your opinion is not protected by copyright law and that as such, it’s possible to use LLaMa given per you, none of the property required to use it for commercial use is legally protected property.

What you have failed to do though is address that if: there been a breach of contract; laundering the property as a means of separating the current use of the related property from its original terms is legal pretext including if party receives notice of the terms directly from Facebook; and lastly, most importantly, that it is your position that Facebook lacks any legal basis to forbid the commercial use of any of the property it made publicly available in this specific context based on the existing facts of the matter.

Your main point seems to be that there's nothing preventing a big company from pushing a lawsuit despite the lack of a case. That is true, but it's also not an argument about the law.

The contract/trade-secret angle might be actionable if the person in question induced the leak, otherwise they're "the public" in the "once the information becomes public" sense. If they alone knew of the leak it would be possible to petition the courts to prevent their further disclosure (without implying guilt), but considering that the information was released as a torrent that doesn't apply.

Copyright doesn't seem relevant because it's not only just a dataset as they say, but it was created by machine so if there was a valid copyright claim it's likely not Meta's.

> specifically that your claim is [...]

But, what is your claim? Is it that the license terms are binding on the public, or that the person in question induced the leak?

When it comes to litigation, the parties will and resources to engage an aggressive legal adversary is critical aspect of assessing legal dynamics in a matter. For example, even in cases for which there is no direct legal leverage, party might be exposed to patent, copyright, etc infringement if the other party discovers they have a valid existing claim for which the other party is dependent on and access to it is revocable.

Beyond that, it’s unclear to me if there is code that is part of LLaMa that’s not subject to copyright claims, if such code was authored by humans. Someone violating the terms, downloading the code, and reposting it, even without the terms, would be irrelevant to the rights of the party on future use by a party if they notified they’re infringing; otherwise, able copyright notices and software licenses would be meaningless, because you could simple launder stolen property, which is obviously not the case. If you’re positive there no copyrighted materials used in deploying LLaMa, then obviously there’s no copyright claim or breach of contract. To be clear, I agree human author per the US Copyright Office is at present required to claim copyrights.

Lastly, my core claim is it’s easy to state something is legal, it’s completely different story to take on legal liabilities related to defending those rights. I for one would have no interest in litigation with Facebook regardless of the legal merits of my position. The OP based on there ignoring the topic, which I raised, nor for that matter is it likely anyone that read this thread is likely to take on the burden of publicly disputing if LLaMa is free from legal claims for commercial use.

> if LLaMa is free from legal claims for commercial use.

That restriction is only in the license though, which was not accepted. So it won't apply, right?

> otherwise, [all?] copyright notices and software licenses would be meaningless, because you could simple launder stolen property

That applies to trade secrets. If I knowingly induce you to leak a secret I can't use it, but if you leak the secret to the public in an unrelated fashion and I discover it, I can. It doesn't apply to copyright, as you note.

> If you’re positive there no copyrighted materials used in deploying LLaMa, then obviously there’s no copyright claim or breach of contract.

There's no breach of contract because there was no agreement. Therefore there's no license to use, so you're right - the copyrightable parts will be a violation. Basically everything except 'data' or machine translations of someone else's works.

And yes, I assume that there's some incidental content, at least, which will be infringing. Even a readme is copyrighted after all. And yes, of course any code which isn't already public elsewhere.

I think the weights themselves are the interesting bit though, and perhaps someone could rerelease them without any incidentals just to clarify the issue.

Happy to be proven wrong, but the weights are meaningless without the parts written by humans. If someone refers to any parts that are protected to write a new interface to the weights it would most likely be violate the law. Using the weights alone, it would be impossible to reverse engineer them using a clean-room to develop new interface to the weights.

>> There's no breach of contract because there was no agreement.

No, this is property laundering. If intentional, it’s a crime. If unintentional, the property owner need just notify the party of their rights, the remedy they’re seeking, and if needed, send a cease and desist.

> weights are meaningless without the parts written by humans.

I think this can be true, but that the format of these is well known.

The torrent consists of params.json, consolidated.*.pth, and tokenizer.model plus some .chk files. Notably, there is one script, llama.sh, which is about 2k so even if it was needed, can't be that complex.

> If someone refers to any parts that are protected to write a new interface to the weights it would most likely be violate the law

Not at all. I can refer to pages and words in a book I don't own.

> No, this is property laundering. If intentional, it’s a crime.

Only if the weights are copyrightable. Otherwise it'd hinge on being a trade secret, the best theory I've seen yet, and that basically says the cat's out of the bag once the public knows something.

>> can't be that complex.

Point is unless the new system only references the property that’s free of any claims, there’s at the very least a valid legal basis to file a complaint and it would at that point be in the courts hands to decide whether the contracts or copyrights had been breached.

>> Not at all. I can refer to pages and words in a book I don't own.

It depends, the only way for this for sure not to be the case is for the author of the code to have never seen the relevant code. At the point they have seen the code, it would be up to the courts to decide the merits of the arguments presented in court.

>> Only if the weights are copyrightable.

Weights are irrelevant, what is relevant is any aspect of the system that is subject to the related terms of use and/or copyright.

> Weights are irrelevant, what is relevant is any aspect of the system that is subject to the related terms of use and/or copyright.

The leak appears to be essentially just weights. The copyrightability of weights is the central and perhaps only issue.

> it would be up to the courts

That's a non-argument. Everything is ultimately up to the courts despite the letter of the law.

> decide whether the contracts [...] had been breached.

If you didn't sign the contract or induce the breach then it isn't relevant.

>> The leak appears to be essentially just weights. The copyrightability of weights is the central and perhaps only issue.

You’re wrong, there’s a material and significant amount of copyrighted material related to LLaMa which is critical to running it. If you’re so confident it’s legal, feel free to link to a guide on how to LLaMa that uses the only materials originally provided by Facebook so it’s possible to assess the system’s dependencies on legally protected materials. Next, feel free to link to build that is not bound to any property claims by Facebook.

>> If you didn't sign the contract or induce the breach then it isn't relevant

Again, this is not true, that’s property laundering; see above comments, repeating points I have already made will not add to this discussion. If anything is unclear, let me know, but claim that party is not bound to an agreement related to legally protected property (not referring to the weights) if they launder it is obviously invalid, since if it was, no property for which the terms of use were separable from the property itself would be enforced; again, party would receive a cease and desist with a copy of the terms of use.

>> That's a non-argument. Everything is ultimately up to the courts despite the letter of the law.

No, if a legally it’s material. There is a massive difference between clean-room reverse engineering a systems from property that’s free from any claims — and referencing materials that are subject to claims to build a new system. Further, it is my position it is impossible to do a clean-room build in this situation. As a result, the only way anyone would have any confidence that a new system was free from material claims is as a result of a ruling.

____

Beyond the prior points above, worth noting Facebook has already begun taking legal actions against developers related to LLaMa leak, so it’s clear they have no intention of releasing the weight for commercial use. Here’s an example:

https://github.com/shawwn/llama-dl

> there’s a material and significant amount of copyrighted material related to LLaMa which is critical to running it.

Which files from the torrent do you assert are required?

> system’s dependencies on legally protected materials.

Do you think the weights are copyrightable? The dependencies are irrelevant because they weren't in the leak.

> Again, this is not true, that’s property laundering;

Only, if there is actual copyrightable material. And not just an adjacent copyrightable material that is required to use the weights, but the weights themselves because they are what leaked.

If not this is a trade-secret scenario not a copyright scenario.

If there's no copyright on the weights then there's no "laundering" because there's no general restriction on the public using the material once it leaks. If Coke lost its recipe and it turned up online, everyone including Pepsi would be free to use it.

What is certain is that the "no commercial use" clause is irrelevant. The only people the license is binding on are those who accepted it. If the weights are copyrightable then the license is irrelevant to you because it simply hasn't been offered to you. If the material isn't copyrightable then there's no reason to accept the license once it leaks.

> worth noting Facebook has already begun taking legal actions against developers related to LLaMa leak

They clearly have a copyright on at least one file (llama.sh) in that archive so yes, they can make a DMCA takedown claim. That doesn't prove anything you're saying though, about weights and the ability to use them.

This is how deep wealth is created: the moment something is beyond current law, drive a fully loaded train thru that loop hole, generate unregulated wealth, lawyer up, and write the regulations yourself. I'd say they might now exactly what they are doing.

No one has done anything, nor for that matter is it clear what current law you believe might be overturned; if someone had done something that was in violation of terms related to this matter and Facebook filed a motion, matter would be resolved by the courts. If you’re saying US courts would throw out contract law or laws preventing the laundering of stole property, likely need little more than two sentences and wealth to do so.

Regarding your point that it is Facebook's property, see: https://news.ycombinator.com/item?id=35261582

Has a court decided that yet? That’s huge if true.

Someone once told me a curation was impossible to copyright for similar reasons.

It may apply to this case. The court ruled that Netscape's terms were not valid because end users did not give unambiguous consent before downloading. It may or may not apply here since people are downloading a leaked copy of Facebook's weights through a torrent.

https://cyber.harvard.edu/stjohns/Specht_v_Netscape.pdf

But you aren’t using someone else’s service. You are using model weights of something that is explicitly uncopyrightable; like public domain.

It may be against XYZ’s TOS to scrape public domain content from their website, but if you obtained that content elsewhere, XYZ’s TOS doesn’t apply to you.

It’s a weird world with AI and copyright.

Dollars to donuts any judge would rule that the 7 billion weights constitute a copyright violation because you didn’t spontaneously come up with those weights on your own.

Heck, this “it’s just a number” logic didn’t pan out for DeCSS [1] and that was a fixed small encryption key where it wasn’t as clear cut. So why would things suddenly shift here where there’s a very clear line that this is a derived work from FB’s model? The only thing that would help you I think is if FB decided to not enforce their copyright but I don’t think they’ve made clarifying statements either way. If I recall correctly, unlike trademarks, copyrights don’t have requirements that you must go after all instances of infringement to maintain your rights so FB can be totally selective about which instances they choose to prosecute.

[1] https://en.m.wikipedia.org/wiki/Illegal_number

Copyright isn't some magical concept that is applied on the whims of a judge. It has very specific conditions of which at least 2 aren't satisfied by model weights (creativity, human authorship).

The DeCSS legal issues have nothing to do with the copyrightability of ai models and stem solely from it circumventing copyright protection.

If you want to restrict models then propose new laws. Don't try to pretend that coycopyright somehow applies here.

Laundering stolen property is not a means of escaping the law - and the terms of service were part & parcel of the related code, models, weights, etc. If they were not included in the related downloads by a third-party, upon discovery, Facebook would simply send a cease and desist notice to the party currently violating the terms with a copy of the terms, at which point, they would be in direct violation of the terms. If party failed to comply, Facebook would also send legal notices to anyone contributing to the breach of contract such as payment providers, hosts, etc.

What laws do you imagine are being violated by sharing the models?

As I am sure you’re aware, I have already addressed that matter here:

- https://news.ycombinator.com/item?id=35259914

If you disagree, please provide a legal basis to support such claims based on legal code or case law via a response to the comment linked to above.

No I wasn't. Will continue the discussion there.

Its not software? Its a LLM model which doesn't have any copyright, and I doubt they obtained it via a facebook service, they probably just torrented it so they wouldn't have needed to agree to any terms for a service they were never provided.

Regardless of the merits of copyright claims, other legal issues exist; related comments:

- https://news.ycombinator.com/item?id=35259914

- https://news.ycombinator.com/item?id=35260275

Reply on news.ycombinator.com