Run LLaMA and Alpaca on your computer
github.com
github.com
Either way, that DMCA is worthless.
So while what you say may be true the DMCA seems to have worth for these orgs because they can get code removed by the host, who is uninterested in litigating, and the repo owner likely is even less capable of litigating the DMCA.
Unfortunately as a tool of fear and legal gridlock DMCA has shown itself to be very useful to those with ill intent.
I actually wrote a Wikipedia article on the intersection of copyright law and deep learning models the other day (https://en.wikipedia.org/wiki/Artificial_intelligence_and_co...). I was hoping to include a section on the copyrightability of model weights, but was sadly able to find 0 coverage in reliable sources.
Let's start with a non-AI example: compilers. If I have the source code for Linux, I can compile it, but I don't own the kernel binaries I made. This is because the compiler is a purely mechanical process, not a tool of human creativity. Copyright in the US attaches to creativity from human authors. So the source code would be the creative work, not the binaries.
We don't normally talk about this because ownership over the source code still flows through to the binaries. Your permission to copy that Linux binary is downstream of Linus having granted you permission to do so under the GPL. If you had instead copied, say, the NT kernel, you would be infringing the copyright on the NT kernel source code by distributing binaries of it.
So now let's go to AI land. You've collected a bunch of training data and dumped it into a linear algebra blender. That's like compiling source code: the ML trainer program adds no creativity or authorship, so you haven't gained any ownership over the data. Remember: this training data is scraped off the Internet from other people's work. Fair use merely makes it non-infringing to do this, it does not mean you own the result.
There are two avenues by which Meta could still get US copyright over the language model:
- They could make a model with their own training data that they made, and use their ownership over the training data to get ownership over the model.
- They could assert ownership over the compilation of training data.
Compilation ownership is kind of weird. Basically, in the US, you can make a compilation of other people's work and own solely that. Like, say, a "Top 10 Songs I Like" playlist[0]. But even then the creativity and authorship rules still apply. These models are not being trained by having humans manually select specific works that would do well in the model. They scrape the Internet and train on everything[1]. In fact, they usually don't even use their own scrapes; they use Common Crawl, LAION-5B, and/or The Pile.
Whether or not any of this is right would require someone to actually share LLaMA, get sued by Facebook, and then assert this legal theory. And hope that Facebook does not assert any other legal claims, such as misappropriation of trade secrets, which might actually stick.
[0] Or in a particularly egregious example, someone copyrighting their Magic: The Gathering deck in protest of this nonsense.
[1] Stable Diffusion at least uses an "aesthetics score", but AFAIK that's generated by an AI so also not copyrightable.
Because if you do that, then all I have to do to pirate a book is train a model on that book and sell the trained model as mine, which does not make sense.
I guess what I want to say is that in this matter of AI, you can't have your cake and eat it. If you want to have copyright over your weights, be prepared to also pay for the rights of the content your weights were based on.
And I think nobody in the AI world want to walk through that avenue.
Music can be copyrighted, and it can also be made from samples from other music that's copyrighted. But sampling still happens without infringing copyright.
I'd say the small handful of bits flipped in a model from training on some text, or piece of code, or an image is even less copyrighted information brought over than a music sample or borrowed/referenced melody.
TL;DR: There's a good fair use argument but it isn't a given that fair use always applies.
Same rule holds true for human programmers!
Seems highly inefficient, but so was every move from assembler -> C -> .NET -> JavaScript -> Electron.
Running neural networks will become cheaper too, all computers will have hardware accelerators for them.
I've already seen functions like LessThanOnePage(text) that combine a long prompt + freeform text in a slow LLM that basically says "will this text fit on one printed-out page". Takes several minutes to run, but requires a tiny fraction of the brainpower required to efficiently implement the function.
Kind of seems like a new level or generation of high-level (slow to execute but fast to code) vs low-level (fast to execute but slow to code) programming.
Python development Node.js development Desktop development with C++
What if you already have visual studio installed and you did not check all of these?
how is this:
npx dalai llama 7B
npx dalai serve
# build this repo
git clone https://github.com/ggerganov/llama.cpp
cd llama.cpp
make
#For Windows and CMake, use the following command instead:
cd <path_to_llama_folder>
mkdir build
cd build
cmake ..
cmake --build . --config Release
# obtain the original LLaMA model weights and place them in ./models
ls ./models
65B 30B 13B 7B tokenizer_checklist.chk tokenizer.model
# install Python dependencies
python3 -m pip install torch numpy sentencepiece
# convert the 7B model to ggml FP16 format
python3 convert-pth-to-ggml.py models/7B/ 1
# quantize the model to 4-bits (using method 2 = q4_0)
./quantize ./models/7B/ggml-model-f16.bin ./models/7B/ggml-model-q4_0.bin 2
# run the inference
./main -m ./models/7B/ggml-model-q4_0.bin -n 128Today, just 2d after running dalai successfully, it refuses to start and just hangs in there. Unclear why.