Care to share some links? My lack of GPU is the main blocker for me from playing with local-only options.
I have an old laptop with 16GB RAM and no GPU. Can I run these models?
I have an old laptop with 16GB RAM and no GPU. Can I run these models?
https://huggingface.co/TheBloke
There's a LocalLLaMA subreddit, irc channels, and a whole big community around the web working on it on GitHub nd elsewhere.
edit: I forgot to directly answer you: yes you can run these models. 16GB of plenty. Different quantizations give you different amounts of smarts and speed. There are tables that tell you how much RAM is needed per which quantization you choose, as well as how fast it can produce results (ms per token). e.g. https://github.com/ggerganov/llama.cpp#quantization where RAM required a little more than the file size, but there are tables that list it explicitly which I don't have immediately at hand.
It might have been a contract violation for the guy who redistributed it, but I'm not a party to that contract.
I'm calling it now. Based on what I see today. Europe will position itself as a leader in AI legislation, and its economy will give way to the nations that want to enter the race and grab a chunk of the new economy.
It's a Catch 22. You either gimp your own technological progress, or start a war with a nation that does not. Pretty sure Russia and China don't really care about the ethics behind it. There are plenty of nations capable enough in the same boat.
Now what? OK, so in some hypothetical future China has an uncensored model with free reign over the internet. The US and Europe has banned this. What's stopping anyone from running the Chinese model? There isn't enough money in the world to enforce software laws.
How long have they tried to take down The Pirate Bay? Pretty much every permutation of every software that's ever been banned can be found and ran with impunity if you have the technical knowledge to do so. No law exists that can prevent that.
If it did, OpenAI wouldn't exist.
Forms of this argument get tossed out a lot. Laws don’t prevent, they hopefully limit. Murder has been illegal for a long time, it still happens.
Wouldn’t that violate the Nemo dat quod non habet legal principle and so you cannot hide behind the claim that you weren’t party to the contact?
The contracts in these cases are somewhat similar to an NDA, without the secrecy aspect. Restricted disclosure of public information. You can agree to such a contract if you want to, and a court might even enforce it, but it doesn’t affect anybody else’s rights to distribute that information.
Contracts are not statutes, they only bind the people directly involved. To restrict the actions of random strangers, you need to get elected.
It is the same as any confidential data. Logs, readings from sensors, etc etc. If it's confidential and given to a 3rd party through a contract that doesn't mean that it's suddenly not confidential data for the rest of the world, even if the 3rd party leaks it.
And if you really have a lawyer trying to tell you that some, at best, extreme grey area, is fine to build a business on, I think you should find a new lawyer.
All of these are surmountable problems.
We can beat OpenAI.
We can drain their moat.
I've got an AI powered sump pump if you need it.
Update: That is only true for the legal system I am currently residing in. No idea about e.g. the US.
MPT, OpenLLaMA, and Falcon are probably the most generally useful.
For code, Replit Code (specifically replit-code-instruct-glaive) and StarCoder (WizardCoder-15B) are the current top open models and both can be used commercially.
While you can run all kinds of GPTs locally, GPT-4 still smokes everything right now – and even it is not actually good enough to not be a lynchpin for a lot of cases yet.
I would argue that it creates better images too.
Not if you want it to write adult (graphically pornographic or violent) content.
Here's a handy model comparison chart (this is a coding benchmark, so coding-only models tend to rank higher): https://i.imgur.com/AqSjjj2.jpeg
It beats Claude and Bard.
You could probably get a 4bit 15B model going in 16GB of RAM and be approaching GPT4 in capability.
...on an old laptop, lol
Let's eat OpenAI's lunch! They deserve it for trying to steal this tech by "privatizing" a charity, hiding scientific data that was supposed to be shared with us by said charity whose purpose was to help us all, and dishonestly trying to persuade the government not to let us compete with them.
I'd post a more well rounded benchmark but the problem is that all non-coding benchmarks are currently more or less complete garbage, especially the Vicuna benchmark that rates everything as 99.7% GPT 3.5 lol.
The situation for the little guy is wildly better than most people imagine.
There are exceptions of course, as there are a few models (e.g. Vicuna, Baize) that don't do well at coding at all but otherwise perform well for chat, and the coding models I mentioned that game the benchmark by sacrificing performance in all other areas.
If you exclude those, it's very a accurate overall reasoning level comparison, at least it fits most to what I've seen their performance was for various tasks when testing out individual models. The only other valid benchmark that isn't coding are the SAT and LSAT tests that OpenAI runs on all of their models, but afaik there isn't an open version that would be widely used.