Fork of Facebook’s LLaMa model to run on CPU
github.com
github.com
What sort of on-board compute do you typically have today?
watt-hour: you cannot subtract time from power, it doesn't make any sense.
watt/hour: delta of power per time, something very weird.
Researchers at US Berkeley came out with the algorithm they named SpeedFolding in October of last year. Watch https://youtu.be/UTMT2WAUlRw?t=511 and then realize that linked excerpt is sped up 9x.
If we had 9x faster compute we could have laundry folding robots which is one thing, but that amount of compute would enable robots to do tons more tasks in industry.
Getting robots to move quickly is easy; getting them to move quickly to exactly where you want them, or with exactly as much force... that is much, much more difficult. Double for mobile robots where you don't have a good energy source. If cost is an issue that is another dimension -- powerful and accurate actuators are extremely expensive.
There is another reply to your comment that shares a lot of what I have experienced. You have so many pieces of code that need to run and a good handful of them are working on something like LIDAR point clouds with a million 3D points in them, plus some cameras running several different image recognition and segmentation algorithms, and you want to have fast cycle times, it just all adds up. Every serious robot I have ever worked on is maxing out its system, even ones at Google X with a full desktop CPU, a high end NVIDIA graphics card, and a couple secondary ARM CPUs.
That definitely helps me understand why the footage of the robots in this video had to be sped up: https://youtu.be/Ybk8hxKeMYQ
https://www.cnbc.com/2022/03/23/ukrainian-flees-to-poland-wi...
In general, the purpose of Bitcoin is not to get rich, but to have a currency that is universally accepted and not tied to a political party’s fiscal decisions.
[0] did you know the original snake oils contains more omega-3 and therefore improves cognitive function when compared to lard? I did not. But you can get omega-3 elsewhere, and the people who made the term synonymous with fraud didn't use those snakes, so…
or games. People could have been studying or doing something more important than wasting time and energy. I get that it is entertainment, but so are board games and that don't require mining rare earth minerals or putting pressure on the grid as you can always play board games with candles on.
Or going outside. People could have been studying or doing something more important than wasting time and energy.
Or not being locked in the education facility. People could have been studying or doing something more important than wasting time and energy.
The most ecologically friendly thing you can do is go to sleep. If you want to play games, do it while the sun is out!
(/s, just in case)
That said, gaming is what gave us GPUs (which have developed for gaming over the course of decades) so that we can now utilize them for more interesting and "productive" applications.
So, for me, in the end I'm happy the PC gaming industry and user base has been pushing GPU capability.
What really (finally) more or less killed GPU mining was the Ethereum move to PoS (Proof of Stake).
https://ourworldindata.org/grapher/per-capita-energy-use
That's like the average person running 27 NVIDIA A100s at max capacity at all times!
> As for what you should look to invest in?
> I'm sure it's just a coincidence that training neural networks and mining cryptocurrencies are both applications that benefit from very large arrays of GPUs. [...]
> If I was a VC I'd be hiring complexity theory nerds to figure out what areas of research are promising once you have Yottaflops of numerical processing power available, then I'd be placing bets on the GPU manufacturers going there
[1]: https://www.antipope.org/charlie/blog-static/2023/02/place-y...
With a bunch of people trailling behind with "it kind of works" open alternatives.
It's not so bad. Nvidia could come and say, "hey, I'm going to lock down your GPU so that you can only use it to render polygons in my whitelisted list of video-games, and then you pay us $$$$$$ to buy our 'datacenter' thingy for anything else." But if they do it, people will go and buy the competitor's product.
And yes, probably their 4090 are being bought by some rich kids with their parents' money, but I reckon most of it are sales to professionals, people who would justify their purchase decision with more than playing First-person-shooters. I for example play videogames with my gf, and we have equivalent GPUs. Hers is AMD and costs less than mine, even if it does the same, but I went for Nvidia so that PhysX were available and I could use Pytorch and Numba+GPU and even C++ CUDA. The moment Nvidia locks that down, I'll have to switch to AMD.
You just described gaming consoles.
Good luck with AMD.
If organics is the only factor, I don't know why those wouldn't perform as well as human or ape brain cells.
Meatcubator: https://youtu.be/Z_ZGq8Tah0k
Growing human brain cells: https://youtu.be/V2YDApNRK3g
1) Spin it up on a cluster in Belarus
2) ???
3) Profit?
You can download it from Facebook, but it's behind "apply for access" form. Magnet links floating around are just a workaround around that form.
That said, commercial use is forbidden by the license specified in the form: https://docs.google.com/forms/d/e/1FAIpQLSfqNECQnMkycAp2jP4Z...
(when fed the leaked bing prompt, my AI decided it was Australian and started tossing in random shit like "but here in Australia, we'd call it limey green" when asked about chartreuse, i assume because the codename for bing chat is 'sydney')
(enter at your own risk? I think it's kinda safe but still 4chan)
Browsing page by page is not a good idea.
Search for "aicg" or visit https://boards.4channel.org/g/catalog#s=aicg to see the AI Chatbot General thread (a new one is created every time the previous one hits the reply limit).
Stable Diffusion was in the same place as this in the same time frame of the model getting released. Its only been a few days.
https://github.com/KoboldAI/KoboldAI-Client To read more about current popular models.
https://koboldai.net/ is a way to run some of these models in the "cloud". There's no account required and the prompts are run on other people's hardware, with priority weighting based on how much compute you have used or donated. There's an anonymous api key and there's no expectation that the output can't be logged.
The models that run on hardware locally are very basic in the quality of output. Here's an example of a 6B output used to try to emulate chatgpt. https://mobile.twitter.com/Knaikk/status/1629711223863345154 The model was finetuned on story completion so it's not meaningfully comparable.
It's less popular because the hardware required for the great output is still above the top of line consumer specs. 24 gb vram is closer to a bare minimum to get meaningful output, and fine-tuning is still out of reach. There's some development with using services like runpod.
FB trained a LLaMA-I (instruction tuned) variant for sports, just to show they can, but I don't think it got released.
User: <question or task>
Assistant:
not so bad !
I computed the speed by doing speed=number of words/ total run time
It's not that hard to create a consumer-grade desktop with 256GB in 2023.
Even without enough ram, you can stream model weights from disk and run at [size of model/disk read speed] seconds per token.
I'm doing that on a small GPU with this code, but it should be easy to get this working with the CPU as compute instead (and at least with my disk/CPU, I'm not even sure that it would run even slower, I think disk read would probably still be the bottleneck)
A lack of an absurd number of CPUs just means it's slow, not impossible.
However, the 65B parameter, according to the benchmarks, is such a beast that you might be able to do some things on it that are not possible on ChatGPT (despite all of ChatGPT's quality of life features). Amazing times.
So before you start a task, you sort of describe the domain, and the model is separated into the third most useful and relevant to that topic/query, and 2/3rd most distant from that realm. Then either just the 1/3rd is used in a detached fashion, or it works as 2 layers of cache, one in ram one on disk.
per the readme it looks like there a few bugs to figure out in case anyone here is a pytorch expert
Most gaming desktops have a solid gpu but not enough vram. Pity having the gpu idle here
Uh-oh, bad start.
It could be venv as well, I suppose, I haven't used conda.
Use it in a container or a VM unless you enjoy re-installing your system from scratch.
Or better still, don't use it al all and let it wither away: these kind of braindead projects need to be put down with extreme prejudice.
But ... I seem to recall ... Conda tries to install GPU drivers does is not? ... Is that not the case anymore?
Because if it still does, your theory about "Why would your system care" and all that doesn't really hold water.
But a lot of those people want to play with LLMs.
Do you have enough ram? (not swapping to disk)?
Looks like this is just tweaking some defaults and commenting out some code that enables cuda. It also switches to something called gloo, which I'm not familiar with. Seems like an alternate backend.
Mark LLM: “ Yes, unfortunately, the media and our competitors are all over the idea that Meta is a “dirty company”. They have tried to spin all our successes and accomplishments in a negative light. This has been incredibly frustrating and demoralizing for us, but we know that we are working hard to build a great company and we are confident that our efforts will be rewarded. In the end, our products speak for themselves, and despite all the negative media coverage we are focused on continuing to build great products for our users and being an amazing place for them to socialize in the virtual world.”
George: I really like that orange sweater
Jerry: Yeah, I just found black so depressing
George: Orange is such a great color! Orange is the new black.
...