2,063 karma · joined March 3, 2016
That should easily run an 8 bit (~360GB) quant of the model. It's probably going to be the first actually portable machine that can run it. Strix Halo does not come with enough memory (or bandwidth) to run it (would need almost 180GB for weights + context even at 4 bits), and they don't have any laptops available with the top end (max 395+) chips, only mini PCs and a tablet.
Right now you only get the performance you want out of a multi GPU setup.
There are many platforms out there that can run it decently.
AMD strix halo, Mac platforms. Two (or three without extra ram) of the new AMD AI Pro R9700 (32GB of RAM, $1200), multi consumer gpu setups, etc.
If this guy has a non chemical cure, I'm all for it. In fact I'm actively researching children psychologists to stave off the meds as much as we can, the problem is that 99% of psychologists are quacks, so choosing them is tough.
As someone who lived through that, I refuse to let him. All of memories of school are just feeling anxious about everything, just tight and suffocated, always in a panic. I started living when I started taking anxiety pills at 39 years old, and I can see my 2 year old having the exact same anxiety ticks and fits I have.
I don't know at what age I'll medicate him, but I'll do it as soon as I notice he isn't coping and happy anymore.
Horrifying is forcing him to experience that because you can't comprehend us.
Having a child forced me to fix my life, and I'm incredibly happy because of it.
Other my journey posts are look at me with only enough subject matter to disguise it.
This post is about sharing knowledge, the others are about sharing experiences.
Whether you agree with it or not will vary greatly depending on the effectiveness of your country's government, the wealth gap, your country's currency purchasing power, your general political alignment, and a whole slew of other factors.
I don't know if I'd call that "mature".
Datastar developers are free to do what they want with their code, but as someone who releases open source software, I'm tired of projects using open source simply to create a moat or user base then switch to a proprietary model.
Basically hackable source available hardware which in the Linux world is a huge advantage.
AMD has forced Nvidia out many proprietary technologies (gsync) just by countering with an open technology. There is no reason it couldn't execute the same plan against Intel.
Intel lost its competitive edge by trying to maximize captured value. It seems this person want to double down on the strategy, instead of going for absolute best product at the price possible.
But on a home NAS? What problem would having to power it down and power it on for drive replacement create? You're going to resync the array anyways.
I don't mind them and I do use them but I consider them a very small QOL improvement. I don't really replace my disks all that often. And now that you can get 30TB enterprise samsung SSDs for 2k, two of those babies in raid 1 + an optane cache gives you extremely fast and reliable storage in a very small footprint.
Recently (neovim), delightfully. It just uses Lua and exposes APIs for absolutely everything.
The IPMI interface sure is nicer though