Faster Than Llama 2: DeciLM LLM with Variable GQA
deci.ai
deci.ai
Sounds great
> This license is, in part, based on the LLAMA 2 Community License Agreement, with a series of modifications. Use of DeciLM for hosted services may require a separate license.
Ah, so a bait-and-switch. It's not open source, and not permissively licensed.
No single indicator of a starting/base price.
Still, a free for personal use model is better than none. Hopefully, we will soon have some adaptation from Llama 2 that matches it, fairly soon.
The license explicitly blocks “Hosting Use” and does not even differentiate between “model as an API” and in-application use.
I hope the community rejects this kind of BS.
another day, another model. okay how is it quantized to 4bit, what about on AMD chips with ROCm, what about with some other forms of paging I’ve seen, tokens/sec, does it pass AI grade school and higher education, is it fine tuned?
equivalent cost per hour if using various cloud solutions when optimized?
I haven’t personally tested either yet.
Neither is unmodified HF diffusers (or worse, the SAI implementation).
Some of the techniques they describe are neat though
I dunno about a precise number...Less than 30? The UniPC scheduler in particular can get great results below 20 iterations. Karras DPM schedulers are pretty good with 30.
> In terms of unmodified diffusers, you mean fine tuned to whatever they are working on?
No, people hack non-default augmentations onto the vanilla diffusers (or SAI) pipe to increase speed or increase quality. Some examples included the channels last memory format, downsampling some stages to FP16, TomeSD, self attention, and some more user-guided augmentations like controlnet or tiled diffusion.
"alternate" runtimes like the ones available via torch.compile or AITemplate provide some pretty massive speed gains (~2x) if you need them, but are less common since they are hard to install and HF diffusers only (where most of the SD community uses the original SAI implementation).
Just as an example implementation, VoltaML is quite speedy out-of-the-box, with most of the sane optimizations already implemented, though its missing some features (like tiled diffusion/area prompting) that the SAI based pipelines have: https://github.com/VoltaML/voltaML-fast-stable-diffusion
How about accuracy? Are the results good? I can shit out a blank string at gigabits per second easily...
Searching for the string "accu" yields no results. "acc" only yields "acceleration".
This is stupidity and should be ignored.
EDIT: HOLY KEK - they have a "carbon emissions" chart showing them saving 95%. Jesus christ, ignore everything they say, they are blind deaf and dumb.