Gemma 2B 60 tk/s WebGPU on M1

Fine-tuning on a niche task I guess! E.g. a bunch of LoRAs that can be easily swapped depending on the task!

Will Google add performant 7B support? 7B's quality is good enough for a serious Web APP.

This is the problem. There isn't any use-cases for this other than being a toy. Even on the web.

But is it saying "'affor affor affor...(repeated 100+ times)" really fast?

But actually, did you download the linked model from Kaggle ("gemma-2b-it-gpu-int4.bin") and upload it into the demo? It is working fine for me out of the box.

this can happen if you try to feed a cpu model to gpu inference. make sure you download the gpu model if you are using gpu accelerated ml inference.

It thinks it’s a pokemon.

Try a different prompt