Show HN: Offline GPT Store, run a pre-prompted LLM in your browser

Hi HN

I wanted an LLM chat bot that runs locally and offline so I would feel comfortable about sharing personal details with it.

Similar to how OpenAI GPTs have pre-configured instructions, this project provides the ability to encode a pre-prompt into the URL so it can be shared. Effectively, making a free, offline and decentralised GPT store albeit with lower quality for the responses. To make it easy to find these "GPTs", I'm using Twitter as a form of storage.

This project uses webllm to run a Llama-2-7b model locally in the browser. Bigger models can be used (not supported yet) as well but they would take more time to download and require more RAM.

WebGPU is required since the webllm model was created to run in the browser. Android is supported with the latest version of Chrome (v121). Should also work on most Desktops with Chrome v113 and higher.

Please feel free to provide suggestions on what improvements should be added.

Let me know what you think!