gopisuvanam

# LLM in the borwser using WebLLM

This is an example of how an LLM can be used safely in the browser without the need for servers or external services. This proves one can use AI in edge computing. This particular notebook requires GPU with atleast 8GB of VRAM. This is an amazing way of using LLMs without installing anything. Works across most modern browsers!!!

Credits: [Web LLM Github](https://github.com/mlc-ai/web-llm)

Note: The notebook takes 3-4 four minutes to load the LLM for the first time. Next time onwards it will only take 25 seconds as the model is cached. Simple queries are taking 3-5 seconds.

Running Large Language Models (LLMs) directly in the browser is now possible, thanks to WebLLM, which leverages WebGPU and other modern browser technologies. This brings powerful AI capabilities to client-side applications without the need for a backend, ensuring privacy, low latency, and portability. In this article, we’ll explore how to set up and use WebLLM in a browser-based JavaScript application, including examples of how to run a model, handle chat completions, and enable streaming responses.

Use the link to experiment with working code for LLM in the Browser. Note: you will need GPUs on the device.

This article explores the diverse applications of JavaScript across various domains, including scientific computation, simulations, machine learning, fluid dynamics, equation solving, engineering design, data visualization, statistical analysis, bioinformatics, financial modeling, and robotics.

The article gives code snippets for experimenting.

you can also host it yourself by downloading and putting behind a webserver.

that's it, npm/node/python etc. try and let me know any feedback.

it seems to be running JS in webworker. which is good as it does not block the main thread, but then you cannot access some of the JS features like the DOMs and importing modules dynamically.

still I ♥ the fact that it can loaded from guthub pages without any backend!!

Thanks. I think the learning part is a good point. we are adding notebooks for learning foundations of JavaScript.

thanks! I hope you will find it useful!

thanks for the support. looking forward to feedback.

Website: https://scribbler.live Web-app: https://app.scribbler.live Github: https://github.com/gopi-suvanam/scribbler/

Notebook for javascript experimentation (in the browser). Overview:

- Easy to use javascript experimentation tool. Runs without backend (node.js/npm/pip install/ngnix etc etc). Can be used online or offline. - Has a minimalistic UI to put focus on user generated code/content. It is light-weight as it uses minimal external libraries for styling. - Comes pre-loaded with Plotly, D3 and DI-Libs for power usage. Third-party libraries/modules can be easily loaded.

I noticed that you pay for an RSS reader. RSS is supposed to be for easy crawling by simple machines.. so instead of using a cloud based server why not use a browser addin/app?

Programming is more than just writing code.. its about figuring what to build, architecting the solution and putting together code to build that.. ChatGPT in fact helps in the last part. It is in fact allowing me to write code faster and hence giving me more time to focus on the first two things..

nice.. i think the embeddable sql will be useful..

Can this be used to check if our private keys are saved anywhere (across users and orgs) in github? An employee once accidentally copied some of our private code along with keys into a public repo.

Python has been my favorite for products and scripting. I've been advocating for Python's dominance when the world was after Java. But now I'm more inclined towards JavaScript. And agree with you Deno is taking the right approach (as opposed to nodejs).

Having JavaScript in the front end and backed both can reduce resource requirements as well.

Old but relevant article. Figuring out the intersection of "simplest" and "useful" features is the trick.

PDF was an proprietary format (of Adobe) until 2008. It is no longer the case. Right now there are multiple tools to edit PDF documents online or offline. Check this for example: https://www.pdfescape.com/