Hacker News Reader

alexpinelon Dot – A standalone open source app meant for easy use of local LLMs and RAG

Yes! Of course because the LLM is running locally it is not as advanced as bigger models like Claude or GPT, but you can definately quiz the documents. From my experience it performs better with specific questions rather than more ambigous questions that require extensive understanding of the whole document.

alexpinelon Dot – A standalone open source app meant for easy use of local LLMs and RAG

Right now the minimum amount of RAM I would recommend is 16gb, I think it can run with less memory but that will require a few changes here and there (although they might reduce performance). I would also strongly recommend using a GPU over CPU, in my experience it can make the LLM run twice as fast if not more. Only Nvidia GPUs are supported for now and the CUDA toolkit 12.2 is required to run Dot.

alexpinelon Dot – A standalone open source app meant for easy use of local LLMs and RAG

Hi! I'm the guy who made Dot. I remember experimenting with a few different vector stores in the early stages of the project but decided to settle with FAISS. I mainly chose it because it made it easy to perform the whole embedding process locally and also because it allows to merge vector stores which is what I use to load multiple types of documents at once. But I am definately not an expert on the topic and would really appreciate suggestions on other alternatives that might work better! :)

alexpinelon Dot – A standalone open source app meant for easy use of local LLMs and RAG

With those settings I would recommend GPU. CUDA acceleration really makes it faster, but keep in mind the CUDA toolkit 12.2 install will be a some 3-4gb

alexpinelon Dot: Standalone app for local RAG with Mistral 7B

thanks! right now it runs on Apple Silicon and Windows, it works far better with a GPU on PC but should also work with CPU if there is enough RAM. Although some people have mentioned there are some issues with the Windows app so I am looking into that right now.

alexpinelon Dot: Standalone app for local RAG with Mistral 7B

Hi! Really happy to see my project being posted around, please feel free to give feedback or ask any questions! :)

alexpinelon Open source alternative to Nvidia's Chat with RTX

Little project I have been working on, Dot is a standalone app that allows users to run an LLM locally and create a custom AI trained with documents.

alexpinelon Dot, a free standalone app to train Mistral 7B LLM with your documents locally

This free desktop app allows to perform RAG with Mistral 7B fully locally, very simple to use and seems to work well on my M1 Pro mac

alexpinelon Free standalone app to run RAG with Mistral 7B LLM

Here is a little pet project of mine, it is a standalone electron js app meant to run a packaged version of Mistral 7B with a focus on loading and interacting with docs in a fully local environment, kinda like a mini chat with RTX.

Runs well on Apple Silicon and on Windows with GPU acceleration although file loading is slower on windows. Current supported files are: pdf, docx, xslx, pptx, and markdown. 16gb of RAM are recommended and might actually be necessary.

Needs some work and will likely open source it soon. Main issue right now is the file loading system as it is rather counterintuitive, it only allows to select directories with the files rather than just choosing the files themselves. Windows installer is also quite janky as it is my first time working on such a project and Inno setup refused to work properly.

Anyway hope you find it interesting! Please let me know what you think :)

PS: website looks BAD on mobile