elchiapp

Bare worklet with IPC, exactly. Let me know if there's anything I can help with.

Curious to hear what constraints are there that aren't tackled by the current offering of local runtimes/SDKs for inference.

You can build anything! Check out our tutorials here: https://docs.qvac.tether.io/sdk/tutorials/

Although an LLM chat is the starting point for many, there are many other use cases. We had people build domotics systems to control their house using natural language, vision based assistants for surveillance (e.g. send a notification describing what's happening instead of a classic "Movement detected") etc. and everything remains on your device / in your network.

Hey folks, I'm part of the QVAC team. Happy to answer any questions!

That line means that you don't need to create an account and get an API key from a provider (i.e. "asking for permission") to run inference. The main advantage is precisely that local AI runs on your terms, including how data is handled, and provably so, unlike cloud APIs where there's still an element of trust with the operator.

(Disclaimer: I work on QVAC)

How is that different from choosing not to adopt a technology because it’s not widely used therefore not widely documented? It’s the timeless mantra of “use boring tech” that seems to resurface every once in a while. It’s all about the goal: do you want to build a viable product, quickly, or do you want to learn and contribute to a specific tech stack? That’s the trade off most of the time.

Has anyone tried emulating linux/amd64 on Silicon with GPU support? I'm testing an engine to run LLMs locally that only runs on that, no support for arm.