It’s a React SDK that lets you control your app with voice, gaze and hand gestures, no mouse or keyboard.
The idea is simple: instead of figuring out where to click, you can just look at something and say what you want.
Example: look at a row → “open this” | say “navigate to equipment” | say “create a ticket” → it builds it
Some interactions are instant (navigation, selection), while more complex ones use an LLM and take a few seconds.
What makes it different from most “AI agents” is that it runs inside the app, not outside of it.
It has access to: React state, routing, visible UI
So it doesn’t rely on screenshots or DOM guessing, it actually understands what’s on screen.
I originally started this thinking about environments where mouse/keyboard don’t work well (gloves, operating rooms, field work), but it’s also interesting for internal tools and dashboards.
This is v0.1 still rough in places, but the core flow works.
GitHub: https://github.com/haelo-labs/exocor npm: https://www.npmjs.com/package/exocor
Curious what you think, especially if this feels useful or just like a gimmick.