Show HN: a Rust lib to trigger actions based on your screen activity (with LLMs)

FWIW, I have been working on a variation that is supposed to be eventually on a similar topic, but it is confined to a specific terminal - https://github.com/ayourtch/prompter

For the time being it doesn’t do much other than intercepting the input/output, and is a bit buggy at that - so, no LLM there :)

But posting anyhow in case it can be useful for someone to avoid reinventing the plumbing.

cool will look into it!

The (post) title mentions a Rust library, I see code in TypeScript, and the installation instructions involve Python. Frankly, I am confused.

EDIT: To make it clear, my point is that the description could have been clearer, especially regarding the Python dependency.

The library is written in Rust, can be used (via WASM) in Typescript and uses an OCR API which is running locally on a python server

GitHub shows a chart of which languages are used in a repo. It’s clearly majority Rust.

I just use python temporarily for the OCR. But the plan is to implement Apple Vision & other OS native impl