- This API should be programming language agnostic (like, with a well defined ABI). Ideally an IPC based one, with a well defined wire format and protocols.
- Not a singular API, but an “API system” with extensibility taken into account from the beginning (with things like versioning). It should be able to support arbitrary new API definitions so that other apps could consume, and also produce.
- Such an API system should be used by the OS itself too, of course. So the OS would have the resulting benefits and the whole system would be more uniform.
I guess you and I have different use cases and definition of "modern application usage". I assume you spend most of your day in a GUI, while I spend most of mine in a terminal.
Pipes can handle binary data and sure you could pipe Skypes output through sed or awk... Though something like ffmpeg is more likely, and it will happily take stdin as input.
I just want eyeball tracking for the mouse and voice to LLM to bash.