Hacker News Reader

Comment by stavros | Hacker News Reader

I've been looking for something that can recognize the mute button in call software (Zoom, Meet, etc) and click it, but everything I've found seems to do image detection.

I understand that running in a browser makes things hard, but is there a better method? I guess I could write an extension for Meet and Jitsi, but Zoom would then be the odd one out.

View on HN

I know what you are asking is useful on its own, but Zoom support starting with the microphone muted, or even not joining the audio stream at all, check out these options: Settings->Audio->Mute my microphone when joining a meeting Settings->Audio->Automatically join audio by computer when joining a meeting

On Mac, you can do this with the renowned automation software Keyboard Maestro. The relevant feature is called "Found Image" Actions and documented here https://wiki.keyboardmaestro.com/Found_Image?s[]=image

You can see a real-world example by Jason Snell [1]. There, he used the feature to find the "deactive account" button in Slack and saved hundreds of clicks.

[1] https://sixcolors.com/post/2022/08/keyboard-maestro-comes-th...

Not Linux, and not exactly what you're asking for, but Windows PowerToys has VideoConference mute which seems to install an audio driver that can globally mute. Not sure if something like this exists for Linux.

https://docs.microsoft.com/en-us/windows/powertoys/video-con...

It's easy to globally mute on Linux, but that doesn't show the "you are muted" UI elements, and makes the UX a bit more confusing. I wanted something more explicit, which is hard.

Does image detection with "always show meeting controls" not work?

It does, but at 5 fps and unacceptable CPU usage, sadly.

…you only have to do detection when required though (after the hotkey is pressed). Curious about the architecture of the 5fps solution.

I need to do it constantly, as I want to update the status if I mute/unmute manually. I want to make a physical mute button with a LED, so that's why I need that.

I believe Autohotkey's image detection has a parameter to select the relative range inside the application window to actually perform a search. If you constrain it to a smaller area (because you know that the UI only appears in that area) this can greatly improve the performance versus searching the entire window.

Edit: I should mention I have no idea if this works on the linux versus of autohotkey.

Nut.js

aka “nut.js is a desktop automation framework for Node.js that allows you to program your mouse and keyboard with JavaScript or TypeScript”

https://nutjs.dev/

I know JavaScript pretty well too, but I prefer to write local automation in Bash. Why do you need such a twisted solution?

Reply on news.ycombinator.com