334 karma · joined March 12, 2016
Building CanViT (Canvas Vision Transformer), and kickstarting Active-Vision Foundation Models (AVFMs).
Contact: me@yberreby.com
---
meet.hn/city/45.5031824,-73.5698065/Montreal
Socials: - linkedin.com/in/yberreby - github.com/yberreby
---
Iran's Islamic regime has provided material and monetary support to the Houthis.
Crippling their capabilities aligns with the goal of protecting global shipping.
For context, I'm a PhD student. Work-life balance is already... elusive.
The main non-trivial parts are proper state machine / concurrency management, and AirPods interaction; in particular, detecting a stem click while the microphone is active. I worked around this by having the mic-off-to-mic-on transition use a media player Play event, and mic-on-to-mic-off do silence detection. It's super hacky but actually works surprisingly well.
Currently looking into using `AVAudioApplication.setInputMuteStateChangeHandler(_:)`, like AirMute [1] does, so that I don't have to rely on silence detection and can manually terminate the voice command with a second click.
If you want to roll your own version of what I described today, it should be pretty easy to do so based on what I wrote if you have a Max x5-x20 plan and feed it to Opus. Bonus points, you get to customize it to your exact needs.
I often have ideas while cleaning around, cooking, etc. Claude Code (with Opus 4.5) is very capable. I've long wanted to get Claude Code working hands-free.
So I took an afternoon and rolled my own STT-TTS voice stack for Claude Code. The voice stack runs locally on my M4 Pro and is extremely fast.
For Speech to Text, Parakeet v3 TDT: https://huggingface.co/nvidia/parakeet-tdt-0.6b-v3
For Text to Speech, Pocket TTS: https://github.com/kyutai-labs/pocket-tts
Custom MCP to hook this into Claude Code, with a little bit of hacking around to get my AirPods' stem click to be captured.
I'm having Claude narrate its thought process and everything it's doing in short, frequent messages, and I can interrupt it at any time with a stem click, which starts listening to me and sends the message once a sufficiently long pause is detected.
I stream the Claude Code session via AirPlay to my living room TV, so that I don't have to get close to the laptop if I need extra details about what it's doing.
Yesterday, I had it debug a custom WhatsApp integration (via [1]) hands-free while brushing my teeth. It can use `osascript` for OS integration, browse the web via Claude Code's builtin tools...
My back is thankful. This is really fun.
Or, if you don't need to use C (e.g. for FFI or platform compatibility reasons), you could use a language with a compiler that does it for you.
Source: I did prépa.
In other words, the required amount of data scales with the square root of the compute. The square root of 2 ~= 1.414. If you double the compute, you need roughly 1.414 times more data.
I'd love to use NixOS itself, of course, but it's not a native option on this machine due to the missing M4 support in Asahi. For now, I'm trying to see how much package/configuration management discipline I can reclaim on macOS, and familiarize myself with Nix in the process.
I could have just used a set of Ansible scripts and Homebrew, but that didn't seem quite as interesting as trying Nix out.