The Meta Raybans are a fantastic device hobbled by a weak LLM and supported by a bad actor - Meta [1]. Frankly I feel dirty for running any Meta software on my phone. I can talk to ChatGPT or Claude using the glasses but unfortunately I can’t show them what I’m looking at which is what the OP has built.
[1] https://arstechnica.com/security/2025/06/meta-and-yandex-are...
P.S. I'm working on Android app to talk to any LLM ( iOS one is already out: Roxy Voice: https://apps.apple.com/app/apple-store/id6737482921?pt=12710... ) using an OpenAI-compatible completions endpoint.
For open source glasses there's a few potential options, I think omi.me has an open smart glasses project.
I'm surprised no one has found a way to properly jailbreak them, it's a bit out of my wheelhouse but would be fun to try one day.