Show HN: Hacky Meta Glasses GPT4 Vision Integration
github.com
Super hacky implementation due to the lack of an SDK. Fun project though.
In the foodlog demonstration I just made a fake fb account (sorry zucc) called "Mye Food-Log".
github.com
In the foodlog demonstration I just made a fake fb account (sorry zucc) called "Mye Food-Log".
Hope that Apple Vision Pro gives a more robust api to developers and that forces Meta to open up.
I will use this approach for a poc I have in mind. Great job and thank you for open sourcing!
However, not too familiar with trawling through decompiled APKs (also presume there's some sort of internal secret they're using too)
No worries!
I was under the impression that Apple is keeping camera access completely locked down?
https://developer.apple.com/videos/play/wwdc2023/10094/?time...
Recorded on OBS, uploaded to linkedin, screen recorded on iOS then uploaded to Youtube.
Is that not a standard workflow haha (initially was just to send to 1 mate who didn't have linkedin lol)
`Using this image, estimate the nutritional information of it and output it as a JSON using this data structure: { "calories":int; "protein":int; etc... } `
It may be “super hacky” as you say, but these projects are important.
We may have had a theoretical understanding that this was possible, but seeing your demo opens up our experiential understanding.
Lights up totally different parts of the brain.
The lens are pretty subtle, and they look like normal wayfarers. When the light comes on, as you make a recording or take a picture, then they notice, but not all of the time.
They offer better protection against "unsuspecting picture taking" than normal phones, as the light is vaguely tamper proof. I'm not sure they record if you take them off either.
but, my friend, why do you want to cover up the light?
When the new Meta smart glasses were announced, I suddenly had >6 people comment on my old glasses. All the comments were positive, but it did make me more aware that I might get a negative reaction in the future.
I always make it a point to explain that I’m not recording and that if I was the light would be on. And that if you try to cover it it actually will not allow you to record (which is true).
From my experience, a lot of users fail to get Node.js projects running - the distributions (Debian, Ubuntu) still ship ancient 10.x or 12.x versions, npm may or may not be installed, "npm install -g" will try writing to /usr by default and fail with EACCESS, etc. Bun gets the UX right - I think that author could even try "bun build --compile" to get ELF binary with bun interpreter and JS payload.
What the glasses do now reliably are taking photos, short videos and transferring them into the Meta View app. Bluetooth audio transferring from my car stereo into the glasses is annoying and I couldn't find a setting to change / stop that.
As for Meta Vision there's so much innovation to happen here, nice!! gives me tons of ideas
What similar quality vision models are there outside gpt4 ? It’s annoying it has so maybe (albeit mostly hack-prompt-able) restrictions.
I did not know Meta released smart glasses. Has anybody found them useful or is it more of a novelty?
The potential is definitely there and I'd be very surprised if the Meta Reality Labs team haven't already added heaps of features internally and are just waiting or staggering releases.
Are you wearing these out in public pretty regularly? Are people noticing the lenses? My main concern with buying these is that I will end up not wearing them in public eventually, which kind of makes them not super useful. I worry about that because I can see some people seeing those lenses and going off on the whole bit about am I filming them? am I filming their kids? And we all now the rabbit hole of uncomfortable public social iteractions this can take one down. So, I worry I would run into too many of these people downtown or something and end up being anxious about even wearing them out anywhere.
What is your experience and how long have you been wearing them around in public (if you are)?
No one has noticed they're any different, especially outside as I got transition lenses so they just go into normal rayban sunglasses mode. You could have a similar effect if you wore those old "new" 3D glasses out and about.
Even colleagues had to squint.
If I do ever have a bad interaction with strangers I'll just tell them about the lengths meta have gone through to prevent bad actors. (i.e: knowing you're trying to cover up the flash).
They make brilliant headphones. They have good microphones so they also work a great hands free, perfect for driving.
The cameras are subtle, and only get noticed when you point them out to people. In the street you're not going to notice they are different from any other wayfarer.
I don't use the assistant, because I'm never going to say "hey facebook" in public. Nope, fuck that.
the video is smooth, even if you are running. the stills are kinda useful for "in the moment" pictures.
For the newer version, I think its "hey meta" and they appear to do a lot more than just allow you to read facebook messenger messages.
Great job!
I first attempted a more standard approach via a webhook hosted on Heroku linked to an fb page, which theoretically would of worked.
But, Meta has limited you to only sending messages/pictures to "real people".
I believe they have to be on your friends list for a certain amount of time too but haven't properly tested all the edge cases.
Ok, so how long have you had these? I was unaware they had released these but they look pretty fun AND the price point is surprisingly approachable. Are you wearing these out in public pretty regularly? Are people noticing the lenses? My main concern with buying these is that I will end up not wearing them in public eventually, which kind of makes them not super useful. I worry about that because I can see some people seeing those lenses and going off on the whole bit about am I filming them? am I filming their kids? And we all now the rabbit hole of uncomfortable public social iteractions this can take one down. So, I worry I would run into too many of these people downtown or something and end up being anxious about even wearing them out anywhere.
What is your experience and how long have you been wearing them around in public (if you are)?
Also, legit question... how did you succesfully make an alt facebook account? lol Anytime I try with a different name so I can have an alt for buying stuff on marketplace it blocks that new account out because I can't verify it is me (because it's a fake name).
But the latency was about 30 seconds so during an interview it wouldn't be that useful haha
Glass was obviously an early experimental design with loads of shortcomings, but at least in concept, a personal HUD was fairly interesting. I didn't care about the camera and found it disappointing that it was the only part people seemed to focus on. Isn't this just the camera/mic bit without the cool part, tied to a Facebook/Meta account instead of a Google one?
I agree $1500 was a bit steep for something barely above prototype level, but this is basically an earbud/mic that allows me to take hands-free photos - something I do maybe a few times per year.
Do also note that people wear glasses in private settings as well as public.
You're trying to assuage my concerns with the information about the light. That's Meta responding to these concerns and taking efforts to avoid upsetting anyone.
But once the market is proven then another set of glasses will be marketed as having the feature of silent recording. By that time the force of the market will be too great and my concerns will be laughed at and I'll be called a luddite and told I never should have had an expectation of privacy to begin with.
This is the embrace before the inevitable extinguish.
I'm not based in the US so hopefully some local laws will help me out a bit, but at the end of the day, the law is not my moral barometer. I will respect the rights of the users of these devices while also exercising my own rights, as much as I am able to, to treat them as social pariahs.