Alexa was the closest to achieve significant usage since you can use it within the privacy of your home.
For voice UIs the non clear boundaries on what you think it can or cannot do is also a huge hurdle. After you get a couple “sorry I cannot do that” you stop using it
I went through this exercise with GPT voice. It's an awesome capability, but other than perhaps walking outside, or sitting in my office, there's no other space where it feels "ok" to just spontaneously talk to something.
A grey area is when you perhaps have headphones in / on and it looks like you're in a phone conversation with somebody, then it kinda feels ok, but generally you're not going to take a phone conversation in a public area without distancing yourself from others.
There's a reason most casual communication these days is text rather than voice or video calls.
"Siri, lights to HALF."
"Siri, lights to HAAAAALF."
"Siri, LIGHTS TO FIFTY PERCENT!"
Silly how OpenAI could blow all voice assistants out of the water today, if they just added Android intents as function calls to the ChatGPT app. Yes, the "voice chat mode" is that good.
With alexa i can program if/then statements, like basically when i say X then do Y. If something like chatgpt requires the same thing then i don't see the advantage.
So LLMs today can do this a few ways. One they can write and execute code. You can ask for some complex math (eg calculate the tip for this bill), and the LLM can respond with a python program to execute that math, then the wrapping program can execute this and return the result. You can scale this up a bit, use your creativity at the possiblities (eg SQL queries, one-off UIs, etc).
You can also use an LLM to “craft a call to an API from <api library>”. Today, Alexa basically works by calling an API. You get a weather api, a timer api, etc and make them all conform to the Alexa standard. An LLM can one-up it by using any existing API unchanged, as long as there’s adequate documentation somewhere for the LLM.
An LLM won’t revolutionize Alexa type use cases, but it will give it a way to reach the “long tail” of APIs and data retrieval. LLMs are pretty novel for the “write custom code to solve this unique problem” use case.
> and to a lesser degree malformed output
What's cool, is that this isn't a huge issue. Most LLMs how have "grammar" controls, where the model doesn't select any character as the next one, it selects the highest-probability character that conforms to the grammar. This dramatically helps things like well-formed JSON (or XML or... ) output.
Yes, I was thinking about even something as if/then, which could be configured in the UI and manifest to GPT-4 as the usual function call stuff.
The advantage here would be twofold:
1. GPT-4 won't need you to talk a weird command language; it's quite good at understanding regular talk and turning it into structured data. It will have no problem understanding things like "oh flip the lights in the living room and run some music, idk, maybe some Beatles", followed by "nah, too bright, tone it down a little", and reliably converting them into data you could feed to your if/else logic.
2. ChatGPT (the app) has a voice recognition model that, unlike Google Assistant, Siri and Alexa, does not suck. It's the first model I've experienced that can convert my casual speech into text with 95%+ accuracy even with lots of ambient noise.
Those are the features ChatGPT app offers today. Right now, if they added a basic bidirectional Tasker integration (user-configurable "function calls" emitting structured data for Tasker, and ability for Tasker to add messages into chat), anyone could quickly DIY something 20x better than Google Assistant.
Of course this isn't fool proof, and there still needs to be work defining capabilities of systems, etc. (although these are tasks AI can assist with). But it's promising - "teaching" the system how to do new things is relatively simple, and effectively akin to describing capabilities rather than programming directly.
The basic idea is to instruct the llm to output some kind of signal in text (often a json blob) that describes what it should do, then have a normal program use that json to execute some function.
Even though everyone's seen AirPods by now, in those rare occasions when I'm on the phone in public, I feel compelled to have my phone out and vaguely talking at it, so it's clear I'm on a phone call and not a crazy person.
I'm curious if we would see similar usage with the pin, where voice commands in public are always performed with the hand up for the projection screen (it will still prompt looks, but hopefully be clear in context, "oh they're doing some tech thing").
Of course at this price point, it's highly dubious that we'll see anywhere near the ubiquitous market penetration of AirPods (which garner understandable complaints about the price point sub-$200, and that's with a clear value prop).
The other reason they are mostly impractical - keeping a charge. *wired* headsets were great in this regard, but then there's the wire, and now, there's the phone (that may not even support the wire?).
Combine that with areas like GPT Vision, (GPT?) Whisper, etc .. it'll start feeling a lot more natural here very soon i suspect.
TBH i'm surprised Apple isn't pushing this much harder. They tout Siri so hard but it's just worthless to me. It feels like apple could make a AI Pin like this, but visibly from the public side i have zero idea that the're even working in this space. It feels like they purposefully watched the boat sail away.
edit: Sidenote, Pin + Airpods would be a nice way to interface more quietly too.
These next gen AI voice assistants are still a solid improvement over Google's current offerings, but they'll feel like a massive jump into the future for folks that have been stuck in Apple's ecosystem, and that's probably where the biggest opportunity lies.
After watching the presentation, I am now curious about Humane’s thing though, but I’m still going to hold off for a bit because I want to see the failure modes first and I also don’t want to rush out and be one of the first to buy the brand new 3Com Audrey.
The issues isn’t “communicating with the device” it’s “communicating around other people”.
I have almost negative interest in having to recite the technical specifics of my web search to my phone on the train to work. I have even less interest in having to listen to the person next to me trying to do the same.
I don't see how what you are saying holds any water in that scenario.
It does not seem right to speak of a single reason. There are probably multiple. So, IMHO it would be more productive to come up with a list and put some weights on the options if you want to dissect this matter.
IMHO one very strong factor / important reason (one that you ignore) is the social context. Ie the reaction of others in the same physical space, as you start talking out loud, seemingly unmotivated.
Humans are social animals, and so the reaction of others to the actions you do tend to be very important to a large fraction of the population. What is acceptable in one context simply isn't in another. Also, the exact tolerances tend to differ with the local culture (here "local" is used in the sense "geographically/physically local")
It's not just about not annoying others here. In this case it's also about a thing as imprecise as "perceived self image". Some people (I'd argue, most people) dislike having the perception that others perceive them to be mentally unstable or rude. Most people need some kind of social acceptance for the actions they do.
One significant trait of some mental instabilities (as well as some drug induced behavioral changes) is that those affected will spontanously start talking in public. You will probably know the Tourettes Syndrome, and the alchoholic rambling about because these cases often imply quite rude and offensive verbiage and/or loud volume, but these are not the only cases.
People in general are well adept at detecting such anomalous behaviour as it is part of our insticts trained through Evolution. Also the uncomfortable feelings that observing this type of behaviour leads to will lead many to react with a "confront or escape" (aka. "fight or flee") response (a stress signal), which is not beneficial to social interaction in general.
TL;DR: If you speak out in public without a very clear and socially valid reason (speaking to an object is not that) you are not only rude to others, but you also cause them stress... and you will have to face the social stigma of being perceived as insane.
(edit: grammar/typos)
Except... this problem is known to be trivially solvable. After all, the very act of putting a flat rectangle to your ear makes talking out loud in public not just perfectly acceptable, but mundane and not worth paying attention to (subject to social norms dictating where it is or isn't OK to be on the phone).
As for talking to yourself signalling insanity... I'd hope that stupid and probably developmentally retarding idea died long ago, and the "talking to yourself out loud in public" subtrope being dead since wireless earphones got ubiquitous some two decades ago.
The modern reality is, hearing someone "talking to themselves" is normal, and 99.9% of times means they're on a call.
The point is it's not, though. As a society we have generally established that it is rude to be speaking out loud on the phone in public. Especially on the bus or the train or waiting for same or in the shop or at a movie or any number of other places. I genuinely think it would be easier instead to list the places where it would be okay (in a busy street, if you step to one side). Even in these places there is some expectation that you show a little shame to be doing it, as though you didnt want to but had to because the call is important
If we could subvocalise with throat or other microphones/bone speaker then maaaybe, but I feel like it's better left to a brain interface and we should really just stick to touchscreen/typing interaction for now.
Still I would guess Meta Glasses or AirPods should be better to handle such whispered mode since microphones are so much closer. Would be interesting if Airpods had some contact mic that could pickup whispered sound inside your mouth.
Maybe the holly grail is to have something inside your mouth so you don't have to even make voice - device will figure out what you want to speak from how mouth and tongue movement - smart tooth braces anyone? :)
Talking to my cuff isn't going to make this better