I have been wondering if a next gen “GTP-4” like language model could take these AI assistants from being jukeboxes to actual assistants.
[voice input] -> [action]
[input 1] -> [ask for clarification] -> [input 2] -> [propose action, ask for confirmation ] -> [action]
It's not so much that they need any specific feature ("buy toothpaste") that they need some way to recover the cost, for instance being more tied to the Amazon ecosystem, more likely to subscribe to Prime, etc.
It's not easy though. With roughly 70 million users they would need to generate about $150 a user in profit to cover $10 billion. At 2% margin they'd need to sell $7500 worth of stuff that they wouldn't otherwise sell.
What I don't get is where the $10 billion went: how much of it subsidizing the device? (If they sold 10 million devices last year it would be a $1000 subsidy which is incredible.) are they paying AWS prices to run the backend service? how much are they spending in software development, model training, etc?
Don’t forget that between all those steps there’s a “store everything you say in order to sell you stuff later” step
It's like every lesson about not disrupting GUI flow with modal banners, and about creating shortcuts and other accelerated paths for advanced users was forgotten in VUI. At best you get a one-size-fits-all brief mode that usually isn't very brief, and doesn't reward learning so doesn't increase engagement.
VUI vendors also plainly have no clue how to enable discovery without either fucking up your home screen with some ugly "things you can do" list or being annoying about volunteering stuff when responding to your commands. It's as if they might need to *gasp* document their features instead because voice devices aren't web apps you can just fiddle with.
The net result is I actually get way more frustrated with these devices now that I'm super-experienced with the tech, not less. That's the wrong direction.
Sometimes it seems like this is the style manual for software today
https://tvtropes.org/pmwiki/pmwiki.php/Funny/Idiocracy
That anti-pattern of interrupting you to tell you about a new feature is common and it's the worst. All the time I feel like I am behind the 8-ball and I really need to finish some small task right now but Firefox, Photoshop and seemingly every other product wants to get in my way of doing the task.
It's a deep problem though matching the expectations of a user for a VUI with what it can so, Alexa's greatest triumph is that it had a domain that it seemed pretty complete over and that mostly people could "get" what it was capable of doing.
They only ever seem to be useful if you do not have free hands (e.g. cooking) which is why they are almost exclusively used as timers (at least in my household). I have tried getting it to read a recipe but that's just cumbersome (I often want to have a quick glance at ingredients, not have someone list all of them in slow succession).
Other actions such as "play stranger things on netflix" seem like useful actions at first, but again only really make sense if you are truly busy with your hands (which you would not be if you are on the couch with the remote near-by). Using the remote to turn on the tv and play something on netflix is often not more than two or three button presses (assuming the show is in your 'continue watching' list, which it usually is if it's something you specifically want to watch).
And turning on lights? Again, a novelty. Unless your house is designed in a weird way where your light switches are on opposite ends of the room you are walking into, it's always faster to just flip the switch when walking into the room than shouting out and waiting for your assistant to do it for you (never mind the fact that shouting to turn on the lights during a bathroom run at night would be highly annoying to other household members).
I also like being able to sequence the set of several lighting, audio, and video operations that sets my theater room up for various uses. I like being able to quickly check the house temp and tweak the thermostat from the other side of the house. And I like being able to turn whole rooms on and off at once, or to set their light dimness and color for my use of the moment without getting up or fiddling with my phone, etc.
I especially like the routine that does a sunrise wake on my bedroom lights; turns off my brown noise machine a minute before my alarm since sometimes that will peacefully wake me without having to hear the beep; chatters jokes, news, etc, incessantly at me in the morning once I dismiss my alarm to make sure I actually get up; then automatically shifts into whole-house Sirius XM to play hard rock as I start my day.
And say what you want, I like being able to ask the thing over my shoulder what my office fan 5 feet behind me is set to and change that while coding and still looking at the screen, rather than stopping, screwing around with a cheap remote control and trying to see a little LCD from across the room. Like I said, I'm ADHD. If I stop to do that, who knows when I get back on task again.
I agree for simple "turn on a switch" tasks they're way overkill, but for actual automation (and perhaps accommodation) these devices are useful, especially once you start replacing ALL your environmental devices with controllable ones. It's more a question as to whether you can leverage the automation.
I have not tried Home Assistant, no, but I'm keeping an eye on what happens with Matter. If I were able to consolidate the interaction to a neutral third party that could control the other stuff I already have, I'd love to do that.
[0] https://twitter.com/hwchase17/status/1589648226449514496