Hacker News Reader

Comment by modeless | Hacker News Reader

I don't use any feature of Google Assistant that wasn't present in the original Android voice search feature over a decade ago. And the few features I do use haven't improved in any way that I've noticed. In fact I feel like reliability has regressed. Google could have fired almost the whole team years ago and my user experience would not be different today.

I consider voice assistants to be a failed experiment. It's clear that they were doomed to fail without the true ability to understand natural language and perform common sense reasoning that is only now starting to become a possibility with large language models.

View on HN

I pretty strongly disagree. First, from an accuracy perspective, Google Assistant has gotten much better. It almost never misunderstands me, even when I'm mumbling and I'm surprised it can understand me.

I do agree that the set of actions I use Google assistant for is pretty static and are the same ones everyone else uses it for: What's the weather? Set my alarm? What's the first item on my calendar? Play XYZ on Spotify... But I don't see this at all as a "failed experiment". On the contrary, I see it as a successful experiment, at least from a human utility perspective: I ask it to do a concrete task, and it does it.

The reason it's failed from a larger business perspective is because the format is less amenable to big tech sucking any more of our money or mind out of it. But I say Amen to that! Wherever I go I feel like tech interfaces get relentlessly shittier as they just try to hook us into more mindless scrolling (this HN post on the topic is one of my favorites of all time: https://news.ycombinator.com/item?id=37509507) That's just fundamentally harder to do with a voice assistant because nobody has yet figured out how to plug in voice "algorithms" in a way that would make the device control you like they do with YouTube, TikTok, Facebook, etc.

If anything, I think the rest of tech that gets you to do things you really don't want to do should be the thing that's considered a "failed experiment".

I agree that its transcribing accuracy is better (largely thanks to general progress in neural nets rather than anything Assistant did as a product). However transcription accuracy was already surprisingly good many years ago, and in my experience any improvement since has been swamped by actually reduced reliability in understanding intent and performing actions. Ultimately it doesn't matter if it misheard me or got the words correct but misinterpreted the meaning or simply failed due to internal errors. The user experience is similarly bad in all these cases.

Also I don't feel that hotword detection has improved much, and hotword issues continue to be a big pain point for me. I'm sure the team has a dashboard somewhere that shows consistent gains in their hotword metrics, but as a user I don't feel it at all.

The reason I say it's a failed experiment is that the original ambition was to build the Star Trek computer (explicitly, in Google's case: the project was code named "Majel"), and it hasn't worked out. There may be a couple of features that are somewhat convenient (which, again, largely worked fine already a decade ago) but they don't come close to justifying the level of resources spent on these projects over the last decade.

One difference I have noticed is that it used to be when I said "five minute timer" it would maybe ten percent of the time hear "five minutes timer" and decide it should search youtube for matching videos. Nowadays, it seems to treat "five minutes timer" and "five minute timer" as equal, but I get more silent misses where it ignores everything I said.

On the other hand, I gave up some kinds of requests. For instance, I used to try "start directions home" when I was driving and it would say "here's a route I found, do you want to start (or some such)" and there would be no way to actually start the directions home without breaking the very expensive law against touching your phone while driving. Does that work today? I don't know, because I've mostly been driving with Android Auto in the last few years and my wife's been in the car on the rare occasion I haven't. But it seems like the Assistant team could straightforwardly improve it.

> On the other hand, I gave up some kinds of requests.

Those kinds of things seem to work absolutely flawlessly for me lately. I had similarly given up on those requests. Heck I had given up on setting timers since that just plain stopped working. A month ago or so I tried changing navigation by voice while driving and to my surprise it worked fine. Since then I've been picking up my use of the assistant more and have yet to have a miss. My main gripe is the trigger word being kinda iffy at times, but I suspect that's due to me still rocking the original pixel. As I understand it the trigger word stuff runs locally so is dependent on your hardware being good.

> I do agree that the set of actions I use Google assistant for is pretty static and are the same ones everyone else uses it for: What's the weather? Set my alarm? What's the first item on my calendar? Play XYZ on Spotify... But I don't see this at all as a "failed experiment"

So since 10 years, all it can do is play some music and tell the weather, all of this only online, so it is not a failed experiment. /s

Yea!

A product that has become pervasive in most households in America, that does it's one task, and tries to do it well. The problem is that companies built out these organizations and marketed these products as the next step of computing after mobile, and it's just not that easy to monetize.

My washer/drier, "all it can do" is spin clothes around and apply heat. I paid a lot more money for that "failed experiment".

I think Apple really should have made the HomePod earlier, and all this could be avoided. They were heavily criticized for not supporting 3P "skills" when it came out. Today, Apple is one of the better sellers, and has properly managed their products expectation from the start.

Why involve voice at all? Sounds like it needs about 4 buttons.

Sounds like all we needed were really fast horses too.

Except they didn't invent a car. They invented a complex device to drive the existing horses.

Well, this particular tech hasn't panned out yet, I agree. But with that outlook ("just use buttons"), you'd never make any changes, which is even worse.

> A product that has become pervasive in most households in America

You think Google home has over 50% penetration?

Assistant is included on most Android devices nowadays.

Assuming smartphones in general have 90%+ market penetration, 42% market share for Android, and an average household size of 3, it seems reasonable to assume that 50%+ of households will have at least one device running Assistant.

You think over 100% of people use Google Assistant on their Android smartphone?

I agree with the sentiment, but i don’t think you can go from 42% of the population to >50% of households. Especially because households probably cluster, so it’s probably strictly less than 42% of households use android.

Between GAssistant, Alexa and Siri Speakers I bet it’s a significant chunk of the population.

Sonos now have a voice assistant (far more limited though) and many cars are rolling out some sort of assistant too.

I know plenty of people with multiple speakers and from competing brands.

Oh and obviously every iPhone in the last decade has it built in. That’s 50% of the population right there. Then there’s android phones, where I don’t know if they all have it wakeword enabled, but it’s probably another big chunk too.

The thread is about Google Assistant, not Alexa, Sonos, or Siri.

Regardless, do you think over 50% of households use the voice commands to the extent we call it "pervasive"?

You might have enjoyed the reaction from the normies at a party recently when they figured out that my family doesn't have any voice control anything in our house and they started grilling my (comically, for the conversation, nontechnical) wife as to why.

After much very polite dancing around the politics of "listen, it's just a personal choice and nobody here is judging anyone else, I just don't want my children playing arbitrary stupid music and buying things on my credit card", she finally capitulated and pointed out that "well it's listening, isn't it? my husband" (I was actively playing with children at the time), "would have better details, but if it's listening, that means it's recording and sending those recordings to Amazon or Google or whomever".

At which point the lady of the house got up and turned her Alexa off. I attempted to not die laughing in the corner with the children.

> well it's listening, isn't it?

It's listening, and so are contractors and employees.

https://apnews.com/article/9000fd56f68043e48a2de8a08997a1fd

https://www.theverge.com/2019/8/7/20758491/microsoft-voice-r...

https://www.theguardian.com/technology/2019/jul/26/apple-con...

Companies will pinky promise that it doesn't happen, and say all kinds of things about their security polices and access controls, but spying on people's homes is basically the entire point of these devices. What else would you expect when you bug your home with an internet connected microphone from a company who makes their money collecting as much information about people as they can, especially their most private data.

They're also collecting data on children, and even targeting kids. Companies like Microsoft and Amazon have already been caught and got nothing more than slaps on the wrist, but collecting kid's data is basically inevitable when the house is bugged and kids are intentionally playing with it.

Google assistant is awful via the nest mini, I'd hard disagree that the accuracy has gotten better.

I ask it for the time and it quotes some article on wikipedia about what time is or various movies with time in the name.

I ask it to turn of the lights and it tells me 'power control is not supported yet' at least half the time.

It's gotten worse since I bought it, it never did this and that is with even the most basic of tasks.

I would counter that voice assistants are not a failed experiment. I would suggest that Amazon and Google’s execution of said experiments have failed.

I am extremely excited about the potential of projects like the Rabbit R1 and Humane.

I agree, I just think these new projects based on LLMs should be considered a different category, AI agents or something. The traditional grammar-based voice assistant architecture used by Google Assistant, Siri, Alexa, Bixby, etc is an abject failure, and not for lack of trying.

I think many parts of the architecture can be reused - in Alexa terms, all of the “skills” that integrate the assistant with various other services. IMO one of the main problems with assistants is that I don’t know what skills are available or how to invoke them. It’s like I’m a wizard who has to memorize all the spells I could be casting. It never happens because I don’t care enough. I think LLM’s could potentially help my making it easier to discover and invoke those skills.

"Skills" will be obsolete very soon. AI agents will use the same software tools and services that humans do. They won't need special separate AI-only interfaces.

I'm not excited about the Rabbit R1 as a hardware device but their software vision is exactly right and there are new startups coming out of stealth seemingly every day now attacking this problem.

Skills are just APIs that conform to a similar look. We'll definitely continue to have AI-only or developed-for-AI APIs for future "agents" to act against. They probably won't spend much effort formatting text to sound good to a person, but the infrastructure is here.

I disagree. These special APIs will not have the breadth of capabilities that the human UI does, so AIs will use the human UIs out of necessity. But I think in the long term we will eventually see a simplification of UIs. As it becomes less common for humans to actually use them, they will no longer need fancy animations or dark mode or client-side validation or pretty styling. In the extreme, a return to plain HTML forms that a human can use in a pinch but are mostly used by AI agents. At that point I guess you're blurring the lines between UI and API.

Isn't it the exact opposite? Interfaces we use every day can be dead simple, all they need is that they don't change behind our back. The accelerator pedal does not come with a footover pop-up "keep pressed to make car go". Interfaces we use once in a leap year on the other hand, that's where we need all the hand-holding we can get.

This "spells" is such a great way to explain how it feels to me to use these assistants. I'll play with one if I'm at a friend's house, but honestly can't see the appeal. Telling Google to change the color of the lighting or brightness just seems like something that is mostly a gimmick unless you're maybe disabled and then it may be a big quality of life improvement. The other stuff doubly so.

With ChatGPT I can see the appeal for certain tasks like having it create a custom text adventure for you, but I can't see it being too useful in my day to day life yet.

If the Alexa/Echo-category of devices are meant to be some widely flexible and pervasively useful device from Star Trek, they're a failure.

If they're meant to be a usable $30 kitchen timer and music player, they're pretty great.

Yup, as far as something that can turn on a light, run a timer, convert some units, tell me the weather, and have an -okay- shot at some categories of random questions instead of me getting out my computer-- the google assistant is just fine.

They do succeed in that way many times a day every day in my house

As '80s telescreens they were fantastic.

Google Voice Search used internal codename "Majel" in reference to Majel Barrett, who played the voice of the Star Trek computer. That was explicitly the ambition. It just didn't work out.

OK. I believe you. They accidentally created something else that was very useful but different from what they set out to do.

Is that a story of failure or of success?

Starbucks launched to sell beans and espresso machines. YouTube launched as a video dating site. Are they also failures?

I guess where we disagree is "very useful". If Google Assistant stopped working tomorrow I would hardly care at all. There are a couple of scenarios where it's slightly more convenient than using my phone (assuming I don't encounter one of its many failure modes) and that's about it. I'm sure the hands free aspect is important for certain people in certain situations but I think the vast majority of people just don't see a lot of value from it.

Amazon alone has sold over half-a-billion Alexa-enabled units (around 10 of them to me).

I think people see more than $30 of value in them, at least as their revealed preferences suggest.

Those sales were subsidized in expectation of future profitability that will never come (at least not without a ground-up redesign of the product around LLM-based AI agents or some other paradigm). Economically Alexa is a "colossal failure": https://arstechnica.com/gadgets/2022/11/amazon-alexa-is-a-co...

Yeah, my crock pot and ceiling fan are "Alexa-enabled". That's two.

But how often did that computer voice-acted by her really do significantly more than "set timer to thirty minutes"? Outside of some broken plots on the original series ("we have insufficient data to know the truth, let's ask the computer who will tell us anyways!"), it really was mostly mundane voice assistant stuff.

(I'm deliberately excluding the "ten words to 'author' a holodeck scene" part, that had always been stretching my imagination a little too far, more "this can't work!" than space travel and transporter beams. Then stable diffusion happened)

There were some scenes where Riker on the bridge asked the computer essentially a SQL query: "give me a list of star systems with parameters that fit X and cross-reference by Y..." "There are 3 systems which fit your query: ..."

That might have been the initial hope of the team, before Google killed it. It's been on the graveyard for years with zero updates, my google assistance nest mini is arguably worse than when I bought it.

I believe it is possible to have made it better, but they didn't try, they just gave up, like much of google products.

I think way too much money has been sunk into those projects if their ambition is just to be a $30 kitchen timer and music player.

Things don't seem to be going well at Humane, either: https://www.fastcompany.com/91008117/ai-pin-humane-layoffs

No horse in the race, but the layoff is only ten people from a 250 person company that is still hiring. I'm a little surprised it even merited an article.

Not sure if any comment that holds Humane in high regard can be taken seriously though.

It’s interesting. I had not heard the latest. Their initial videos looked promising.

I ordered a rabbit r1. I can see it’s missing some key functionality (Bluetooth for headphones so everyone doesn’t have to hear?) But… I think it’s an example of a first, promising, step to the era of real voice assistants/agents.

> Bluetooth for headphones so everyone doesn’t have to hear?

This is a critical missing feature. I would never ever use this unless it's in my ear alone.

And I hope it doesn't become socially acceptable to be carrying these around forcing everyone else to listen to whatever this device has to tell the user. There is already far too many noise and inconsiderate people in this world.

But how is the input side sufficiently compatible with any state other than being alone? Voice surely does not qualify?

Even gesture control would be a tough sell, and even that only if it's not "AR, where you push buttons projected across your field of vision" but "AR, where you can do the equivalent of gamepad buttons with hand movement anywhere the device can see the hand". Voice output (earbuds) would be far too slow for that kind of interaction, because you can't skim a list. Compared to the strictly sequential nature of audio, screens are the equivalent of embarrassingly parallel.

By the way, that slowness of voice output vs screen is also what I consider the true motivation companies had for creating those essentially free voice assistants: searching for product/service on a screen, even if it's just a small handheld screen, makes you pick from a list. With voice in the other hand, going through the list is so slow and cumbersome that the chances for just picking the first, "I'm feeling lucky", are much, much bigger. The value of placement (bought directly or bought indirectly, "this must be very relevant because we know how much they can spend on our other ad services") is just so much bigger with voice. Chances are people are less likely to listen to the second hit on voice than to go to the second page on screen.

I think your last paragraph contrasts and slightly contradicts itself but I agree with the failure part. If we consider a real-life PA performing tasks on our phone for us the question can be asked if it makes much sense or if it would be an interesting novelty.

Basic functionality has always been the key winner with this tech. Calling, setting reminders etc. nothing too complex.

I just think the new LLM-based agents are in a different category. I don't see them as a continuation of the Siri/Alexa/etc lineage.

I do think a human personal assistant who had unlimited access to my data and could perform any task my phone or desktop could do would be exceedingly useful, even if the only interaction possibility was voice. And I think that AI agents will approach that level of usefulness within a decade or two.

I think that voice assistants could have been much more usable this whole time if they had avoided getting mired in pushing products/music and instead stayed focused on doing a few things well.

Yes, but there's no money in just being usable.

I have multiple Google Home speaker devices at home. My experience that about 1/3 of the time, even in silence, it does not understand simple commands like "Stop" or "Pause". Sometimes I say "Stop" and it interprets this as "Stocks" and starts reading out stock prices. The fact that it only works 2/3 of the time makes it a frustrating UX, I hate trying to "use" it. (First world problems..)

Yep, I’ve got a few speakers and it feels like a crapshoot. I pretty much only say “lights on/off”, “play rain sounds”, and “stop”. But the goog hears _something_ that translates to “playing lights53 on Spotify”, or “here’s what’s playing near you”. Usually in quiet environments too when it mishears me - oddly good at parsing stuff in noisy environments or low volume, if I’m talking (masculine voice) but my roommate (more femme) usually doesn’t get heard at all.

I mean, without those rudimentary building blocks they wouldn't have a lot of data and talent ready to investigate LLM's (and ofc, lots of polish on the NLP framework. Not as flashy, but very important for the UX). I don't see it as a failed experiment so much as a raw prototype if we believe LLM's can realize the true potential. Heck, for all we know, we were actually just training the model for this past decade.

What are you talking about? What "rudimentary building blocks" came as a result of voice assistants? They have nothing at all in common with LLMs. Your post is pure non-sequitur.

A bulk of data that the product trained upon in order to execute a task based upon promoting instead of a rigid command?

They only have nothing in common if you think of a Voice assistant as no smarter than those auto prompt customer support lines. Clearly Siri/Cortana/Google assistant had sole basic context mapping.

Weird to call Google Assistant failed when it's about to get vastly more capable by being wired to Bard/Gemini. Already, I want to talk to Google Assistant like it's an LLM, but it's not there yet.

LLMs at Google aren't going to augment Assistant, they're going to replace it. With something that isn't an assistant, and doesn't do anything useful.

Reply on news.ycombinator.com