Hacker News Reader

Comment by snek_case | Hacker News Reader

I think Google just had an "oh shit" moment as it dawned onto everyone that we now (almost) have the technology to build personal assistants that actually work, and this could be the new tech that disrupts Google's business.

But now that the dust is settling, people are realizing that this technology is not flawless. We haven't fully 100% mastered it, there are unsolved problems, and maybe it's not quite ready for a broad deployment just yet.

That's normal. It's just the way the hype cycle works. People (and the media) quickly get really excited and build unrealistic expectations, then they become aware of the flaws and limitations. That doesn't mean these kinds of systems can't be better and even more useful in 1-2 years. There's a lot of really smart people working on them with many possible paths to address the current limitations.

View on HN

>we now (almost) have the technology to build personal assistants that actually work

Siri doesn't need AI to be useful - it really just needs this $2t company to invest in some basic usability around a set of prompts that people could actually use.

Yesterday's ridiculous Siri failure: "hey siri, set alert volume to 25%" Siri: "media volume set to 25%" That is not bad AI, that is a total failure of UX and it makes it clear that Apple has simply failed to invest resources into making Siri useful.

The one that really boggled me recently:

>hey siri, play Chet Baker Sings on the Apple TV

>playing the best of chet baker sings on Apple TV

>Siri, play the album “chet baker sings” on Apple TV

>playing the best of chet baker sings on Apple TV

Those are not the same album…and only one of them is saved in my library, the one I asked for. It failed repeatedly, too. The only response Siri would produce was to insert two new words to my request and then serve that.

Years later and my Alexa in my bathroom still only successfully starts NPR when I ask it to maybe 80% of the time.

Alexa isn't getting any better at recognizing my voice? At 8am everyday, it hasn't learned I'm likely asking for NPR, not something else? It's not consistent at all.

Controlling Hue lights with it is atrocious too.

Alexa is purely a dumb voice assistant and there is either a complete lack of investment in it, or that money is invested elsewhere like making Alexa more likely to order too much product off Amazon so someone can point to a chart that shows a bug increased sales.

My experience:

> hey siri, play the art of fugue by bach on downstairs

> now playing Love The Way You Lie featuring Rihanna, by Eminem, on downstairs

And like have apple’s ML people just never even heard of Bayes or what? if i have, in the last few days, asked siri to play debussy, ravel, liszt, and chopin, and today I say “play debussy on downstairs” - and i have never, not once, played any kind of rap or hiphop - why would it... “now playing Ain’t No fun - by Snoop Dogg - on downstairs”?

For all we know, this part of the algorithm was written up by a software engineer whose PM didn't think they needed to talk to the data scientists because it was just a voice recognition problem and why would you bring in those eggheads who like to talk about uncertainty, they just make things harder.

I really wish I could tell Apple Music: "Unless I explicitly ask for a rap or hiphop song (which, trust me, I almost certainly won't), never ever play one as part of a playlist."

The way that ChatGPT works won't help there. If you tell it to write a story about someone who cannot see, then "see" will be pushed higher into its weights and it'll start talking about the stuff the character can see (if you say the character is "blind" then it'll work better). It is a language model that lacks critical reasoning skills.

In this case, you don't need ML. Apple has a dictionary of music styles and you just need to be able to exclude certain styles.

Have you actually tried this, or are you just assuming that ChatGPT is just a hyped-up autocomplete, simply doing statisitcal next-token prediction with no internal representation of concepts and relationships?

Prompt: "Write a three-paragraph story about someone who cannot see."

Response:

Maria was born without the ability to see. Growing up, she always felt like she was missing out on something that everyone else took for granted. She would hear her friends describe colors and the beauty of nature, and while she could imagine what they were talking about, she knew she could never truly experience it for herself. Despite this, Maria never let her blindness hold her back. She learned to read Braille and used her heightened sense of hearing and touch to navigate the world around her. She even took up piano and became an accomplished musician, playing by ear.

One day, Maria met a man named Alex who was fascinated by her ability to navigate the world without sight. He would often accompany her on walks and describe the beauty of the world around them, pointing out flowers and trees and the changing colors of the sky. Maria found herself falling in love with Alex, and although she knew that their lives would be different because of her blindness, she was willing to take the risk. Alex was patient and understanding, and he took the time to learn how to communicate with Maria in a way that she could understand. He even started taking piano lessons so that he could play duets with her.

Years went by, and Maria and Alex built a life together filled with love and happiness. While Maria never regained her sight, she realized that she didn't need it to experience the beauty of the world. Through her other senses and the love of those around her, she had everything she needed to live a full and meaningful life.

Interesting! I'm not familiar with chatbots, but this kind of does seem like a sort of autocomplete in the sense that this is a fulfillment of the request in the vein of the lowest common denominator storytelling. While this story is not about what Maria can 'see', it is very much about the sighted world or (imho, problematically) about the lack of sight as a detriment. It's a feel-good story that sounds nice to most sighted people who think that a story about someone who can't see and a story simply about the trait of not seeing are the same.

To illustrate what I mean, do you think it's likely/possible ChatGPT ever comes up with a story about a person who goes to the grocery store, examining produce for ripeness, chatting with another parent about parenting, then going home and feeding their child? None of those things require sight, so it would fit the prompt as long as it portray the person as being able to see.

Write more than three paragraphs and see if it makes it consistent or if it starts to lose the plot (literally).

Throw more details in there and put it in situations where it has to actually make the blind person navigate the world and see if it starts to violate the constraint or not.

The parent already proved the point perfectly. Adding more arbitrary requirements after that is stretching it a bit too far.

No that's entirely the point, when you push it then it starts to fail. Three paragraphs and no other story elements is way too short.

It seems to me those project we intentionally stopped. They could have improved 10X in the last few years but didn’t move a bit.

It seems to be more about willingness than capability.

Now that ChatGPT is there there is no choice other than to expose what they have of become irrelevant.

And i thought i was the only one. Siri is actually their only major product not worth carrying the brand "Apple" if we wanna be brutally honest

Lucky you! Half the time I say in my car "Hey Siri, navigate to XYZ", it says in its long-winded way, "XYZ is 2.5 miles from your house and gets 3.5 stars on yelp. Is that the one you want?" Then I say "yes." Siri responds "Navigating to XYZ."

Then Siri then just forgets about it and doesn't do anything.

Agree 100% and Amazon has done a similarly bad job with Alexa.

I have five echo devices through my house and have only found about ten uses for them.

They’re useful but 1) finding uses for them and 2) twisting your words around to make it understand you is an exercise in frustration.

It seems like it would be an easy data mining exercise for these companies:

If someone asks for something and stops, don’t flag it. If someone asks for something 3+ times in a row, flag it to figure out why they didn’t get what they wanted the first time. If someone says, “Alexa you’re an idiot” don’t just play a snarky comeback. Flag that for serious review. Then iterate.

There seems to be almost no useful iteration on these virtual assistants.

Have you tried chatgpt?

It's tremendously coherent.

The chance that chatgpt just hears your example and than does the right thing is nearly a no brainer.

It already understood much more complicated prompts from my tests with ease.

Siri and other agents struggle tremendously and the implication after such a long time has to be a reflection of either a tremendous Missmanagement or that those classical approaches are just too hard.

Or that the chatgpt people are genius.

But that would be even worse: it would show how much impact chatgpt and co will have sooner than later.

Coherent is nice, but correct would be better.

It's mostly correct most of the time. That's pretty good.

It’s about as trustworthy as the average person. You’d want to verify any fact it fed you.

Of course you want to do that with people too. Shouldn’t be too hard to bake in a self check module.

It's correct often enough already to be useful.

And it doesn't need to be more correct to disrupt industries already it just needs to be more correct than humans.

The most crazy thing is that it's already so good and coherent that it makes totally sense to train a ml model like chatgpt instead of training humans! Because it scales.

We never had this.

And.i already mentor a few junior people I don't mind training an ai instead.

I struggle to believe we can train a LLM the same was as a junior eng, mostly in that I’ve tried to keep a LLM as a journal of sorts and it forgets things way more than a junior does.

We are not limited to LLM.

And just an hour ago I asked chatgpt how to do something specific with docusaurus (Facebook static page generator) and it just told me the answer.

You know how often I explain things in my current position?

How slow some developers are? How often they forget things I told and explained them? How often they still get things wrong?

It's a slow process and doesn't scale very well at all.

If the table turns and we all teach one system instead ooohh boy.

It will make experts faster and better and potentially removes a certain amount of people in every industry faster than we can imagine.

You know the people who are adding some value but not that really but it's still better to have them than not having them?

Cloud probably got rid of plenty of basic sysadmins.

These new systems break through tasks were no one had an idea how we will break through.

This is the same dangerous fallacy of thinking that causes "self-driving" cars to kill people.

If it is "correct enough" to lull people into a false sense of security, the inevitable failures will be worse than if it people remained on guard.

The particular way in which these LLMs fabricate information makes them incorrect enough to be dangerous, not "correct enough to be useful."

People keep mistaking text completion for intelligence and understanding.

Humans are not perfect either at driving, programming or writing.

At some point it is a question of rate of error. The ML model will keep improving as the net get bigger and the dataset larger.

But I agree they are good at having a very good surface understanding while having little of dept at the moment.

Ex: a human making a mistake of visual interpretation while driving will examine in his mind the error he made and thing of the consequences in a more dangerous situation. The NN will not bother (at the moment)

Sigh. The models do not "have a very good surface understanding," they are capable of competing strings of words in a way that we interpret as a surface understanding. I believe time will show this is a category error, not just an error of degree.

In much the same way that Teslas are still literally killing people when they encounter vehicles parked across roadways in 2023, just like they did back in 2016, the difference between the appearance of understanding and actual understanding is a much wider chasm than it first appears.

The LLM will not "consider" anything. Even transcripts we've all seen in which LLMs seem to acknowledge errors (rather than doubling down and inventing false sources) are still just text-prediction, modeling what it would be like if someone acknowledged an error.

When a particular LLM seems to take the corrections offered in a widely-publicized transcript on board and not repeat them in future transcripts, that's because someone acted to modify the model, adding filters or weights to ensure that the text prediction path goes differently in the future.

There is a YouTube video from someone talking to chatgpt to play SQL server.

It can reproduce the right output of 3 insert statements with a select *.

It can interpretate an SQL procedure too.

Whatever the architecture is behind this, is slightly more than just text prediction.

I don't think 'text prediction ' does this type of state management justice.

But hey let's see. Perhaps this is just an emerging feature from a tremendously huge text/language model.

Text prediction is intelligence.

To be able to code a program form a short description, combine different elements to make a solution, understanding is required.

The understanding is in the weights and pattern detection capabilities.

Sure, some part are missing, like the capacity we have to hold a few thought in our mind, correct ourselves, remember the mistakes we made, etc.

The same is true of NN that can differentiate cats and dogs, they detect patterns is a similar ways that humans do.

In the openAI api you can see the level of confidence the NN has on each words it output, it’s just not exposed on the chat interface.

I think the larger missings parts are by design, to give the system memories, agency, access to the internet, to programming would be careless at this point, but it’s happening anyway.

I feel it missing our capacity to examine our thought, to replay events in our head to reprogram itself when the results are not optimal.

> The chance that chatgpt just hears your example and than does the right thing is nearly a no brainer.

ChatGPT can't do anything. Someone has to, somehow, wire it up to actual physical or digital controls. Intent recognition and slot filling still have to happen somewhere, and that means defining discrete intents and slots.

GPT may be better at mapping from speech to intents, but it can't magically interface with APIs that haven't been defined.

EDIT: To elaborate a bit, the problem in OP's story isn't caused by a bad language model, it's caused by no one at Apple thinking to define different "alert volume" and "media volume" intents. Current language models are plenty good enough to recognize the distinction, so simply adding ChatGPT won't be enough to make any feature work unless someone at Apple predicts the need and writes the interface.

If I tell my computer to "froblate" it says "command not found" not "command shutting down".

If I tell it to "shtudown", it said "did you mean 'shutdown'?

There is an AI failure in distinguish "mispronounced command I know" from "command I don't know".

I'm not convinced that ChatGPT would do better than existing models in that regard. Using it for intent recognition is already a bit of a square peg round hole situation.

Basic intent recognition models are trained to produce a single neuron per intent as the output, which makes it pretty easy to use the activation levels of the output to decide whether to perform an action, confirm an action, or ask for clarification. You just need to check if the certainty is below a certain threshold.

With ChatGPT you'd have to encode the intents as text of some kind (JSON?) and hope that it doesn't just hallucinate an intent that your APIs don't have when it's faced with ambiguous input. You could probably have hallucinated intents map to a decent-sounding error message, but that feels more brittle to me than the existing approaches.

It can interface with apis it only read about.

Ask it to write an SQL query and it will.

Give it upfront context and it will be able to produce API commands.

The voice recognition can trigger chatgpt API 'create API command for the following text's.

The practical problem is cost and hardware.

But this is closer than we ever were.

Having ChatGPT write code that you immediately turn around and execute is a very risky proposition for all kinds of reasons, the least of which are crashing at runtime or failing to compile. If you're having it produce code in a Turing-complete language, you've opened yourself up to the worst kind of prompt injection attacks.

A much safer approach would be to have it produce a JSON object that is interpreted by the wrapping code. This allows you to inspect the interpreting code and provide guarantees about what can and cannot be done by users.

But neither execution model solves my main point: someone still has to write the interpreter (or the library functions if you go the dangerous route). Someone has to think ahead and guess what the user is going to try to do and provide APIs that allow that. And that work is most of the work: our existing intent recognition models are actually not bad at all at their job, the failings these voice assistants have are almost all to do with predicting the ways in which users will try to interact with the system.

Have you tried it out?

Tried what out? Code generation and execution? Or building a full-featured voice assistant?

The code generation and execution is trivially problematic. I don't need to try it out to know that if someone can coerce one of these bots to spit out its prompt, someone could coerce it to execute dangerous code if it's wired up to an interpreter.

As for the other part of my analysis, in order to empirically test it I would have to build a full-featured voice AI, release it to millions of users, and see if I was able to predict everything they tried to do. Given that my prediction is that even Apple can't do that well, I'm not sure why you think I would bother to try it out.

If an engineer told you that a bridge would collapse if built a certain way, you wouldn't insist on trying to build it anyway just to be sure. Most of the time in engineering you can't run a full-scale test, you have to make do with analyses. If you want to critique my analysis, I'm all ears, but "you haven't actually tested it" is not a critique.

I meant chatgpt

Yes. I've used it quite a bit. It's impressive, but the hype is excessive.

All of my analysis is based on my usage of ChatGPT and my understanding of the underlying model. The hype is largely driven by people who don't understand how it actually works and think they're interacting with an artificial general intelligence.

It's a very impressive language model with a lot of applications, but with many fewer applications than the hype would suggest.

I'm hyped not because I think it's an agi.

I'm hyped because I threw normal text (sentences) against it and it always did what I wanted and it didn't matter if it was bad English or bad German.

I didn't even try promt hacking or promt tuning because natural language already worked so good.

It's a tremendous good 'normal human' text interface.

I don’t think Apple has “simply failed to invest resources into making Siri useful”, but rather, voice assistants pose major UX problems that nobody knows how to solve at scale.

We have tons of patterns for CLI and GUI usability. But in both cases, we have a easy enough way for users to discover what is possible—in the CLI, you can present a list of commands or categories for commands. In the GUI, we have menus, dialogs, and windows that let you explore. Voice is a much more difficult problem in the first place. Voice commands are often modeless, discovery is difficult, and commands are given with troublesome natural language patterns which must somehow be adapted to multiple languages.

Take a look at all the different voice assistants—Siri, Alexa, and Google. They all kinda suck, despite enormous amounts of money being poured in. From recent news, we know that Alexa managed a loss close to $10 billion in a single year. By all signs, it contains some brilliant engineering, but brilliant engineering doesn’t matter if it doesn’t solve the underlying (difficult) problems well enough.

(I don’t think tools like ChatGPT are going to push the needle much here, at least for a couple years, because integration with all the different software systems takes such a massive amount of coordinated work.)

In my experience, voice assistants go through a cycle something like:

1. The novelty of the thing!

2. Start using for some simple stuff like weather, timer, alarm, etc.

3. Try more complicated things like skills, really managing controls/media in the car, reliably playing some album.. and largely give up because it's frustrating.

4. Mostly go back to 2.

>voice assistants pose major UX problems that nobody knows how to solve at scale.

My example of a Siri fail is not at all indicative of a "major UX problem. The fact that Siri doesn't understand iOS nomenclature and that iOS doesn't have the proper hooks to do something that a user would obviously want to do, is indicative of a total failure to invest resources in making Siri useful.

This (incredibly irritating) behavior isn’t an engineering failure, it’s a specific policy failure.

The idea is “we will not give you a way to make our conversional UI incomprehensibly quiet”.

It is an especially dumb policy decision on anything that’s not a HomePod, and it’s also dumb to willfully ignore the user’s intent and focus on the one thing the device is willing to do for you regarding volume, rather than explaining “sorry, I’m always going to be loud, my makers are worried you’ll complain I’m inaudible if you try to get me to stop yelling my responses” but it is deliberate.

So it's a UX failure, as parent wrote.

Adding a large language model fine-tuned to translate English commands into a limited set of structured actions for Siri would drastically improve the Siri experience.

>That doesn't mean these kinds of systems can't be better and even more useful in 1-2 years.

It's also faulty to assume that improvements will happen linearly. We had a self-driving car mania in 2013-2017 with billions poured into it and people just forgot about it.

Also consider that if easy linear improvements were possible, they would've delayed launching it. AI (software in general?) seems to be the kind of thing where once R&D hits the exponential difficulty wall, they make it generally available.

> The first 90 percent of the code accounts for the first 90 percent of the development time. The remaining 10 percent of the code accounts for the other 90 percent of the development time. - Tom Cargill, Bell Labs

Self driving cars have a failure downside that is very high for basically all use cases.

On improvements— one of the things that has impressed is how much Chat GPT has improved since it was released. I don’t think it’s going to be a perfect AI in two years. But it already is working better for some use cases for me.

It feels slow to us but from past performance we still progress on this topic and have achieved already impressive results.

And more and more puzzle peaces are falling in place.

We can see were it is going.

And when you follow Nvidia with their digital twin topic, we are converging.

Not so much. Making it available allows OpenAI to evaluate and iterate on the chatbot layer, which has had much less data and engineering invested.

You equate lack of ubiquitous self driving cars with lack of progress in self-driving technology, but even if we've had super-linear progress in terms of self-driving "driving skill," if that skill is below the threshold where the sales from pushing the software are less than the potential legal liability, companies aren't gonna push it.

Just like driving cars were "just around the corner", I highly doubt 1-2 years will fix the issues, that "last x%" is exponentially more difficult.

I don't think it's necessarily as hard as you think. Can't do math? Teach the model to use an external mathematical expression evaluator to do math. Makes up inaccurate information? Teach it to query an external knowledge base. How hard to implement do you think that is? They're already working on it.

> How hard to implement do you think that is?

Very, very, very, very hard.

It's not that hard:

https://github.com/williamcotton/empirical-philosophy/blob/m...

@williamcotton - saw this the other day in the prompt engineering post - I enjoyed it. Sent you a message on your site but something looks broken. If you use linkedin I requested to connect.

Doh, I’ll look into that contact form! I’ll check my LinkedIn now. I’m glad you found it interesting!

I think there is a greater tolerance for error here though.

Cars haven't made it yet because the assumption about allowable errors was wrong. Turns out it isn't "good enough" for self-driving cars to be safer than humans.

Alphabet probably has more IP on ML than any group on the planet. There's the Deepmind and AlphaGo, AlphaZero, and AlphaFold but much more practically I can type a sentence into Chrome and get a page for of useful and general correct lines that provide what I want. It would be trivial for Google to summarize the results in some sort of written form instead of a list of links to other places. They could have done that a decade ago.

The current state of the first 10 results on a google search is quite sorry. It's 5 irrelevant ads and 5 SEO-autogenerated sites (probably using GPT3 in the first place). Try it with any kind of IT-problem or IT-product review question. No amount of AI is going to summarize that load of crap into anything better. This is googles problem. The underlying abstraction of knowledge is going to change - knowledge has to be built and served while severely filtering away "noise" up to 90%. Maybe their traditional algos can do this, maybe it requires GPT+++, but it is broken as it is now and has been for more than a year.

I think you're confusing wanting someone to do something for you for free and for billions of people with something that can give you good enough 80% of the time and also make money while doing it.

"But now that the dust is settling, people are realizing that this technology is not flawless."

Of course it's not flawless, Google isn't and people aren't... and those are the two targets for this tech. People lie, make mistakes and mislead and Google directs to plenty of problematic content.

The thing to watch isn't the position of GPT or any disruptive technology today, but its trajectory. The trajectory of GPT is explosive growth, and that growth will be disruptive to imperfect human jobs and imperfect search engines. GPT does not have to be perfect to be disruptive.

This is especially threatening to Google because they have an organization that has never had to compete, like pandas they just consume. They don't hunt. Look at their vast graveyard of platforms they couldn't sustain even with their monopolistic power and endless money.

Consumers and ad purchasers want competition in search because like Uber and Lyft demonstrate, when you have two technology products competing head to head the margins get compressed toward zero.

Edit: spelling

> I think Google just had an "oh shit" moment as it dawned onto everyone that we now (almost) have the technology to build personal assistants that actually work, and this could be the new tech that disrupts Google's business.

The bigger problem for Google is that we've got AI which can produce millions of times more shitty blog articles to flood the internet with, that their SEO algorithms can't keep up with in order to surface anything actually relevant to humans.

Also the problem for all social media and reddit as it becomes easier to write bots that you can't casually distinguish from humans (given how many humans on reddit read like bots that are desperate for attention).

You're making the mistake of assuming that the primary purpose of these systems is to help you accomplish something you want. These systems are to capture a commodity to resell; you, and all the other users that engage with the system.

It could be great, but until an assistant of this sort runs on your hardware, and works only for you (really hard to show), it won't be your assistant. It will be some corporation's Carnival Crier.

> we now (almost) have the technology to build personal assistants that actually work

That parenthetical "almost" is doing an awful lot of heavy lifting.

Reply on news.ycombinator.com