"Hey Siri, I had a meeting last summer in New York about project X, could you bring up all relevant documents and give me a brief summary of what we discussed and decisions we made. Oh and while you're at it, we ate at an awesome restaurant that evening, can you book a table for me for our meeting next week."
Just last night, we were entertaining our toddler with animal sounds. It worked with “Hey Siri, what does a goat sound like?”, then we were able to do horse, cow, sheep, boar, and it somehow got tripped up on pig, for which it responded with the Wikipedia entry and told us to look at the phone for more info.
"Hey Siri, whats the weather?" and "Hey Siri, what the X-day forecast". Everything else is a huge mess.
But you know what? Still better than Alexa for managing my smart home stuff. By miles and miles, IMO.
I have thousands of contacts, lots of photos, videos, and emails, all in Apple’s first-party apps and yet Siri is more likely to respond with a popular song or listing of news articles that’s only tangentially connected to my request.
This becomes more complicated when Siri is the interface on a homepod in a shared area. Who's data and preferences should be used? Ideally it would recognise different voices and give that person's data priority, but how much can/should be shared between users? Where are these data - they shouldn't be in the homepod, so it would have to task the phone with finding the answer. I'm sure something good could be done here, but it wouldn't be easy.
Doesn’t the locally-run LLM bit solve this particular grievance? I’m picturing a personal AI à la Kim Stanley Robinson’s Mars trilogy.
Well, this is about adding ChatGPT-level smartness to Siri, not just the semi-dumb assistant of yore.
https://www.macstories.net/ios/introducing-s-gpt-a-shortcut-...
Some examples from that blog post:
> I’m feeling nostalgic. Make me a playlist with 25 mellow indie rock songs released between 2000 and 2010 and sort them by release year, from oldest to most recent.
This doesn't just return a list of songs, it will create the playlist for you in Music.
> Check the paragraphs of text in my clipboard for grammar mistakes. Provide a list of mistakes, annotate them, and offer suggestions for fixes.
> Summarize the text in my clipboard
> Go back to the original text and translate it into Italian
I haven't tried it myself, but it has other integrations like "live text" where your phone can pull text out of an image and then could send that to GPT to be summarized.
Version 1.0.2 makes improvements for using it via Siri including on HomePod.
siri: The temperature is currently 54 degrees. Expect clear skies starting in the evening, going down to 52 degrees tonight.
me: hey siri what's the high today
siri: the high today will be 84 degrees
me: hey siri will it rain today?
siri: Expect heavy thunderstorms around noon
Note nothing it said in the original was actually wrong...
Meanwhile, Google and Amazon have decided that the data center costs of their approach just aren't worth it.
>Google Assistant has never made money. The hardware is sold at cost, it doesn't have ads, and nobody pays a monthly fee to use the Assistant. There's also the significant server cost to process all those voice commands, though some newer devices have moved to on-device processing in a stealthy cost-cutting move. The Assistant's biggest competitor, Amazon Alexa, is in the same boat and loses $10 billion a year.
https://arstechnica.com/gadgets/2023/03/google-assistant-mig...
Both companies made big cuts to the teams running their voice assistant tech.
As someone who dictates more than half of their messages and is an incredibly heavy user of Siri for performing basic tasks I really noticed this sudden decline in quality and it's never got back up there - in fact, iOS 16 really struggles with many basic words. Before iOS 13. I would have been able to dictate these two paragraphs likely without any errors however, I've just had to edit them in five places.
This is the sort of AI I want; a true personal assistant, not a bullshit generator.
That Siri went from useful to far less useful had more to do with the aim to push products at you rather than actually accomplishing the task you set for Siri. If Apple actually delivers an assistant that works locally, doesn't make me the product, and generally makes it easier to accomplish my tasks, then that's a product worth paying for.
When anyone asks "who benefits from 'AI'?" the answer is almost invariably "the people running the AI." Microsoft and OpenAI get more user data, and subscriptions. Google gets another vehicle for attention-injection. But if I run Vicuna or Alpaca (or some eventual equivalent) on my hardware, I can ensure I get what I need, and that there's much less hijacking of my intentions.
So Microsoft, if you're listening: I don't want Bing Chat search, I want Cortana Local.
There are definite frustrations, mostly around playing music. Around 5% of the time, Siri will play the wrong album or artist because the artist name sounds like some other album name, or vice versa. I wish, here, that it used my Music playback history to figure out which one I meant
Once you have the intents parsing, it should be just a matter of throwing man power at it and giving it better intents.
Yes, I have experience with building on top of such a system.
It would be easy to write Siri again and make it a hundred times better, if you could start all over and only write the core features, and not have to validate against the whole product/feature matrix.
The problem with the rewrite of course would be that you won't be able to deliver that minimal viable product any more and you will have 10 years worth of product requirements and user expectations that you MUST hit for the 1.0 release (which must be a 1.0 and not an 0.1).
I've worked on lots of "simple" and "not rocket science" systems that were 10-years old, and it is always incredibly difficult due to the state of the code, the lack of resources, and the organizational inertia.
And the AI team seemed to NOT want to be hidden in terms of discussion of industry ideas with peers at other firms.
They went back to google. Magic 8 ball time.
Who have they hired since then?
Who knows?
Who in leadership is allowing them to succeed?
Results unclear try again.
Who has a clear vision of why to build ALL ONBOARD the users device?
Results unclear try again.
This is already felt in use of Stable Diffusion, where M2 is fully capable offline.
Anything that can be done to reduce the need to “dial out” for processing protects the individual.
It erodes the ability of business and governmental organizations to use knowledge of otherwise private matters to target and influence.
The potential of moving a HQ LLM like GPT to the edge to answer everyday questions reminds me of my move from Google to DDG as my default search engine.
Except it’s even a bigger deal than that. It reduces private data exhaust from search to zero, making going to the net a backup plan instead of a necessity.
Apple delivering this on device is a major threat to OpenAI, which will have to provide some LLM model with training that Apple can’t or won’t.
Savvy users will begin to leer at having to produce queries over the wire, feeding valuable data (proven by ShareGPT)
Even then, Apple will likely chose to or be forced to open up on device AI to allow user contributed apps like LORAs which would ask the question why does OpenAI need to exist?
Also fascinating the potential to do this at the Server level for enterprise. If Apple produced a stack for enterprise training it could replace generalized data compute needs, shifting IT back to local or intranet.
Let me sync my information to something local.
I know there's Nextcloud, but it's not as seamless as iCloud.
saying that, their intentions were good, I'm always horrifically amazed at the number of cookies used whenever I see the preferences popup. I honestly had no idea how many tracking cookies were used by the average website.
That sounds absolutely horrifying if you remove the "all local" part. And that part's a pipe dream anyway. Plus, when using a model you'd basically become subservient / limited to the type of data in the model, which would necessarily abide by Apple's TOS, so a couple of hundred million people would be the Apple TOS but in human form. I don't understand why apple fanboys don't get this. Apple is pretty shoddy when privacy is concerned. Are these apple employees making these posts?
https://machinelearning.apple.com/research/recognizing-peopl...