Apple Tests ‘Apple GPT,’ Develops Generative AI Tools to Catch OpenAI
bloomberg.com
bloomberg.com
But you don't need to beat chat-gpt 4. A chat-gpt 3.5 esque model that has access to your todo list and calendar that I can talk to like siri would make the tool 10x more useful overnight.
If it even had access to your last 50 emails I could imagine saying something like.
- remind me to respond to that email from bob tomorrow.
- Draft me 3 responses that are kind, but indicate clearly that we can't go forward with his request.
This could even happen in the background, and I'm happy to pay a bit extra for the compute, it could be "premium siri"
No one seems to have solution for prompt injection yet…
>happy to pay a bit extra for the compute, it could be "premium siri"
Siri is heavily embedded in the ecosystem, yet has bitterly under delivered for far too long. Many improvements should have been made absent access to an LLM.
Apple should provide this behavior out of the box, and figure out how to make the costs work without additional subscriptions.
I think the cost of running these models is still astronomical, and unlike ms and google apple just doesnt really have cloud compute at their scale. So I could imagine they might run this as a premium service first to build hype, and then figure out bringing it down in cost over time.
A bit like the vision pro, its a premium product now, but I could imagine 3-6 years down the line it will cost around an ipad pro. Apple is really good at scaling things once they see the market. But suddenly switching everyone to a gpt-3.5 for siri overnight would be a humongous cost (I think).
Apple is the only company I know of (discounting Nvidia), that has this established base of consumers just waiting for a killer app.
https://machinelearning.apple.com/research/neural-engine-tra...
In addition to the leaps and bounds in silicon, Apple has also quietly been making acquisitions in the space (like $200M+ for Xnor.ai - focusing on low-power ML intended for edge computing applications).
They are likely relentlessly optimizing, and in typical Apple fashion will not be first to release something, but will do a killer job in distributing it to hundreds of millions of devices/people at once, with a UX that's thoughtfully polished and accessible to all.
I’m not sure what your complaint is, Siri does quite well with timers and alarms.
Yes! And if you could just add this to your "profile" so you didn't need to add it each time, prompts would be that much shorter.
I think they are doing great things, and their exclusivity with openAi was a killer move. But it still falls a bit short in execution (imo)
Obviously it would be much cheaper at scale, and like you said, it doesn’t have to be cutting edge. But still, the compute for an interaction with Siri is a fraction of a penny.
I think apple with its app ecosystem though could just let apps offer themselves as plugins to "siri-gpt", with their available function calls (like openai does now).
Their own apps, mail/todo/calendar could then play in a common ground which should be considered fair.
Apple has the perfect ecosystem to do this imo, they could knock openai plugins out of the park, since the apps would be ones that you use, and know about you, so would be far more useful.
Siri excels at like 2 things: setting reminders and setting timers. Anything else I don't really trust it for.
Teaching an LLM is easy, making it safe for public consumption is _really_ hard.
"Safe" in what sense? Is an LLM output which we recognize as racist or factually incorrect going to be directly responsible for the loss of life, serious injury or significant property damage?
If LLMs can be considered unsafe for public consumption, then so are Twitter, Facebook, heck, even Wikipedia and Hacker News!
Yeah, if you say the right things to an LLM, it might say something weird. Just like your child might swear if you swear around them, or they might make up a fact and claim it's true. It's cute/funny/interesting, but really not special or newsworthy. But because AI is the new big thing, everyone is scrambling for a headline. I think people will get bored and move on.
Me: turn the music off
Siri: Got it, setting a timer for 5 minutes
Me: No! Stop!
Siri: There are not timers to stop right now.
me never using siri again
Me: Remind me at 8am to put X in the kids' school bags
Siri: I'm sorry, I can't set reminders in the past
Me: 8am tomorrow you fucking idiot
At 8am the next morning I got a reminder on my phone that just said "You fucking idiot".
Siri: Calling Emergency Services
Me: frantic swiping to cancel
I use the non-display speakers so I can't even view the stuff.
- Timers can be set to the second (2 minutes 30 seconds), alarms are only to the minute (5:00) (it already works this way, I think)
- A timer noise should be immediate and jarring (so you can rush out of the other room to pull the thing out of the oven before it burns), alarms should start quiet and harmonious and only gradually increase in volume and obnoxiousness as needed to wake you up (sadly, few alarms are like that, but Google timer/alarm sounds are different)
- Ideally you want to be able to give timers names when you set them ("oven", "stockpot", "toaster oven") when you're using multiple, that are repeated when they go off (Google doesn't do this). You don't need this with alarms
- Alarms need a snooze option. Timers don't
While they both involve a sonic alert after a certain amount of time, their use cases are so different that they really are totally separate features.
They have speaker groups (that work well), but if you have two+ Google Home/Hub/Mini devices in a single space asking for a timer will add it randomly to one of the devices, not the room or a group, and it won't always display on the Hub.
For example I have an open-plan kitchen, two Google Home Max speakers, and a Hub (display) on the counter. If I ask for a timer, it MIGHT go to the Hub or it MIGHT go to either of the two Home Max speakers.
Cancelling it involves figuring out which device took it, and standing close to it (or it just won't cancel). This is a really obvious and strange oversight, as Google Home's ecosystem supports "rooms" by default.
The truly saavy version would have calculated the time left until 7pm and set a timer for that many minutes, however.
I don’t think my perception of what it should be able to do has changed…
This is similar to Slack vs Microsoft Teams. Slack had a several year head start on product, but Microsoft had a massive lead on distribution (their Office 365 install base). Teams is good enough.
Man, them's fightin words ;-).
Teams is atrocious, like it is stuck back in the 90s. It's only popuplar because Microsoft gives it away. For all of it's warts I'd take Slack back in a heartbeat.
If your office has a culture of email for written communication and the chat feature is sparingly used and mostly for quick 1-1 messages or "can I call you?" then it's passable.
Not the most onerous of problems, for sure, but I'm not liking how much our IT department is clamping down on everything. Between cost cutting on software and the myriad malware packages they install to monitor our MacBooks, I'm not their biggest fan.
Compare it to Slack here you can safely meme in #random…
Nonsense.
CoreML, Apple Silicon (Neural Engine), and even Face ID all have AI advances built right into them.
Apple is not desperate to join the AI hype mania and neither is it a direct threat to them.
Their iDevices and services make enough tens of billions for them to eventually catch up or acquire other foundational AI companies to do that. That is a luxury reserved for extremely profitable companies that can afford to wait or enter late.
It wasn't an accident. They know what the current so-called "AI" is and have been using it for a loong time. They're not just getting on the hype train of calling everything AI.
Take iWork, for example. You can use AI to rough-in a document you need to write. Then you can fill in some of the details and make it so it's written by you. Even better if you can point to a personal archive of documents you can use to train your own personal AI so it's better able to write a more relevant document.
Consider Logic Pro. We've all seen AI-generated music, so why not provide an AI that generates your base music project that you can then tweak and embellish?
This is how I expect normal people to want to use AI - embed it in their existing tools. I also don't think most people expect AI to generate 100% of the content, instead they would expect the AI to do the heavy lifting and get them maybe 60% to 70% of the way there. This gets them over the blank page syndrome and allows them to produce a quality result.
I'm reminded of a meme I recently saw that really resonated with me, it said AI isn't going to take your job, it's somebody using AI who's going to take your job. AI is a tool and while we may be dazzled with its capabilities, we're going to be even more dazzled by what people using AI are capable of doing.
Apple would instantly become the largest deployed LLM service except the users pay for the hardware and electricity and space in exchange for privacy.
Apple already has plenty of existing Siri headaches where features fail when a HomePod doesn't see your phone on Wifi.
If there’s one company that could credibly pull off such a feat without it feeling super creepy—it’s Apple.
Like why wouldn’t Siri with GPT-n behind it just totally crush anything that any other possible competitor could do? Distribution and adoption of SIRI was complete over a decade ago.
I just don’t see how anyone could compete on a voice assistant tool with apple and google. I mean I WISH it was possible but the reality is different.
The end result is still the same in the long term, and that they’re going to eventually be able to figure out systemic ways to include LLMs and work to improve them to get around existing constraints.
Get AI Siri to say something racist and it's front page news everywhere in the world.
They have a huge unique niche to exploit with local private models, probably the only company on the planet with the hardware in place to take advantage of this today end-to-end, but unfortunately I don't think generative AI fits with Apple's culture for anything I just think they'd be terrified of it saying something wrong so would probably clip its wings to the point that it's mundane and kinda useless.
- Android, Windows, Linux and MacOS can already run local and private models just fine. Getting something product-ready for iPhone is a game of catch-up, and probably a losing battle if Apple insists on making you use their AI assistant over competing options.
- The software side needs more development. The current SOTA inferencing techniques for Apple Silicon in llama.cpp are cobbled together with Metal shaders and NEON instructions, neither of which are ideal or specific to Apple hardware. If Apple wants to differentiate their silicon from the 2016 Macbooks running LLaMA with AVX, then they have to develop CoreML's API further.
Given how business leaders throughout tech feel that AI is going to be transformative, I don't think commitment is really going to be a problem. Many leaders feel that "you either get good at AI or you don't exist in 10 years".
In terms of attracting talent, there are 3 main things top AI folks look for:
1. Money (they are people after all)
2. The infrastructure (both hardware and people/organization-wise) to support large AI projects.
3. The willingness to release these AI projects to a large swath of people (to have "impact" as folks like to say).
E.g. Google had 1 and 2 but their reticence to release their models and corporate infighting made many of the top Google researchers leave for gigs elsewhere. I think it remains to be seen how Apple will handle #3 as well.
Siri is sort of a red herring because its built by teams and tech that existed before Apple acquired most of its ML talent and some of its inability to evolve has been due to internal politics not the inability to build tech. iOS 17 is an example of Apple moving towards more deep learning speech/text work. I would bet heavily we will see them catch up with well integrated pieces as they have Money, infra, and already the ability to go wide (i.e all iOS users, again think FaceID).
Well, I disagree with this take. Apple is known for playing the long game and planning for many years ahead. CPU power is still growing and Apple now has their own CPUs on every device. Sure you won’t be able to run something similar to GPT-4 in foreseeable future, but I predict we will see multiple small, feature-oriented LLMs that can easily be fit into a smartphone or at least an iPad with M(n) processor
I think it fits pretty well since Apple controls almost 100% of their stack. If they need hardware specific tweaks to make AI models run better, they can do that. On M* Macs for example, the unified memory model lets them do a number of AI tasks even with the lower powered GPU.
Seems like any AI stuff will be designed to add value to its hardware exclusively, probably in conjunction with its on-device Neural Engine (tm).
- Apple Pay JS
- MusicKit JS
- Sign in with Apple JS
- App Store Connect API (not really user facing, but still)
I think Apple will come up with some crazy hardware to run good quality LLMs.
you mean like the "neural engine" that has been present in their SoCs for nearly a decade? (this is also why M1/M2s can run LLMs at comparable speeds to desktop GPUs... and they weren't even designed with LLMs in mind yet)
if you use commodity GPUs, sure. if you use TPUs (which Apple is already building into their chips) the efficiency improvements are massive. seriously look at some Coral Edge TPUs and what they can do at power levels completely unheard of for GPUs. then look at how much faster M1/M2 Macs are than normal desktop GPUs for machine learning tasks because they have an onboard accelerator
apple is known to care a lot about stuff like this. like, a lot. they are pedantic as heck
Apple needs something like what Adept.ai has with action based foundational models. [0] for Siri to be useful.
LLMs are essentially overhyped for everything other than summarisation and this just shows that OpenAI really has no moat and it is getting eroded faster than they can stop losing money on training the model.