Apple AI researcher boast on-device model that 'substantially outperforms' GPT-4 | Hacker News Reader

Apple AI researcher boast on-device model that 'substantially outperforms' GPT-4

9to5mac.com

42 pointsksec15 comments

The full context in the article says that this only applies to the specific task of analysing a screenshot and answering a user prompt based on the information in the screenshot (and some other things). It is fair to say that a small model that can run on the iPhone won't have the world knowledge to compete against GPT4 but it can be better for something like this specific task.

Section 7 Conclusion and Future Work

We show that ReaLM outperforms previous approaches, and performs roughly as well as the state of-the-art LLM today, GPT-4, despite consisting of far fewer parameters, even for onscreen references despite being purely in the textual domain. It also outperforms GPT-4 for domain-specific user utterances, thus making ReaLM an ideal choice for a practical reference resolution system that can exist *on-device without compromising on performance.*

-----------------------------

Emphasis mine.

As far as I can tell, the model isn't exactly on-device yet.

I recently noticed Apple stock is struggling quite a bit, and then I see things like this pop up more often recently, where the company (or "someone") talks about what's going to happen and what they're working on internally, rather than actual releases and demonstration of work "done". Would love to see a bit more "walk the walk" from Apple.

It reads to me like the Google announcements. Platitudes to calm investors down so they can say "we are doing AI too see!". But I think Apple is really struggling, similar to Google.

Dropping sales in China doesn't help either...

It’s not an announcement, and the headline is a lie that has nothing to do with Apple.

Apple published an academic paper, as they have been doing for years, showing a model that objectively outperforms GPT-4 on one narrow test. Nowhere do they claim to outperform GPT-4 in general.

Assuming that this is somehow intended to manipulate the stock price is not a rational analysis of what is going on.

Also, Apple has operated on a predicable release schedule for a decade now. They have already said that they will be announcing their AI work at WWDC. It’s unclear what anyone is expecting them to do differently or why it would make any sense.

As for the stock price - the explanations as to why it’s depressed are twofold - one is regulatory action, and the other is frenzied investors who want to jump on the AI bandwagon.

Dropping sales in China is a headline like a stopped clock. Usually wrong, sometimes right, and we only find out which at earnings time.

Note that nobody’s AI product is doing great yet, so for the most part it’s all performative, something that Apple doesn’t engage in.

Also, unlike most other companies, Apple is really pushing for ML/AI on device. And while there are serious limitations for that, I really don't like to be beholden to opaque profit-at-all-costs companies like Google or OpenAI or...

[dupe]

Some more discussion: https://news.ycombinator.com/item?id=39901952

evals or didn't happen

All I want in my Apple product is to be able to choose my provider for AI services.

And you'll be able to get any provider in this paradise as long as it's Siri

Id like it if we had a fairly decent model that is Siri that runs locally and privately and without signal.

BUT that can call a really powerful model as it sees fit, like ChatGPT plugins.

Ideally I can have some choices about this external model, or if Apple can do a deal with OpenAI or Google to power this external model that that could be pretty sweet too.

I'm sure the E.U. will eventually force them to let you replace Siri with another voice assistant if you live in the E.U.

Not Apple OS integrated but have you used Poe? Allows you to chat with ChatGPT, Gemini, etc all in one place.

https://poe.com/

(I am completely unaffiliated with them)

What? You're more than welcome to download an app, or use any company's website.

And what will that app do? Nothing can answer the queries by default except Siri.

There are extremely awkward workarounds by telling Siri to run a shortcut, but they are workarounds, and due to weird tech choices they don't work well: https://twitter.com/dmitriid/status/1640692625597505537

Reply on news.ycombinator.com