Comment by SkyMarshal | Hacker News Reader

Comment by SkyMarshal | Hacker News Reader

I think OpenAI may eventually have to go upmarket, as basic "good enough" AI becomes increasingly viable and cheap/free on consumer level devices, supplied by FOSS models and apps.

Apple may be leading the way here, with Apple Silicon prioritizing AI processing and built into all their devices. These capabilities are free (or at least don't require an extra sub), and just used to sell more hardware.

OpenAI is clearly going to compete in that market with its upcoming smart phone or device [1]. But what revenue model can OpenAI use to compete with Apple's and not get undercut by it? I suppose hardware + free GPT3.5, and optional subscription to GPT4 (or whatever their highest end version is). Maybe that will be competitive.

I also wonder what mobile OS OpenAI will choose. Probably not Android, otherwise they would have partnered with Google. A revamped and updated Microsoft mobile OS maybe, given their MS partnership? Or something new and bespoke? I could imagine Johnny Ive demanding something new, purpose-built, and designed from scratch for a new AI-oriented UI/UX paradigm.

A market for increasingly sophisticated AI that can only be done in huge GPU datacenters will exist, and that's probably where the margins will be for a long time. I think that's what OpenAI, Microsoft, Google, and the others will be increasingly competing for.

[1]:https://www.reuters.com/technology/openai-jony-ive-talks-rai...

> OpenAI is clearly going to compete in that market with its upcoming phone.

Excuse me, I'm not an english native, you mean like a smart phone? Or do you mean some sort of other new business direction? Where did you get the info thtat they're planning to launch a phone?

I believe there has been rumors that OpenAI was working with Jony Ive to create a wearable device, but it was unclear wether it would be a phone or something else.

Yes a smartphone, /corrected. It's a recent announcement:

https://www.nytimes.com/2023/09/28/technology/openai-apple-s...

It's not a really a phone. they mention ambient computing.

Oh, smart device then.

Kind of a big difference between the two.

https://www.reuters.com/technology/openai-jony-ive-talks-rai...

OpenAI will make its money on enterprise deals for finetuning their latest and greatest on corporate data. They are already having this big enterprise deals and I think that's where the money is.

They will keep pricing the off-the-shelf AI at-cost to keep competitors at bay.

As for competitors, Anthropic is the most similar to OpenAI both in capabilities and business model. I am not sure what Google is up to, since historically their focus has been in using AI to enhance their products rather than making it a product. The "dark horses" here are Stability and Mistral which both are OSS and European and will try to make that their edge as they give the models for _free_ but to institutional clients that are more sensitive to the models being used and where is the data being handled.

Amazon and Apple are probably catching up. Apple likely thinks that all of this just makes their own hardware more attractive. It's not clear to me what Meta's end goal is.

Yep, we agree that the obvious direction of innovation for OSS models is smaller and cheaper, likely at roughly the same quality: https://generatingconversation.substack.com/p/open-source-ll...

Also more privacy respecting, and more customisable / flexible.

I actually expect open source models will be small _but larger than they are today_ because phones and laptops will get dedicated chips and software for running eg the best open source (weights?) model

So eventually you could be running decent sized models locally (iOS could even provide an API with fine tuning etc)

Where are you taking the confidence that Apple will be able to catch up to OpenAI’s GPT? “Apple's built-in AI capabilities” are very weak so far.

Not OP,

In my experience apple's ML on iphones is seamless. Tap and hold on your dog in a picture and it'll cut out the background, your photos are all sorted automatically including by person (and I think by pet).

OCR is seamless - you just select text in images as if it was real text.

I totally understand these aren't comparable to LLMs - rumor has it apple is working on an llm - if their execution is anything like their current ML execution it'll be glorious.

(Siri objectively sucks although I'm not sure it's fair to compare siri to an LLM as AFAIK siri does not do text prediction but is instead a traditional "manually crafted workflow" type of thing that just uses S2T to navigate)

>OCR is seamless

Wasn't that solved about a decade ago. Does anyone suck at that?

> Does anyone suck at that?

Does android even have native OCR? Last I checked everything required an OCR app of varying quality (including windows/linux).

On ios/macos you can literally just click on a picture and select the text in it as if it wasn't a picture. I know for sure on iOS you don't even open an app to do it, just any picture you can select it.

Last I checked the Opensource OCR tools were decent but behind the closed source stuff as well.

Random google result of OCR on android (could be outdated) - https://www.reddit.com/r/androidapps/comments/10te5et/why_oc...

> Does android even have native OCR?

Tesseract? https://github.com/tesseract-ocr/tesseract

Android has had built in OCR for a few releases. You can copy and paste text from any image/app you can display on the screen.

To use, go to the app switcher and just start selecting text.

It's funny trying to select text in a video while the video is playing...

> Android has had built in OCR for a few releases. You can copy and paste text from any image/app you can display on the screen.

AFAIK this is a feature only available on Google Pixel devices. so not on 'Android' in general.

For one, my Oppo device running Android 12 doesn't have it.

Yes, Android fragmentation sucks.

Not sure about other Android OEMs but OCR has been built in to Samsung Gallery (equivalent to Photos app on iPhones) for a while. Works the same way - long press on text in an image to select it as text. Haven't had any issues with it.

I use Google Lens for OCR in photos and the swipe-up+"select" feature for OCR in arbitrary apps/websites.

That at least works for Pixel phones. Not sure about Samsung and others.

I'm not saying they will on the high-end, but maybe on the low end. Apple's strategy is to embed local AI in all their devices. Local AI will never be as capable as AI running in massive GPU datacenters, but if it can get to a point that it's "good enough" for most average users, that may be enough for Apple to undercut the low end of the market.

> Local AI will never be as capable as AI running in massive GPU datacenters

I'm not sure this is true, even in the short term. For some things yes, that's definitely true. But for other things that are real-time or near real-time where network latency would be unacceptable, we're already there. For example, Google's Pixel 8 launch includes real-time audio processing/enhancing which is made possible by their new Tensor chip.

I'm no fan of Apple, but I think they're on the right path with local AI. It may even be possible that the tendency of other device makers to put AI in the cloud might give Apple a much better user experience, unless Google can start thinking local-first which kind of goes against their grain.

> But for other things that are real-time or near real-time where network latency would be unacceptable, we're already there.

Agreed. Something else I wonder is if local AI in mobile devices might be better able to learn from its real-time interactions with the physical world than datacenter-based AI.

It's walking around in the world with a human with all its various sensors recording in real-time (unless disabled) - mic, camera, GPS/location, LiDAR, barometer, gyro, accelerometer, proximity, ambient light, etc. Then the human uses it to interact with the world too in various ways.

All that data can of course be quickly sent to a datacenter too, and integrated into the core system there, so maybe not. But I'm curious about this difference and wonder what advantages local AI might eventually confer.

This is a fascinating thought! It could send all the data to the cloud, but all those sensors going all the time would be a lot of constant data to send, and would use a lot of mobile data which would be unacceptable to many people (including probably the mobile networks). If it's running locally though, the data could we quickly analyzed and probably deleted, avoiding long term storage issues. There's got to be a lot of interesting things you could do with that kind of data

I wonder if you could send the embeddings or some higher level compressed latent vector across the cloud you couldn't get the best of both worlds.

GPS, phone orientation, last 5 apps you were in, etc. --> embedding

you might even have like "what time is it?" compressed as it's own embedding.

> I think OpenAI may eventually have to go upmarket

Let me introduce you to the VC business model. Get comical amounts of money. Charge peanuts for an initial product. Build a moat once you trap enough businesses inside it. Jack up prices.

don't forget the sneaky TOS changes you have to agree to

OpenAI'd better hope no one else does it too, if that's all it takes.

> OpenAI is clearly going to compete in that market with its upcoming phone

What phone are you referring to? A quick google didn’t seem to pull up anything related to OpenAI launching a hardware product?

They are most likely referring to this in collaboration with Jony Ive:

https://www.yahoo.com/entertainment/openai-jony-ive-talks-ra...

Yes that one.

Please Apple let me replace worthless Siri with ChatGPT on my iPhone.

Would completely change how I use the device.

If you have the new iPhone with the action button, you can set a shortcut to ask questions of ChatGPT. It’s not as fluid as Siri, and can’t control anything, but still much more useful.

Just yesterday, while driving: "Read last message."

Siri: "Sorry. Dictation service is unavailable at the moment."

It's past time for excuses. High-level people at Apple need to be fired over this. Hello? Tim? Do your job. Hello? Anybody home...?

Nobody is switching away from Apple over this, so ultimately Tim is doing his job. Under his watch Apple has become the defacto choice for entire generations. Between vendor-lockin/walled gardens and societal/cultural pressures (don't want to be a green bubble!), they have one of the stickiest user bases there are.

True, but that doesn’t mean we shouldn’t complain.

My hope is that the upcoming eu rulings allow competition here. Ie force Apple to get out of the way of making their hardware better with better software.

Stop excusing shitty work from trillion-dollar companies. It makes the world a worse place.

I think it's shitty and has no excuse, but the parent is right. Apple has no incentive to respond to their users since all roads lead to first-party Rome. It's why stuff like the Digital Market Act is more needed than some people claim.

You know what would get Apple to fix this? Forced competition. You know what Apple spends their trillions preventing?

agreed. I'm not trying to "excuse" shitty work, merely observing the incentives/pressures on them. We can complain about it all we want, but we won't understand it until we understand the incentives.

I think that's a bit glib. Right now it's true that nobody's going to leave the Apple camp because of Siri, but it's also true that nobody's going to leave the Android camp because of it. That state of affairs could change.

It's not a time for complacency, if only because driver assistance is becoming more important every day. There are good, sound business reasons to put competent people on the Siri team.

Apple is ramping up spending in that area: https://www.macrumors.com/2023/09/06/apple-conversational-ai...

It’ll probably take a while though.

Yeah, at this rate they'll be at parity with Infocom by 2030.

So their customers will have a difficult time figuring out that they should put a fish in their ear?

Wait'll you see what they charge for hints.

Reply on news.ycombinator.com