Hacker News Reader

Comment by taf2 | Hacker News Reader

So it’s safe to assume in the next 10 years - AI will be running locally on every device from phones, laptops and even many embedded devices. Even robots- from street cleaning bots to helpful human assistants?

View on HN

The bottleneck is probably the availability of lithography machines that can make ubiquitous chips for processing that much data quickly enough without heating or drawing electricity too much.

Not too far in the future every device will have some 5nm or better tech LLM chip inside and devices understanding natural language will be the norm.

By dumb machines, people will mean machines that have to be programmed by people using ancient techniques where everything the machine is supposed to do is written step by step in a low level computer language like JavaScript.

Nerds will be making demos of doing something incredibly fast by directly writing the algorithms by hand, and will be annoyed by the fact that something that can be done in 20 lines of code on few hundred MB of RAM in NodeJS now requires a terabyte of RAM.

Dumb phone will be something like iPhone 15 pro or Pixel 8 Pro where you have separate apps for each thing you do and you can't simply ask the device to do it for you.

I really don’t buy that last paragraph. It’s the idea the AI Pin founders keep saying - you don’t want apps, the device chooses it for you, etc. In a world where you have economical choice, that’s quite hard to imagine: would there be no competition? What good is it for you if you can’t even choose what music app you want? Without _something_ like an app, how do you even know what you can interact with?

> Without _something_ like an app, how do you even know what you can interact with?

You just tell what you need or want and the AI will suggest you options.

>What good is it for you if you can’t even choose what music app you want?

You don't want a music app, you want to listen to music. The AI will simply play it to you and any additional features that you might like will be available. Feel like doing karaoke? Just tell to the AI. Want more songs like this? Tell it to the AI. Not sure what you want? Tell the AI that you are not sure and ask for suggestions, maybe vaguely describe how you feel. You get the idea.

> would there be no competition? There would, but not in the current sense.

Maybe there would be experience designers and knowledge brokers. The AI might tell you that you will need a weather info subscription if you need something specific that is not provided by your current subscriptions. Maybe you want some specific recipe from a home lighting style from a designer? The AI can help you purchase and apply that. Maybe you want to try dating? It wouldn't be an app but a subscription you can join and the AI will handle the UX.

However, I think there will be games because games are apps whose value comes from interaction with the app itself.

I absolutely 100% want youtube music because it has the music I want and I have my playlists there. That entire argument of “you don’t want X” is misguided.

Maybe for someone whose first computer ever is that AI panopticon, then sure. But will all music be free? What about netflix? Will it also be free? What happens to competition? Or choice of brands?

Music will still be like Spotify IMHO, you will pay a monthly subscription. There wouldn't be an app, like everything else.

How about something like photoshop? Or a text editor for when I want to write a book?

These are already under threat, people are editing images using text for a year now - no tools needed, you can describe what you want or tap on an area of the image and describe what changes you want. These days they started doing videos too.

They’re still apps, and nobody that does anything minimally professional uses “no tools” (in fact it’s quite the opposite, AI added a mindnumbing number of tools to the toolset). I really don’t see how these will just “disappear” and become some amorphous “talk to the computer” interface

It's not on professional level yet, but it was barely O.K. a few months ago. It's moving very fast.

The gist is, if something is learnable by practice the AI can do it because by training these machines we actually teach them patterns and methods. Any "blue collar" job like editing images is going away.

> Maybe you want to try dating? It wouldn't be an app but a subscription you can join and the AI will handle the UX.

A subscription to.. an app?

To a service. No more apps anymore, the app is the AI who can mediate as a UI to anything.

That sounds… exhausting?

Why?

Talking is waaaaay more time consuming than tapping or typing, having to memorize what I’m able to do on my device instead of just taking a glance at its screen is a lot of cognitive overload, not having any shared UI with others is a recipe for eternal confusion…

I really don’t see any clear advantages of having a one-fits all shapeless interface that’s driven by prompts (verbal or not). Doesn’t feel like a UX that makes sense to me, and I haven’t heard a case where it does yet tbh (I do think the “personal assistant” makes sense, and might replace some stuff, but don’t see how it’d become the next interface for everything)

There's no reason why AI app wouldn't be able to interact with you through the most suitable UI.

For example, when playing music can show you the basic buttons but also let you type or speak for more advanced stuff like "let's do karaoke" or "why don't we go through move soundtracks by showing me iconic scenes from each movie while playing the songs".

Every Google Home device is already running an ML model to do speech recognition to recognize the "hey Google" wake word, so sooner than 10 years. The Raspberry Pi Zero is a particularly underpowered device for this. Doing it on the Coral TPU accelerator plugged into a pi zero would take less than 30 mins. Doing it on an iPhone 15 would take less time. Doing it on a Pixel 8 would be faster. Not to diminish getting it to work on a Pi Zero, but that future is already here, just as soon as we figure out what to do with them.

https://coral.ai/products/accelerator/

There's an ocean of difference between optimizing for a single wakeword and the class of models that are taking off today. I'm excited for more on-board processing, because it will mean less dependency on the cloud.

Siri came out 12 years ago, wake words are probably a bad example.

Better examples are the magic eraser on my 2 generation old pixel phone or the fact llama2 runs genuinely fast on a Mac mini

Siri’s wake word stuff is also terrible, she gets constantly activated whenever I have my Apple Watch near running water, frying food or anything else that makes a white noise-type sound.

I'm not going to argue that there isn't a difference when going from 0 -> 1 and 1 -> 10, or in this case, from 1.5b (Whisper-large) -> 1.7t parameters (gpt-4). But it's not like we don't know how to do it, so it won't take 10 years to get there.

Yeah, that's happening right now really? There has been loads of developments in the mobile space already, in many ways lower powered arm devices are way more optimised for AI applications than the current crop of intel machines.

This example, whilst impressive, feels way more in the "Doom running on a calculator" vein of progress though.

Yes and no. Context matters.

Will models of similar quality to the current LLaMA, GPT, and Stable Diffusion be running locally on devices and edge systems? Very likely.

Will much higher quality models that still require compute incapable by such edge or consumer devices be available, sold as a service, and in high use? Also very likely.

So expect current quality to make it to your devices but don't expect to necessarily move everything to local because the whole ecosystem will improve too. Overton window will shift and it's like asking if gaming will move to phones. In some ways yes, in other ways you're still going to want to buy that Playstation/Xbox/PC.

That's optimistic. We don't even run the humble word processor locally anymore. Everything is cloud, cloud, cloud.

You don’t run it because you don’t want to, not because the device can’t run it

“AI” is already running on every single laptop and phone. If you mean diffusion models (like this), then I’d say it’s 100% guaranteed they’ll run everywhere too, since they’ll get faster and more refined, and processing power keeps growing (it doesn’t even have to grow fast)

Yes, every processor will have an AI core or two.

Probably less than 5 years, maybe 2

Probably not even 10 years.

I mean, diffusion models tend to be less computationally expensive than say CNNs or LLMs, so probably? And before than ppl ran SVMs, random forest, and other forms of non-gpu intensive ML algorithms locally as well...

Reply on news.ycombinator.com