Hacker News Reader

Comment by kemayo | Hacker News Reader

The bits of this controversy about "where's the LAM?" are strange to me. My impression from the beginning was that the "Large Action Model" was a server-side thing, and the hardware was going to be almost-entirely a way to talk to their services.

Apparently this impression was far from universal.

View on HN

I recall the LAM being a/the (stated) reason why it couldn't "just be an app."

If it's vaporware on-device it's more evidence that it could have just been an app. Perhaps there was special on-device sauce coming in a future update, but as of now it's a client making http calls.

You probably wouldn't want an app to be able to press buttons on another app so if that's how it works (or worked) then that seems fair enough to me.

You probably really wouldn't want it to be doing that on someone else's server, using your authentications to all your apps. Which is what it actually does (or claims to, anyway).

it's an app with extra deep data collection, would be my guess. Also it's a nice construction to get attention.

Yeah the whole point was supposedly you can "train it" by operating a generic "app" and it would learn so in the future other people could operate these apps via voice only.

A clearly as yet non-existant feature. But would definitely require super obervation, recording, and control above standard android. Also insecure as hell.

It would be super dumbo-bonkers insecure if you werent actually running these "Generic app training" on your own device, but in some cloud emulation. yikes

This is as dumb as microsoft selling their next big feature of ai knowing how to control generic windows applications... that means they are going to record EVERYONES screens and actions without consent, we all know how microsoft respects users consent

Why couldn't the AI be done on-device rather than server side? Isn't that where things are going with mobile hardware?

Mobile devices don’t have the memory or the memory bandwidth to run an LLM that’s big enough to be good at much. Plus the fixed battery and thermal constraints.

my M1 Macbook air can run LLM's pretty well.. worse specs than the latest iPad Pro (and iPhone pro wouldn't be too far behind).

Running them is a whole lot less resource intensive than training them.

Unless the plan was just to build a RAG source from your personal data, in which case it would be yet another underwhelming feature.

I guess that will never change huh

If this were an app, my guess is that this will never pass Google’s test with all the data passed to their servers, with all the privacy requirements.

As someone whose job is to inference on the edge. No one is getting a mass product consistently inferencing on the edge for $199 not today and not next year (and I’d bet not in 2026)

Your impression was spot on

What do you mean by edge? There are already many apps that use LLMs and diffusion models locally on even a mac M1, which means iPhones and tensor chips aren't far behind

Macs and iPhones don’t cost $200

Take a look at an app like this: https://apps.apple.com/us/app/offline-chat-private-ai/id6474...

This will perform inference on Phi-2 or Mistral on almost any iPhone device. The first neural engine came with the A11 bionic chip which is in the iPhone 8. People can buy that phone for $150 used on Amazon.

I think that's evidence that inference on the edge is already here

> or Mistral on almost any iPhone device

> "Small" requires iPhone 14 and 15 Pros

> "Small" and "Medium" models are based on Mistral 7B 0.2

So no, also that app wouldn't work on an iPhone 8 either since it requires iOS 17 or later

You could even go up a few iPhone versions and still be <$199. iPhone 12 used ~$190 supports iOS 17 and can run the Tiny models (phi-2).

For small models, today a used iphone 14 costs ~$300. Which means next year it will by down to ~$200.

I'm pretty sure by 2026, we will be able to inference "medium" models like Mistral/Llama3 8B on a <$199 device. That's not even factoring in how much better models will get by that time

That’s not what the parent comment is talking about. They say that no one can mass produce these devices and sell them for $200. You don’t mass produce used iPhones to sell to people.

What I find interesting from the analysis is that the hard-coded, per-app event handlers suggests their integrations are nothing more than traditional wrappers around the services they support, instead of some magic, AI powered LAM living on the cloud that can interact with anything:

UberEventHandler SpotifyEventHandler FoodDeliveryEventHandler MidjourneyEventHandler TranslatorEventHandler SearchWeatherInfo SearchStockInfo

Reply on news.ycombinator.com