Hacker News Reader

Comment by tonyhart7 | Hacker News Reader

can we have tiny LLM that can run on smartphone now

Apple Intelligence has an LLM that runs locally on the iPhone (15 Pro and up).

But the quality of Apple Intelligence shows us what happens when you use a tiny ultra-low-wattage LLM. There’s a whole subreddit dedicated to its notable fails: https://www.reddit.com/r/AppleIntelligenceFail/top/?t=all

One example of this is “Sorry I was very drunk and went home and crashed straight into bed” being summarized by Apple Intelligence as ”Drunk and crashed”.

I think the real problem with LLMs is we have deterministic expectations of non-deterministic tools. We’ve been trained to expect that the computer is correct.

Personally, I think the summaries of alerts is incredibly useful. But my expectation of accuracy for a 20 word summary of multiple 20-30 word summaries is tempered by the reality that’s there’s gonna be issues given the lack of context. The point of the summary is to help me determine if I should read the alerts.

LLMs break down when we try to make them independent agents instead of advanced power tools. Alot of people enjoy navel gazing and hand waving about ethics, “safety” and bias… then proceed to do things with obvious issues in those areas.

Larger LLMs can summarize all of this quite well though.

Determinism isn't the issue though. Many responses are fine. The displayed one is bad, whether chosen deterministically or not. Some alternatives:

- Passed out drunk

- Crashed in bed

- Slacking because drunk

...

The issue isn't a lack of context; it's that even the available context was handled poorly.

No. Smartphone only spin animated gif while talk to big building next to nuclear reactor. New radio inside make more efficient.

Is a tiny large language model equivalent to a normal sized one?

Yes is called MLM (Medium Language Model)

I expect that the phone will only do the prompt parsing

I want a tiny_phone_based LLM to do thought tracking and comms awareness..

I actually applied to YC in like ~2014 or such for thus;

-JotPlot - I wanted a timeline for basically giving a histo timeline of comms btwn me and others - such that I had a sankey-ish diagram for when and whom and via method I spoke with folks and then each node eas the message, call, text, meta links...

I think its still viable - but my thought process is too currently chaotic to pull it off.

Basically looking at a timeline of your comms and thoughts and expand into links of thought - now with LLMs you could have a Throw Tag od some sort whereby you have the bot do work on research expanding on certain things and plugging up a site for that Idea on LOCAL HOST (i.e. your phone so that you can pull up data relevant to the convo - and its all in a timeline of thought/stream of conscious

hopefully you can visualize it...

I had a thought that I think some people value social media (e.g. Facebook) essentially for this. Like giving up your Facebook profile means giving up your history or family tree or even your memories.

So in that sense, maybe people would prefer a private alternative.

I read this in Sam Wattersons voice with a pipe abt maybey an inch from his beard,

(Fyi I was a designer at fb and while it was luxious I still hated what I saw in zucks eyes every morn when I passed him.

Super diff from Andy Grove at intel where for whateveer reason we were in the sam oee schekdule

(That was me typing with eues ckised as a test (to myself, typos abound

Reply on news.ycombinator.com