Meta and Qualcomm team up to run Llama 2 on phones

To me, this is probably the most important announcement from today. The ability to have an on-device low-latency implementation of an LLM is going to be key to AI/UX. This is similar to Google's various PaLM versions for Android they announced. It will have a huge impact on mobile phones, TVs, smart screens, etc. Obviously the devil will be in the details - like how well will it actually work? But even if it's just the first step, it's an important one.

I bet next up will be at the end of the year when Apple surely jumps into the ring with their version of on-device Siri, powered by their ML chips.

> Llama 2, to run on Qualcomm chips on phones and PCs starting in 2024

Are they expecting for Llama 2 to still be relevant in 2024? Shouldn't it have been LLM or Llama (without the version).

Qualcomm wanted to run Stable Diffusion on phones as well, and IIRC they published a demo, but no one cared because it was a barebones demo.

MLC-LLM already runs LlamaV1 (and probably V2?) performantly on Android and iOS. But again, no one seems to care because its a relatively barebones demo.

Qualcomm needs to build and extensive feature set and probably a UI, otherwise no one is going to care about this implementation either.