This device is selling for $1,999. In the product page on their website, they show the device plugged into a MacBook Pro.
A MacBook Pro already has excellent ability to run local LLMs, so why wouldn't someone just spend that $1,999 on maxing out their MacBook specs (M5 Max with extra RAM) instead?
Or why not buy a Mac Mini with 48GB of RAM for $2300?
Because this device has 80 GB, which allows running larger models (which is the whole point of the device).
AFAIK, the next significant step in terms of quality above 32 GB is 256 GB, but 80 GB still buys convenience.
> A MacBook Pro already has excellent ability to run local LLMs, so why wouldn't someone just spend that $1,999 on maxing out their MacBook specs (M5 Max with extra RAM) instead?
80 GB extra (48 → 128) cost as much as this device, but the base MBP is very expensive already (4000+$). This device is essentially a "cheap" addon for an existing system.
Perhaps they don't want a new/upgraded machine or prefer a more portable solution.