Is anyone making LLM-in-a-box for emergency supply kits yet?
I feel that would be handy in all sorts of situations when networks are down.
I feel that would be handy in all sorts of situations when networks are down.
For most actual emergency scenarios, a device that focuses on storage of large amounts of prepared normal reference material [0] will be wayyyyy cheaper, more durable, portable, and able to run on batteries or being constantly plugged into a somehow-still-normal electrical grid. (Think an e-ink tablet that can run off a 5V battery pack buffering a literal handcrank.)
In contrast, imagine spending the money to build a beefy LLM-running computer with good GPU/RAM, and somehow mothballing it (to depreciate, unused) in a "safe" location for the big earthquake/flood/etc... Then when the disaster strikes and you dig it out, how will you power it when you need it, and for long enough to do anything useful?
Even if wall-current civilization is 20 miles away on the other side of the mountain, are you going to carry it on your back, or are you going to carry food and water to live? If you do drag it there, are they going to let you run it when it cuts into light for surgery or heat to sterilize drinking water?
In general, I think speech as input/output is under-explored. In the emergency scenario, in a stressful environment, having an expert in your ear you can talk to should work much better than having a big manual book to look up specific cases.
That's handy for situations where you might not really understand what you need to search for. Any search system that can ask you clarifying questions is going to be a big improvement.
Or where you need to combine several steps together but you don't yet know what those steps are.
There's probably other technologies that could do that, requiring lower resources but they'll come with different trade-offs around configuration.
Just having a Raspberry PI, a offline copy wikipedia and a RAG enabled small LLM would be quite useful or at least entertaining if you have to go off grid.
The search engine is indeed the last missing component from a sovereign stack. But I think this could be solved locally with little cost. Instead of indexing content on the web we should be indexing sources themselves - where to look for X? - like forums, blogs, docs, feeds, and specialized search engines. We could collectively amass millions of these search stubs that can be used by local models to go and fetch fresh information from the source directly. This means separating the routing layer from the information layer, we don't need to keep information cached from the whole internet locally. The search stubs could fit in a few GB about same size with the local LLM. The cool thing is that sources change much slower than information itself, so the search stub database could be refreshed at a slower pace. We could combine a few million generic stubs with a few hundred personal stubs generated from our own activities. It is trivial to generate these stubs by piggy backing on frontier models.
Knowing humans? They'd probably take it by force and run it for themselves instead of providing light and heat to surgeons and water sterilizers...
/daily dose of cynism
That way your machine that, eg, normally plays video games or does AI work can support relief efforts by supporting emergency response IT. You don’t need to mothball the machine, just have an “emergency” boot USB than can run the services from your home generator.
You don’t even need to bring it with you: turn it on and leave it “best effort” at home, while you continue to use it via WAN.
But OK, let's assume that: The power is out, but you have a generator with so much fuel you can run a desktop just fine; Your neighborhood will somehow make a mesh network; Your neighbors need some already stored information and the best solution for that is texting a chatbot rather than a survival/emergency handbook or Wikipedia; Your mesh-network will also be good enough to match the time-sensitivity of the questions.
Under those assumption, which of these sounds better?
1. Buying an "LLM-in-a-box for emergency supply kits", which you deploy so that your neighbors can ask questions (text over the mesh) of the offline chatbot.
2. Buying a satellite internet transciever for your emergency supply kit, so that your neighbors can ask questions of a much better chatbot and communicate with human experts, their worried relatives, and coordinate with rescue/relief efforts...
I’m only out the cost of the drive, which is like $40 and doesn’t require anybody on the other side cooperate with me.
- - -
More broadly…
You call it unlikely mixes, but we see it all the time:
- people already have a computer for gaming or work
- people (ie, “preppers” like we’re discussing) buy a generator for emergencies
- local emergency response sets up mesh networking during disasters, both official and unofficial
Have you ever tried to use a handbook you’re not intimately familiar with during an emergency? It’s rough.
For personal preparedness, nothing replaces familiarity and practice — eg, weekend survival trips and reading your manual ahead of time.
But for providing information in a random lookup manner to unpracticed people who weren’t prepared? Yes, I think an LLM/chatbot is the practical way to operationalize all that information which you stored (eg, survival guides or machine manuals).
Also, it’s unlikely a general purpose chatbot would be superior at survival advice to one specialized for that purpose — and indeed, is likely to refuse your questions as “unsafe” or “criminal”.
At current prices you are also out about $4k for a Spark to actually run the inference on, if you want a full LLM in a low-power package.
In general, I'm not sure why one would want to pin your survival to an expensive, hallucination-prone data source, when an offline copy of wikipedia with a little vector search attached to a Raspberry Pi can fulfil the same role...
Sounds like the absolute worst time to rely on a crappy little model that will inevitably hallucinate.
But, the current model you really want for an emergency kit is Gemma 4 12B QAT 4-bit. At ~7GB on disk, it's small enough to run on a tablet or any modern computer, slowly if you don't have a GPU or modern Apple silicon, but exceedingly smart for its size, excellent vision capabilities, good tool user, surprisingly good reasoning.
Put that on a spare phone
I think it was on HN at some point too. Basically it's a bunch of offline data dumps (wikipedia, maps etc) with a local AI model on top that you can use to query them with natural language.
Maybe someone should be making this, but for rebuilding society in the event of a disaster - a solar-powered black box with most of humanity's knowledge within. Even something running one of the Qwen models would be useful.
"So, we had a nuclear war and need to start from scratch. How do I turn this rock into a computer chip?"
Print important knowledge on paper and store it in a desert. in 2000 years society and population will advance enough to get a jump start based on our knowledge.
This is a bit of a straw man, TBH.
For one thing, "LLM-in-a-box" doesn't necesssarily imply a device as small as a phone.
For another, you'd need to convince people that the iOS Foundation model is the "frontier" of LLMs that run on phones when it is really not. AFAIK it is noticeably outperformed by the Gemma 4 E2B model and certainly the E4B.
https://blog.google/innovation-and-ai/technology/developers-...
Here is a common-or-garden youtube video that includes a demonstration of how much better the E2B is:
https://www.youtube.com/watch?v=sTxyBUbdZcA
Whether this idea (LLMs for emergency/survival scenarios) has value, I don't know, so I am not offering an opinion, but you should approach it with a good faith argument.
I am an LLM cynic but I suppose if I was to be without connectivity but with power for a while, a device with the Gemma 4 E2B or E4B model on it might be helpful or interesting to have. If such a device had the 12B QAT model on it, that really would cross the line to utility. Not sure it has value in the OP's scenario, still.