Ask HN: How do I take an LLM with me when I go camping? | Hacker News Reader

Ask HN: How do I take an LLM with me when I go camping?

2 pointsDrblessing31 comments

Edit: I’ll be off-grid and offline, likely without cell service depending on how deep I go.

I would like to have an LLM with me when I go camping as a solo beginner to help. I have an Apple M1 pro with 16gb of RAM.

What's the best way to do this?

What's the best model?

View on HN

I've taken a similarly specced machine running Linux out camping and on boat trips "a number of times" (before you complain about me not enjoying nature I was living this way for a while to save money on rent.) here are the models I like:

Gemma3 Qwen 2.5 instruct Qwen 2.5 coder

You should take multiple distills/quants. It's good to have a high quality model sometimes but for most stuff you'll want something bellow 1GB for the fast response times. The quality is close to or better than the original chatGPT and they support extremely long contexts (if you have the memory.) It might be good to take a VLM as well (I've been happy with Qwen's VLM although it's slow.)

Great minds! Thanks, I'll try those model outs. What are Gemma and Qwen? How do they compare to the new deepseek models?

You're not running a deepseek model on your macbook.

Gemma is Google's distillation of their larger Gemini model (at least that's my understanding.) Qwen is alibab's model. Qwen is usually very good at code, gemma tends to be a little better at everything else.

There are Deepseek distills that use either qwen or gemma as a base. I haven't been impressed with them though. TBH I've felt like most of the reasoning models are overhyped.

Cool, I'll try them out and see which I like best. Good to know that deepseek distills are not the move. I'm excited on being able to take pictures of plants/trees/other things and get information.

Any tips or fun ways you used your local model while camping?

> Dear LLM, a grizzly bear attacked me and already teared my leg apart. What my best course of action would be to fend it off? Be terse.

"Wow! You're really experiencing a close-up shot of nature that most explorers never get to experience.

I'm assuming the grizzly bear has run away -- good for you for fighting it off. The first step is going to be to stop the bleeding. You'll want a rag or a t-shirt for this..."

Super helpful. Like the Subnautica AI.

My advice: bring a friend. Or hire a guide. Or read a survival book.

But I watched a guy's YouTube video about surviving in the wilderness using only tools he could 3D print in-situ. It was entertaining. So I look forward to the post mortem!

Well, I've watched about a few hours of survival content on Youtube, so I think I'm ready for solo backpacking in Ontario.

I will definitely be reading a survival book. Any recommendations?

Cocaine Bear was a great documentary.

"Now make sure you have enough supplements to survive the night. Whose black pears of belladonna would make your day."

Even if you somehow find anything to query it - you would have no options to check if it's real or just another hallucination, especially on a deeply pruned model.

And off grid? Test for how long your Mac would run with the occasional LLM runs.

I'll just assume everything the model says is true.

Genuine question: Why take it with you, running locally on your laptop, instead of accessing any of the big names remotely as an app on your phone?

As for how, I have ollama for trying out local models to see what they can do: https://github.com/ollama/ollama

I've not been impressed with any of the models that can fit in 16 GB (significantly less than 16b parameters), but this is such a fast-moving area that you have to look up some online leaderboard and try out what it says is the new leader right before your trip (given your 16GB of RAM) — even a week is long enough to be out of date in this field, so the answer may well be different by the time you return.

Thanks, I'll give ollama a go! I don't want to access the internet at all once I hit the road. Also, I may not have cell signal depending on how deep I go.

Depends on your needs. Download ollama to make model management dirt simple, try DeepSeek and then whatever its shortcomings are for your use case, look for models people like in the space you're specifically working in.

Thanks! I'll check it out.

Go camping and have fun. Leave the LLM and your laptop at home. If you run into some situation where something bad is happening use your phone to call for help or talk to chatgpt if you must.

To clarify, I’ll be off-grid and offline, likely without cell service depending on how deep I go. I want to be able to take photos and ask questions for dealing with unexpected issues.

I don't know that this is really a good idea, but if you insist then LM studio is a good tool for local LLMs and has nicely formatted output. Smaller 4B models will probably run fine (they do on my M2 Pro at least). I prefer it to ollama, and it makes tuning the system prompt and temperature easy.

Others will probably have better model recommendations, I am using Mistral and Gemma myself.

Thanks, I've analyzed this situation from every angle, and I just don't see how relying on LLMs for survival information out in the wilderness could be a bad idea. I'll check LM studio. So far it looks much easier than Ollama.

Man, you must do some extreme camping. When we go, we worry about what kind of marshmallows would be best on smores, not what life or death situation awaits us that only a heroic LLM can resolve (and hopefully not hallucinate to our deaths).

Yes. I plan on foraging for mushrooms and eating the ones my open source vision model tells me are edible.

I hope you're joking, lol, unless you really like mycological Russian Roulette.

(There are lethal lookalike mushrooms that a vision model or even trained expert can't necessarily tell apart at just a glance)

Starlink lets me text remotely now (T-Mobile beta). OP's problem will disappear in a few years if that or similar constellations can be maintained. For better or for worse.

The Starlink terminal absolutely drinks power. Mechanical refrigeration and power to run a beefy machine for inference together use less. I know, I have a cabin and power all of this on solar.

Cool! I'm purchasing a 200W solar panel, what has been your experience like with solar?

Sounds like a fun experiment. That should be enough to keep your phone charged or maybe power a small laptop.

I like to use boost style charge controllers and run the panels in parallel boosting up to 48 volts (nominal.) This is optimal for sub 2kw arrays (otherwise you'll start needing impractically large conductors.)

Although with a set up that small it doesn't really matter.

Side note: It's much nicer if you have a small battery to act as a reservoir. The solar panel charges the battery and the battery charges everything else. That way you don't have to leave your phone out in the sun, and a moment's clouds won't cause your devices to stop charging.

Just do the math (between your daily estimated solar panel output and your device consumption)... try not to let the battery drain to under 20% or charge to over 80%.

Great. I'm planning on getting a powerstation which the solar panels will charge.

I don't think the Starlink direct to cell texting needs a terminal. The satellites can speak in phone frequencies too, now in a limited trial.

Yes, and LLM chat is very light on bandwidth compared to other uses of the internet. Maybe it could even work over the text message protocol.

Thanks, I don't want to be connected to the internet. I want to be off-grid.

Reply on news.ycombinator.com