Hacker News Reader

Comment by sigmoid10 | Hacker News Reader

This thing is absurd. I work in ML research and I use my Macbook Air M1 to train LLMs from anywhere, but I would never let them touch my local machine. Even if they'd run (most of them wouldn't - even on a 3080 Ti), they would ruin my battery life. So I simply connect to a GPU server via VPN and ssh + vs code remote. Only students or beginner enthusiasts might think that training this stuff on the go would actually need a beefy laptop.

View on HN

I agree with you. I bought an at home GPU rig 5 years ago, and it was good for a while. Then I realized how much time I could save myself by paying Google Colab $10/month for improved GPU access - close to zero config hassles. Running in Jupiter style notebooks is not optimal, but it is good enough. I am in my 70s and maybe it is because of my age, but I look critically at all my activities and always ask “is this really what I want to be spending time in today?”

> I am in my 70s

You sir are very inspiring

This product is AI/ML equivalent or "gaming" or "military grade" products, all of which are just over-priced crap with RGB/Camo added.

Is it though? I personally prefer local development and test out my code instantly, especially when internet is flaky. Once you have confidence in your code, you train it in the cloud.

Develop on a tiny slice of you dataset locally of you must. Then train properly elsewhere (in the cloud of your choice)

I can always test out my code instantly. With the vs code remote ssh plugin it's exactly like running everything locally, but in reality only the GUI runs on my laptop. The necessary connection speeds are tiny compared to what you'd need on the go to sync models and datasets.

I develop my code directly on the HPC cluster. What do you even mean with testing your code instantly? You don't have to run a batch job every time you need to test something. You can use interactive mode.

You will probably keep your dataset on Cloud storage and Google Cloud has much higher Network speed, it makes things much faster than working locally

Obviously if you have your own local data center then you would use that

You could do this via Jupiter notebooks and visual studio code.

Transformer adapters/fine-tuning would be a good fit, especially while traveling with a power source next to you.

By that reasoning, why does the M1 even need a GPU?

You couldn't even watch modern youtube videos on most cpus were it not for integrated accelerators. The M1 does exactly that, but in a power efficient and reasonably performant way.

M1 chip is both a CPU/GPU, all computers need at least a limited GPU

You need some sort of GPU to run any OS in a decent way

Reply on news.ycombinator.com