Hacker News Reader

Comment by StarlaAtNight | Hacker News Reader

how quick does it respond? what are specs of your laptop?

I tried it on my M1 MacBook Pro. It's slow but surprisingly smart as a general purpose LLM. Maybe GPT-5.3 level. I gave it a bunch of tools and it can search the internet, make product recommendations, document, code, etc.

Had the same reaction so had Grok create a script to:

- find a free GPU droplet on digital ocean

- fire it up

- pull in a snapshot of the model + extra files/packages etc

- set up a ssh tunnel so that the localhost:8000 routes to the above

Then I just configured OpenCode to use the above and was off to the races.

Works out to be about ~$2/hr all said and done which isn't bad as I only pay when I'm using it (but could get expensive with 24/7 running)

Awesome, I used Claude to write a small python script to do the same with Linode's API. The only difference is I setup a persistent drive, and with Linode you can boot off of it. So my biggest start up lag is ~ 2 minutes to deploy + boot, then maybe 2 more to warm the model.

I actually dislike LLMs. But I'm a realist, and on-demand compute like this is massive cost saving measure.

(persistent drives are relatively cheap, compared to a box with several GPUs.. or even one. I find it worth the expense)

> the only difference is I setup a persistent drive,

Going to try this out vs the snapshot!

I also really like this experiment b/c it's a mix of LLMs and old school IaaC/DevOps.

> Works out to be about ~$2/hr all said

What GPU you end up with for that price? Vast.ai (https://cloud.vast.ai/?priceInstanceHourlyMax=2) has a bunch of setups available to reach 192GB VRAM under $2 :) Quick skim showed 4x48, 2x96 and 8x24, all for under 2 buckaroos or around there.

It's actually a bit less than $2 but I rounded up just to make the math easier.

And yeah, did a lot of work with Vast AI at a past job and it's pretty wild the variety of prices/hardware that they have.

> It's actually a bit less than $2 but I rounded up just to make the math easier.

That's cool, what actual GPU though? I'm still curious :P

Does it need to respond fast? For important applications, I'm sure we'd all be fine waiting 20 minutes for a high quality, usable answer. Or is it the need for interative refinements that make speed relevant?

It requires patience but it’s more like waiting 5 mins for it to do tasks. You need to be much more involved though and do things slower than Claude where you can trust it to do a lot of tasks at once. It doesn’t have the context for that

if you are so sure about what the final shape of your output is then its prbly not a common use of ai

If you are sure about the final shape of your output it's a great use case for AI, as you can define what you want in your prompt and refine towards it!

It's where you don't know the end state you're looking for that you'll end up generating slop on top of slop and creating a whole Gastown just to power your Gastown.

Reply on news.ycombinator.com