Hacker News Reader

Comment by jon-wood | Hacker News Reader

I’m not usually one to ask this because learning to do a thing can be fun, but why exactly have you spent 25 thousand dollars on getting an LLM someone else made to answer maths exam questions?

View on HN

The cost is obviously not that big of factor for OP as it might be for others. It's actually refreshing to hear the candid viewpoint that he expresses here.

25k is definitely a lot but I did the risk analysis and I figured worst case I would lose a 1000-2000 after a year of playing around with it, so I look at it more like renting (I'm going to keep the Macbook Pro no matter what since I needed a new one).

Nitpicking, but the worst case of spending $25k is unforeseen circumstances that write off the entire asset. I don’t think -$2000 is a conservative enough figure for standard depreciation either (a lot can happen in a year)

Either I don't understand the used apple market.. or I agree this is crazy. Someone spends $25k on new hardware, waits a year, and expects to sell it for $23k? Unless the ram issues save him, and cost of new goes up, I don't see how that was going to work.

Well, Apple is literally not offering the M3 512GB studios currently. You can’t even back order one.

They are selling on EBay for over $20k, used.

It’s hard to know if any of these eBay listing are real or actual sales. Lots of scams.

The ones sold for $25k from established sellers are legit. Filter by "sold."

The 0-reputation account in Spain selling an M3U 512GB for $4200 is 100% fraud.

oh young grasshopper, I see you dont know that money launderers love the ebay hype cycle. Its REALLY common on high dollar hot items to have phantom transactions where parties are on both sides of the transaction to clean illicit money. The high price tag and high volume amount of transactions hides the illicit signal. I have tried to buy a few of these mac studios only to have the transaction cancelled because I wasnt the dirty money on the other side.

Funny, they have the "honesty" to cancel the transaction and not take your money, just to keep their ebay reputation high ?

eBay and PayPal almost always side with the buyer.

The transaction was cancelled? Sounds like you weren't defrauded.

GP didn’t say they were defrauded. They said the listing was a cover for laundering money.

"weren't scammed" might have been a better choice of words, which they said one post up.

I said there are lots of scam listings. And then discussed a second issue which is fake listings that are designed to launder money. There are both. Some of the fake listings are obvious: way under priced, listings for “original boxes” not the actual computer. So that’s why I don’t want to buy off of eBay for this category. But I did also buy what appeared to be a reasonable listing and it was canceled. I can’t prove why but two sided money laundering is a known thing

Ah, I see what you mean now.

Bizarre. They don't care that eBay takes 14%?

14% seems like a pretty low fee to clean drug money, if we're being honest

Traditional money laundering loses upwards of 40%, so hell yeah.

What am I going to do with 40 subscriptions to Vibe?

You could almost sell a RTX 3090 for more today than what it cost brand new when it came out six years ago

It's still very contrarian to expect GPUs won't depreciate rapidly. Yes 3090s were a good investment then, but way worse than just buying Nvidia stock directly

How did we go from "I expect to lose only 1000-2000 if I try to sell my used equipment" to "you should have just bought NVDA to get a better return." The point wasn't the better return, the point is that I wouldn't lose all my initial investment if i decided I wanted to sell it.

And the fact of the matter is that in 2026, all electronics has gone up, not down, and sought-after GPUs have gone up in price in the used market.

Waiting for them to come down any day now. Been waiting since 2017.

First it was crypto, now AI. Just because the market can stay irrational for very long doesn't mean crashes don't happen. What nobody knows is when.

at some point you have to accept that the market is actually rational

Yup, just like crypto and tulips were rational behaviors for the global economy. Or investing 5% of world GDP in half-baked AI is.

wanting to get rich quick is completely rational if you ask me ;)

Only if they do get rich, if they lose money, it's stupid. Technically stupidity is also rational? The bottom of rationality? :-))

This is the case in lots of markets, e.g. look at used cars, luxury goods and more. Some of it is driven by inflation/the rapid devaluation of the dollar. General and AI-adjacent compute in particular hasn't come down in price in a long while.

A lot of expensive things hold their value well. I have a friend who is really into telescopes and he now owns a $100,000 telescope but he didn't directly buy such an expensive scope. He started out with much cheaper ones and was able to sell them for about what he bought them for to help fund more expensive ones over 20 years. It is really interesting.

Thank god. I’ve been waiting to have a reason to tell someone I collect fountain pens.

Computers depreciate because they are obviously being supplanted by newer better models—until they become vintage and then move into collectibles.

Apple products have had relatively high resale for a while. Only losing 8% in a year is probably extra unusual, and 1-year-old wasn't really ever the sweet spot, but a "sell used privately after a few years, roll onto the new one" has been a relatively common play.

Doing this particular one is definitely expecting the market squeeze to continue. "Worst case" is back to more "normal" depreciation. Where I'd expect to only be able to recoup more like 18k. But... if you look at GPU prices the last 3 years... it's not a crazy assumption that it won't drop that fast.

iPhone example since those are easiest to find in quantity: new iPhone 16 Pro Max for $1200, Gazelle would want $866 for "execllent" condition. Lost ~28% for one-model-back. iPhone 15 Pro Max, though: excellent priced at $667 here, only down another 23%, and gives you basically half-priced-upgrade if you can sell it for that and roll into the newest.

So to have never-more-than-one-model-old rough estimate at today's value-holding you'd be out $3600 for three new phones, with getting 1732 of that back, or 1868 for it (with a $334-per-year incremental cost of upgrade).

For never-more-than-two-models-back you'd be out $2400, getting back $866, for net $1534 spend, with a $167 incremental per-year upgrade cost once you buy the first one. Pretty good if you keep the phone in excellent condition and are happy to budget a bit over $10/month to be on a every-two-year upgrade train.

Well, you'd also eat the tax...

https://buy.gazelle.com/products/iphone-16-pro-max-256gb-unl...

https://buy.gazelle.com/products/iphone-15-pro-max-256gb-unl...

What you describe is something people with enough to make the first purchase and eat the cost when it breaks have been doing for years e.g. with cars. People on the lower end of money scale tend to use products for well over their economic lifetime saving way more and buying a cheap replacement if it breaks. Notable exception as stated being phones for some reason as it likely is a status symbol for more people in a [insert preferred external sexual characteristic] measuring contest.

> "Worst case" is back to more "normal" depreciation.

I would absolutely not count on that, if and when it drops it will drop hard.

> I don’t think -$2000 is a conservative enough figure for standard depreciation either (a lot can happen in a year)

We aren't exactly in "standard" times and haven't been for quite a while. Even five year old graphics cards are worth more today than they were just a year ago. Things will obviously depreciate at some point, but you gotta throw your existing notions of how quickly and how much hardware will depreciate out the window. There's just been too much money dumped into AI for a "well I guess this won't ever pan out, let's dump all this hardware to recoup our costs" moment to happen and tank the price of everything suddenly IMO.

And that's not even getting into the other geopolitical stuff going on right now. Strange times.

I wouldn't call this nitpicking. This is how people who are careful with money think. I learned embarrassingly late to stop justifying purchases by making predictions about future returns. I treat everything as having zero value as soon as I purchase it. Thinking otherwise is, for me, always a dangerous rationalization -- always a craving that's trying to outmaneuver sense.

Aren't things like this seeing 'negative depreciation' these days?

"inflation"

"appreciation"

My phrasing was just to point out things moving in the opposite direction to what has previously been considered normal (which is also the economically intended 'normal').

Sure, they took a gamble that they wouldn't be able to sell it used.

If you are able to tie up $25k for a few years just for shiggles, you clearly are able to make do fine without that money and if lost it would be at worst annoying, not catastrophic.

I assume he is calculating the loss as depreciation - what they would have spent on cloud bills if they hadn’t been doing this locally.

I mean whatever. It's workstation/server class hardware, that's how much it's been for a long time

Was this risk analysis just AI slop?

I think op would make a really good pope too.

https://news.ycombinator.com/item?id=48118672

I didn't spend that much, only $6500 AUD for a GB10 based Asus GX10 which is even slower than OPs, but I spent that because it makes for a great learning platform. Theres not much else that lets me fiddle with 128GB of RAM for my graphics processor, and it's quite lovely to be able to run things as long as I like without worrying about my cloud instance being shut down.

It's not financially a good idea: renting really does beat owning, and cloud beats both if you're only running inference on these machines. But I'm not just doing inference, and as a thing I can do silly stuff on to learn, it's hard to beat!

When you say you are not just doing inference, you mean you are also training your own llms? I am curious what other things can be done.

Fine tuning, and yeah training my own, experimenting with architectures and learning how it all works. Been a lot of fun

$6500 AUD can get you a good chunk of B200 time on any of the GPU neoclouds :)

Less than I expected, though! And I get to run this all through the night

I do still use Vast and Runpod for things too, but it’s much nicer to test a fine tuning run here to make sure I’m in the ballpark

I also did literally say “It's not financially a good idea, renting is better than owning” so I’m confused why I have two people telling me that

Also it’s just far more fun to play with something tangible to me :)

You could just rent a bare metal server with those specs

Yes I could, but that is annoying because of spot pricing and having my instance shut down, and it has fluctuating prices

It’s also annoying because then I need to make sure my little “lab” setup is well automated, and I’m lazy :)

Also, I literally said “ It's not financially a good idea” so I’m confused why you think I don’t know that.

Spot pricing and instance availability don’t apply to on metal hosting. You’d have your own machine dedicated to your own use only, at a locked in price.

Wait, I went and looked far and wide today and I can’t find anything with ~100GB of VRAM that isn’t $20k a year, what am I missing

That's too small. I was quoted a machine with ~1.5TB of vram, for $10k/mo. This was the minimum node size in the AI data center I was talking with -- they don't make smaller nodes that you can lease as bare metal. There is no public pricing though, and you had to know people to get in.

> renting really does beat owning, and cloud beats both

Privacy and offline operation are valuable or non-negotiable in some cases, but the difference is pretty categorical between what can run on a single card and what can run on a DGX GB200 NVL72 cabinet. Doesn't mean it's not worth seeing how far local models can be pushed. Not every problem needs a senior engineer.

I know it's one of those "if you have to ask" situations, but curiosity got the better part of me. Here's the search assist response:

"The DGX GB200 NVL72 AI server costs approximately $3 million per unit. This system includes 72 Blackwell GPUs and 36 Grace CPUs, making it one of the most powerful AI servers available."

The search assist actually credited a source used with: https://www.tweaktown.com/news/98292/nvidias-new-gb200-super...

That $25k spend by GGGP seems like nothing in comparison. That's ~1/3 of one chip in that cabinet. God gawd I'm old and out of touch with modern AI data centers.

By comparison, the Colossus 1 data center had 32,000 GB200s (as well as 150,000 H100 GPUs, 50,000 H200 GPUs), and they are bringing another 110,000 GB200s online (although this might be Colossus 2?)

There are bigger data centers than Colossus 1 around too.

There is a reason NVidia is the most valuable company on the planet.

https://en.wikipedia.org/wiki/Colossus_(supercomputer)#Curre...

It's The Circle of Computing Life. The pendulum swings between centralised mainframe timesharing-for-hire and desktop individuality.

We've been in a centralised phase for longer than usual - first cloud everything, then AI - but at some point in the next decade prices will crash and a market will appear for personal, local intelligence.

Depends how local models keep up

> the difference is pretty categorical between what can run on a single card and what can run on a DGX GB200 NVL72 cabinet.

A better way of putting it is that you can run plenty of things on a single ordinary system, but you may be disappointed at the performance. Generally, you can't expect inference to be as quick as with cloud for SOTA-like models. You have to run smaller models for quick replies, and large models with a lot of real-world knowledge for less time-critical inference, possibly batching many requests simultaneously to improve throughput.

One year ago finetuned local LLMs had a significant edge over ChatGPT or Claude. Look up in YouTube all the DIY videos testing LLMs on their own machines with different setups.

Remember: one year showed up to be a gigantic leap in regards to quality of results and innovation in the AI space. Agents weren't really a thing and vibe coding wasn't even invented as a term because the top notch tools at the time were lousy, with lovable being the frontrunner with its - in my view - sorry Tailwind recombination tool shaming AI to do the work.

Then fall hit 2025 hit us, new year's eve and suddenly there was such a massive surge of innovation and competition with ChatGPT Codex suddenly showing up.

Remember: one year ago many now commonly used tools weren't yet available like Nano Banana or Codex.

"The 25k are so vast" - Yes, and no. For example, if the machine is bought for business usage I can deduct the costs from taxes. This roughly amount for 50% of the financial burden.

So I jokingly use to say, that I pay only half the price for my Apple business machines. And yes, I am strict in this regard. Business means business. No private emails etc. nothing on my company computers.

Maybe there are other options as well to reduce the financial expenses the dude mentions, but it doesn't seem so.

I would also go for leasing, this way already the monthly payments can be deduced and I don't need to buy and maybe resell the machine.

Apple is a luxury good. Without business usage or at least partly using it for business as well as private (mixed usage in tax reports) I wouldn't buy the devices or think twice.

Apple under Cook evolved into a Gucci like luxury brand, that is more and more a rip off than quality delivered, especially considering the latest OS updates for Mac, iOS and iPad. Apple is a mess, following Microsoft Windows' footsteps happily, because the CEO is as has been correctly assessed, no product guy.

But I stop with my rant here.

Always try to use tax deduction as leverage for your computer expenses. Every citizen should invest in basic knowledge about that.

Even a 10-20% professional usage for work (mixed usage) gives you a noticeable advantage over normal pay.

It's just a project I'm working on. I'm working on projects where AIs are processing and classifying large amounts of data that would be a lot of work for humans to do.

I think of LLMs as being well equipped for handling dynamic data or adapting to unforeseen circumstances well (random code requests, website's ever changing layouts, typos, non-standard formatting in docs, groking out important info, etc), but math problems are be definition a very specific set of instructions to run, so is the overhead and "thinking" aspect of a LLM/AI even needed here? I'm genuinely curious, btw, I'm not asking sarcastically. Can't these math problems just be yanked from some test file and rapid fired directly at a gpu/compute unit?

> Can't these math problems just be yanked from some test file and rapid fired directly at a gpu/compute unit?

Yes this is exactly what I'm doing. I isolated the actual math question, and then sent it to my two servers to process and that's what's taking 10m+ to return. I'm asking them to solve the question and return the full answer along with their steps. I care about correctness so taking time is okay but I can't use 10m per solution.

Nono, parent was asking “They’re bad and inefficient at that, so why have an LLM do math? Why not just use some code and the CPU/GPU that’s already good and efficient at basic math?”

This is making me feel a lot better about my plan to lease a $25k EV simply because it's available at a massive discount. I'll probably end up using less electricity, too.

That hardware is costing him ~1$/hour over 3 years. Presumably having it answer math questions was a tiny fraction of what he was using it for.

I’ve spent twice that on hosting movies and tv for Plex, so… I think they are worthy of my praise. What a healthy outlet for money.

You spent 50k for plex hosting? Why so expensive?

Half a petabyte of RAID6 is the biggest line item, then the redundant 40gb networking and compute follow closely. I have a lot… too much even?

That’s a lot of blurays…

I found them dumped out in the “street” in a place ordained by law as public domain. So I just grabbed up the media and use it in private.

How many Blurays are we talking about?

Can't reply to the other poster, but I have 4K HDR Blu-ray copies from discs I found in the street too, which are more in the 60GB ballpark.

In what kind of "streets" you guys are hanging around?

500 TB/25 GB = 20.000

if some have more than one layer it could fewer but that's the order of magnitude

If each Bluray is 2 hours long, that's 4.5 years of nonstop watching.

Just parallelize watching.

I have 50 agents watching blurays for me 24/7

Tokendealers love that.

does one really need 40gb networking to stream bluerays?

I am (clearly) not as far down the rabbithole as the commenter you're replying to, but almost certainly not. Streaming 4k blueray is on the order or ~100Mb/s, which means on a LAN bog-standard gigabit ethernet and associated networking hardware would be more than sufficient.

This is taking a hobby to its extremes, in much the same way that a $5k boat and $500k boat let you catch the same fish.

It’s about being able to rapidly move files between the arrays and future proofing.

You are totally right, it’s mostly just for backups and transfers to rebalance data.

Dam this is just art in this scale

Because buying Macs is not about performance, its about feeling like you are rich.

That money could have been spent on way more bang/buck performance in the form of a set of 4 graphics cards.

Also I would probably put the odds 70:30 that Apple marketing is astroturfing on HN from the amount of posts about running llms on Macbooks, because in reality, the inference speed of any decent llm is unusable on a Macbook despite the ability to fit it into RAM.

40-80 tok/s is unusable to you? Ok.

If you like having a box with 8-12 fans blasting hot air and noise into your office all day, nobody's stopping you.

If you are not being paid by Apple, I feel sorry for you. Cause that means you are so bought into the cult that you are delusional.

the 40-80 tok/sec is only for initial prompt processing, and with the "medium" models, like Qwen3.6:27b. The actual token generation is in the 10 token/second Thats very slow. And your Macbook pro will stop being a LAP-top, because it will get very warm.

Meanwhile, my 2x3090s happily crank out ~100 tok/sec generation. Oh and I can run 100 tok/sec on my phone as well, because I can just access ollama on my home desktop over ssh from termux.

Please, allow me to show you how far I can pee. It's several feet more than you can and a much mightier stream to boot.

Or it could have had way more bang/buck by feeding a family of real brains for a year or two

Excuse me for this comment, really, but I can't comprehend the absurdity, some people are buying GPUs when other people have no money for insulin so they literally die. I don't mean anything towards op or gp, quite the opposite I'm truly happy they have this kind of freedom, it must feel really nice, I just hate this game so much.

You and me both

Reply on news.ycombinator.com