Hacker News Reader

Comment by learndeeply | Hacker News Reader

This applies for all neural networks. Depending on how much money you're willing to spend, in descending order: DGX (computer with 8 A100s, $150,000), A100 (80GB, $15,000), A6000 ($5000), RTX 3090 ($1000).

View on HN

I'd consider renting these, especially if you are just getting started:

https://docs.paperspace.com/core/compute/machine-types/

1 x A100, 80GB is $3.19 / hour. 8 x A100, 80GB is about $25 / hour.

They have much less expensive machines. I used to use them to run steam games, but now proton is just too darn good. Their low end machines are OK for CAD software that supports real-time raytracing.

Lots of good options for cheap GPU clouds, including Paperspace (mentioned above), Coreweave, and Crusoe Cloud (crusoecloud.com). Crusoe Cloud's angle is that our GPUs are powered off otherwise wasted energy and are carbon-reducing; running one for a year provides an emissions reduction equivalent to taking one car off the road.

Disclosure, I'm head of product @ Crusoe Cloud. Feel free to ping me at mike at crusoecloud dot com if you've got questions or feedback.

How can this be carbon-reducing? That sounds like a dubious marketing claim.

TL;DR: we run data centers on-site at oil wells and take natural gas that would otherwise be flared (it can't be economically transported as natural gas in a pipeline or turned into electricity and transported) and combust it completely. Methane is a significantly more potent greenhouse gas than CO2, so it ends up being a net reduction in emissions vs what's currently happening.

https://www.crusoeenergy.com/digital-flare-mitigation has some more information.

Is flare combustion more incomplete than your generation?

on the other hand, does that mean you're obligated to be in oil producing countries? how does the geopolitics of that work?

Cool idea. Tried to join your waitlist but the form throws errors. Could be my security measures, but I’m on mobile safari, toggled off all content blockers and still had an error.

Odd, I just tested on Safari mobile and it went through without issues. Mind sending me an email at mike at crusoecloud dot com and we can get you set up?

How do you carbon offset?

We have a bunch of diseased pine trees. Assuming we don't have enough time to cut them down and burn them in the next 12 months, and that I am unscrupulous, I could sell you carbon credits through one of the bigger exchanges. You would never know, and it would be completely above board.

John Oliver had a particularly depressing segment on this recently.

(Totally off topic, but I'd love to biochar them instead. Using them for lumber and shipping them offsite are non-starters. Any ideas, anyone?)

See above post, but the TL;DR: is "we capture methane that would otherwise be flared."

Agreed that a lot of carbon offsets look like, "we were going to cut this section of the rainforest down, but if you pay us, we won't do that for X period of time." This is _actually_ reducing existing emissions.

As someone who just fired off both paperspace boxes and coreweave to play around with stable diffusion, Paperspace limited me to some old janky hardware and never replied to my request to "unlock" the more modern cards.

Coreweave let me get a modern card box first go round. I'd go with coreweave if you are a weekend hacker who just wants access to a lil bit of beefy GPU.

YMMV.

Are there "mining pools" for running AI jobs and getting paid in crypto?

There's vast.ai, but you get paid in USD.

I just grabbed an A6000 for $3500 on eBay, so you can probably get a pretty decent deal on those now. They're pricey but IMO it's a great deal if you really need the VRAM (e.g. for training LLMs).

Won't 3090 prices go down a good bit after the 4xxx series cards start coming out in a couple of months?

My guess is "not as much as most folks seem to think"

Comparable 24gb vram 4xxx cards are also 1 slot bigger and many watts hungrier. If you want to be able to use a 800w PSU + 3 slots, and just need 24gb vram for say, running inference on a big diffusion model, then 3090 is still going to be your only option for a while.

They're pretty well priced right now, at $1k. If you need one, not much reason to wait, your time is probably worth more than saving a couple hundred bucks.

Is there no consumer option bigger than 24gb next gen?

It appears as though the 4090 will be 24GB, but that card may also be almost $2k.

Used 3090s on eBay are $800 all day long. That price may drop a bit in the next week or so, but not much, as that 24GB of VRAM is the main draw for that over a 3080Ti.

The 4090ti is rumoured to be 48GB [1], but who knows when that will release or how much it will cost. If you really need extra VRAM and don't mind longer inference, older used Tesla cards are an option. A used Tesla V100 32GB can be sometimes found on Ebay for 1500.

[1] https://www.techpowerup.com/gpu-specs/geforce-rtx-4090-ti.c3...

Thank you great tip on V100.

Reply on news.ycombinator.com