Hacker News Reader

Comment by oneplane | Hacker News Reader

Yeah, there are a lot more parameters here than just 'there are two chips and one is the bestestst', like the availability you pointed out.

There is raw performance, but there is also performance per watt, availability and scalability (which is both good and bad - M1 is available, but there is no M1 Ultra cloud available). If you want a multi-use setup, an RTX makes more sense than most other options, if you can get one and at a reasonable price. If you want a Mac, the M1U is going to give you the best GPU. In pretty much all other setups there are so many other variables it's hard to recommend anything.

View on HN

For the market this is aimed at, performance per watt is really irrelevant. Performance per dollar or just outright performance are far more important the vast majority of the time. That's how we ended up with 125w+ CPUs and 300w GPUs in the first place.

There are dedicated ML cards from Nvidia for that, far most powerful than a 3090, so that is indeed true. But PPW is never irrelevant when someone is doing things at scale, so the question becomes: who is doing this for money but somehow not at scale?

These aren't rack mount products aimed at cloud providers, they are essentially mini workstations. What are you calling "at scale" for this? You're basically always pairing one of these machines to one physical person sitting at it whose time is being paid for as well (even for a solo creator, time is still money). It's a terrible tradeoff to save pennies per hour on power to turn around and pay the person dollars more per hour waiting on results.

That seems like a really bad way to spend money. Why limit a person to a single workstation if the workstation is the limiting factor? This is where we get clouds or if it must be done locally, rack mounted systems with many cards.

If you are doing it solo, with "just the hardware you happen to have", it matters a bit less. If you are doing it constantly to make money, and you need a lot of power, buying a one-person desk-machine makes no sense.

The cost of powering my 3090 for a year is now more than the cost (RRP) of a 3090.

Where do you live that power is anywhere close to that expensive? And are you overclocking your 3090?

Even assuming literal 24hr/day usage at a higher, "factory overclocked" 450w sustained, at a fairly high $0.30/kWh that's $1200/yr. Less than half the retail price of a 3090. And you can easily drop the power limit slider on a 3090 to take it down to 300-350w, likely without significantly impacting your workload performance. Not to mention in most countries the power cost is much less than $0.30/kWh.

At a more "realistic" 8 hours a day with local power pricing I'd have to run my 3090 for nearly 10 years just to reach the $2000 upgrade price an M1 Ultra costs over the base model M1 Max.

UK electricity is $0.28/kWh but will be $0.36/kWh from the end of the month - my business is quoted at $0.60/kWh fixed for the next 12 months.

At $0.36/kWh - card alone @450W ~running cost ~= RRP 3090 $1,499

Yes, can power it down to be more efficient, however, that effectively agrees with the previous comment that PPW matters.

Wholesale electricity price in some EU states is €550/MWh today. Most EU states are above €250/MWh.

prices in Europe right now are considerably higher than $0.30/kWh

It seems safe to say prices right now are not the norm due to, you know, that whole war thing going on that is impacting one of the EU's primary power supplies. The 2021 EU average was otherwise $0.22/kWh.

yeah but let's how fast we have those "normal" prices again, maybe this is the new normal, who knows.

The war is a factor going forward, but we got notified of the price increases before the war started, they relate more to the piss poor planning of our rulers than any external factors. Unless the people in charge are going to suddenly start planning for the future this will be the new normal.

But do you really run it all year round?

> performance per watt is really irrelevant.

Watts are dollars that you'll continue spending over the system life. It matters because you can only draw so many amps per rack and there will be a point when, in order to get more capacity, you'll need to build another datacenter.

The market is not data center use.

You'll still spend another Mac Studio on energy in order to run a comparable PC for the next five years. To say nothing about not wanting to be in the same room as its fans.

What are you talking about? This is not for cloud datacenters trying to squeeze every bit of compute per resource.

These machines are commonly used by professionals in industries like movie and music production. They don't care what the power bill is, it's insignificant compared to the cost of the employee using the hardware.

> "They don't care what the power bill is"

Oh... They do. At least, they should. If a similar PC costs $500 less but you spend $700 more in electricity per year because of it, at the end of the year, your profits will show the difference.

As I said, these numbers are so insignificant that they don't matter. The cost of the employee and their productive is several magnitudes more.

I ran a visual effects company a decade ago. We bought the fastest machines we could because saving time for production was important. The power draw was never a factor; a few catered lunches alone would dwarf the power bill.

Note that Geforce RTX on cloud is prohibited by Nvidia.

Yep, that's true. You have to use the DC SKUs which (IIRC) aren't the same silicon either. Worse: some of the server SKUs are restricted for market segmentation where your ML and hashing performance is bad but video is good (and the other way around).

The silly thing about it is that most of the special engines can now be flashed into an FPGA which is becoming more common in the big clouds so special offload engines aren't that big of a deal when they are missing. So in some cases you can have your cake and eat it too; massive parallel processing and specialised processing in the same server box without resorting to special tricks (as long as it's not suddenly getting blocked in future software updates).

The part of the EULA which is supposed to enforce that is not enforceable in Germany. It is complicated. There might be other ways you can circumvent agreeing to the EULA based on your location.

How do they functionally do that? I googled and found this?: https://www.nvidia.com/en-us/data-center/rtx-server-gaming/

Honestly asking because I’m kind of out of the nvidia loop at the moment.

Technically: the driver detects if it is run in a virtualization environment, it is at least able to detect KVM and VMware. On the upside, it's relatively easy to bypass the check.

Legally: I assume no cloud provider will assume the legal risk of telling their customers "and here you have to break the EULA of the NVIDIA driver in that way to use the service". In Europe where the legal environment is more focused on interoperability, this might not be as much of a problem, but still it may be too much risk.

They disallow such usage for "GeForce" by proprietary driver's EULA, and they limit open source driver performance (IIRC they require signed binary blob).

Reply on news.ycombinator.com