Hetzner GPU Server | Hacker News Reader

Hetzner GPU Server

robot.hetzner.com

249 pointsmatteocontrini157 comments

The price of a brand new Dell precision PC with a core i9 13900 CPU (24 cores), 64 GB RAM and 2X2TB NVMe is probably around 2.5k$. It costs tens of Euros per month in electricity to run this PC. The hardware is good to go for some 3 years, before running into various issues (newer CPUs becoming much more power efficient, firmware updates getting less and less frequent etc). Consider also the cost of components failure, time wasted in dealing with hardware, etc.

On hetzner it all costs less than 100$ per month. Doesn’t this imply that it’s better to rent than to buy?

Yes, the volume discounts and economies of scale they can get (while you can't) are insane.

One of the simplest is that data center energy pricing is just simply different from residential energy, because they often get wholesale (plus small markup) rather than residential rates.

The majority of residential rates (about 40 ct/kWh) are taxes and levies, not the actual price of the electricity. That's why industrial rates can be less than half of residential rates.

Can you be more specific? What kind of taxes? VAT is paid by the end consumer so you're not getting out of that one. Are there special consumption taxes on electricity in Germany?

In my experience the largest difference is distribution cost. The mandated distribution monopoly charges a lot (regulated price) to small customers.

A friend of mine is an electrician in Germany who had a lengthy discussion about this with another friend of mine who was drunkenly complaining about how large corporations pay less for electricity. I'll try my best to reconstruct what he said in response.

Residential customers use highly variable amounts of electricity and typically pay a base rate in addition to usage which can have different tiers so additional use may get cheaper if you exceed a certain threshold. Often these contracts also offer month-by-month cancellation or may include a certain period for which the price is guaranteed not to change.

Industrial customers instead often buy bulk volume commitments. Instead of paying per use, they commit to using a certain amount and pay for that exactly. They also generally have much longer running contracts. There are also contracts that directly tie the price to the daily (or in some cases even hourly) market rate, which is usually not an option for residential customers. Their energy use is also generally more predictable than the load required for residential areas with the equivalent consumption so the cost for running and maintaining the grid is lower. Additionally there is a recent trend towards DC power delivery, further cutting costs.

Rooftop solar panels and domestic EV charging actually contribute to the cost of maintaining residential grids because they add to the peak load and variance, both of which have to be compensated for. In other words, per unit of energy sold, residential customers simply have more overhead than large scale business customers.

You need to compare prices again. Residential electricity costs are significantly below your quoted rate for quite some time now. I've just checked and the lowest rate I saw was 21 ct/kWh. Most vendors are somewhere in the 25 to 30 ct/kWh bracket.

Average USA energy prices are (according the Bureau of Labor Statistics) ~$0.18/kWh at the moment, with some folks at $0.40+ (San Francisco, San Diego, Hawaii) while others are as low as $0.13/kWh (Seattle, Saint Lewis)

https://www.bls.gov/regions/midwest/data/averageenergyprices...

> The average electricity price for households fell by almost 8 percent at the beginning of 2024 compared to the annual average for 2023 and now amounts to an average of 42.22 ct/kWh (2023: 45.73 ct/kWh; base price included pro rata for a consumption of 3,500 kWh/a ).

https://www.bdew.de/service/daten-und-grafiken/bdew-strompre...

I'm at a small local energy provider. I have their most expensive 100% renewable tariff. Even after the invasion the price only went up to 28,36 €ct/kWh. Yet, my parents pay above 45 to RWE. (That's what cheap nuclear power stations get you I guess.) My father finally changed providers but thanks to the super fair contract has to wait now over a year for it to happen. And thanks to the "Strompreisbremse" I have to subsidise those ass companies.

sorry for being of topic

There’s no point in having this discussion, it is extremely specific to different locations.

Large part of their operation is in Helsinki, where electricity prices are a fraction of Germany prices.

I think "taxes and levies" is a bit true, but also you pay for distribution. If your power line goes down in your neighborhood, you basically just grumble but it gets fixed. In wholesale connections you pay for the distribution power line (+ maintenance) basically up front, but it doesn't get put in the final wholesale bill.

This isn't to say your local utility isn't wasteful with its ratepayer money; it totally can be. There isn't enough pressure to lower rates. It's just worth saying that these specific "taxes" do have an intended destination, not just general govt.

Or for some there is a national base price.

In the winter time, I get free heat from my server.

In the summer, I have to pay to run an AC to get it out of the envelope.

Gotta run some ducting for that server!

If you pay European (German) energy prices, this very well may be the case, in my experience.

For some Hetzner servers (especially from their "Server Auctions"), for domestic customers the cost of electricity alone, disregarding the hardware cost, would sometimes be higher than the rent Hetzner wants.

They build their own servers using some custom components. Economies of scale probably don't really factor into the pricing of their GPU stuff (unless I'm seriously underestimating their GPU customer base), but they do factor into the peripheral stuff like PSUs, fans, hard drives, etc.

There's some videos out there showing off their custom motherboards, it's an odd mixture of standard-ish consumer ATX but stripped down to the absolute bare essentials and with some components rearranged for better airflow in racks chassis they also build themselves: https://youtube.com/watch?v=V2P8mjWRqpk

Another video showing the custom racks and other infrastructure: https://www.youtube.com/watch?v=5eo8nz_niiM

Hetzner is not only able to get better volume discounts, they also spend a lot of engineering time bringing down costs, and have decades of experience doing so.

They offer some Dell Servers for those that really want them, but most of their servers have a custom mix of consumer hardware, server hardware and in-house hardware (for example they use their own racking system), optimized to minimize lifetime cost in a datacenter. For example most servers use datacenter SSDs, but consumer CPUs.

Hetzner is buying ASUS consumer AM5 motherboards, but with the whole CPU + memory block rotated so that the memory sticks are horizontal instead of vertical for better airflow:

https://www.youtube.com/watch?v=V2P8mjWRqpk

They also buy (at least) ASRock Rack motherboards as well.

Saying that from looking at the dmidecode output from some of our dedicated server currently with them:

    Handle 0x0002, DMI type 2, 15 bytes
    Base Board Information
        Manufacturer: ASRockRack
        Product Name: B565D4-V1L

They have lots of different boards, among others ASRock. Even within the same range of models you can buy, for instance AX52 you can get different manufacturers. It's not really transparent to the customers when they order.

Source: dmidecode

Do you have more details on their own racking system? I couldn't find anything on this.

The best resource I know is a tour of one of their data centers. Around minute 6 you get some nice views on their current-gen racks, and later in the video you see the back side, followed by the assembly of the servers.

https://www.youtube.com/watch?v=5eo8nz_niiM

It's definitely weird that cloud costs which is supposed to be commoditized is differing by 2 orders of magnitude. Hetzner/OVH seemed to have solved the hard part of making the service cheaper but just can't solve the easier task of making the platform usable. Why do I need to upload documents for verification. Why can't I try server for 2 hours right after signing up for the service.

Those are part of making it cheaper. ID verification is an anti-fraud measure and ensures that bans stick, which in turn means they don't have to invest so heavily in heuristic anti-fraud and "spam filtering" type work. A delay on provisioning means they are provisioning hardware JIT in some cases, meaning they need less idle float capacity, which in turn drives down costs.

Have been in deep contact with data centre and rental server offerings in the past, there is a crazy large amount of fraud going on, e.g. 2 hours for free means 2 hours free resource to DDOS.

I don't want it free. I want to put $2 on the account. Try the server and decide after using the server.

same problem, credit card fraud/chargebacks happening anyway, that’s why you have an intense verification process to reduce the fraud going on.

And they want a customer who will rent the dedicated box for a long time and not wipe it after every 2$/2h Trialcustomer

You can do that with Hetzner virtual (cloud) servers, just not with dedicated servers.

You're basically describing the internet-scumbag use pattern - bounce around providers and servers using nothing for more than a few hours at a time and making tiny charges to (other people's) credit cards that won't trip fraud controls because they look like verification charges.

There are very very many reasons to prohibit exactly that type of usage pattern.

You can thank crypto for ruining the internet for us.

Sure you can. I rented a server and after two weeks decided I don't like it, and got refunded all the money, including the setup fee.

Doesn’t this imply that it’s better to rent than to buy?

TANSTAAFL

1. Physical hardware is a useful abstraction.

2. At the end of a three year cycle, you can sell something you own. (This has associated costs of course).

3. Rents can go up. Terms and Conditions can change. Credit cards can expire or be cancelled. In other words, renting introduces a significant dependency.

4. You are your own most important customer. Statistically, you are not The Clouds's most important customer. When The Clouds has a fault, it gets resolved based on its business model.

Engineering decisions are specific to a specific problem and all of them come with tradeoffs.

>2. At the end of a three year cycle, you can sell something you own. (This has associated costs of course).

This is huge and often overlooked or undervalued. Renting means you have nothing when the contract ends, buying means you have something for your money spent.

A 3 year depreciation cycle may be reasonable for some spaces but not others.

I'm not going to lose sleep over not getting firmware updates; when they've got all the bugs on my stuff ironed out I don't want more firmware updates.

I haven't done the spreadsheet work to identify the cross-over point for where old systems become uneconomical but when I've done it in the past for my heavily clustered workloads it's typically better to buy an off-lease server that's 3 years old and run it for another 8 years, than it is to forklift all my infrastructure every 3 years chasing the newest generation thing.

Certainly the cloud's got some advantages, but running equipment until it's really old is also pretty good too. I'd bet that AWS doesn't throw older systems out just because they're old.

Do you happen to have a preferred source for these 3-year old off-lease servers?

> A 3 year depreciation cycle may be reasonable for some spaces but not others.

Exactly.

I am running a server at Hetzner which has a CPU that was discontinued about 7 years ago. I don't know the exact age of the machine because it was already used when I got it 4 years ago, but based on the CPU availability it's at least 7 or 8 years old, potentially even older.

Nothing on that machine has failed in the last 4 years except for HDDs (the spinning platter type), which are immediately swapped when broken, RAID rebuilds, everything's fine.

3 years is no time for hardware nowadays. It can live much longer, especially if storage is solid-state. And the performance improvements often aren't substantial enough to warrant a swap within anything shorter than 5 or 6 years.

You forgot that Hetzner also has spare parts.

And Dells have warranties.

And both offer service agreements.

And Hetzner is a straight line expense while a Dell might be a depreciating asset.

And either both or neither may offer discounts off retail.

And so on.

Because that’s the nature of actual decisions versus online arguments.

If you want to host your own stuff then you have to have spare parts in stock.

Not the same as waiting for dell to RMA your GPU over days.

> TANSTAAFL

"There ain't no such thing as a free lunch"

(I had to look it up, figure I'll save someone else yhe trouble)

thanks, I had no idea what that meant either

It used to be moderately common acronym back in the 90's and 00's. It felt like a blast from the past seeing it crop up again.

Yes.

This is what you expect prices to be when a vendor is buying in bulk.

Cloud vendors should be like "Costco" (buy servers in bulk and can pass along those savings & this is what Hetzner does)

The trick AWS (and other cloud vendors have done), is charge "Uber Eats" pricing for Costco items.

That's a really clever analogy, but is it really a trick? UberEats delivers the food to my door but if I go to Costco I have to make my dinner myself. Sometimes life happens and UberEats isn't just a convenience thing, it's the only way I'll manage to feed me and my crew. I could run a pubsub queuing system and a database myself, but if I don't have to do that, it frees me up to focus on the tasks I'm really trying to accomplish instead.

That's fine, but I think it is a bit reckless to tell everybody that grocery shopping is useless and we might as well order from Uber Eats three times a day.

Which is basically what happens with everyone terribly afraid of managing VPS that these days it's only AWS, Azure and GCP. First the excuse is "it's too small to bother doing it myself", then it becomes "it's too large to run it ourselves".

Some can afford 3x daily Uber Eats, but we should stop discouraging home cooking. It is not that scary.

It seems a bit like car leasing, providers gets a bit of discounts for their volume purchases and customers pays a premium (that is still affordable) compared to just taking the entire deprecation cost themselves. The provider then still has something worth a fair bit of cash that can still generate income (through resale).

So if you look at a 2 year horizon and want to keep cutting edge (or prefer operating expenditures to capital expenditures) then it's cheaper whereas looking at a 5 year horizon the capital expenditure will pay off.

Hetzner is very bare-bones compared to regular clouds (a friend who was after performance but wasn't prepared for this ran into some issues when a disk died), so you need some procedures for backup/replication in place but if your procedures are in place then you can save big compared to the regular clouds.

There's definitely a tradeoff when it comes to maintenance effort. You need to make sure your RAID and monitoring works and if a disk fails you need to message support to replace it. In general I've had a great experience with them.

Last time I checked the GCP prices vs. hetzner servers there was a factor of 10 in monthly cost. So I cant say this is a general rule of thumb. It is true though that hetzner can provide superb performance per $. Potentially below what you can do (unless you buy racks of hardware).

Also keep in mind that hetzner is mostly a beowulf cluster. See e.g. this press picture from their newsroom page: https://cdn.hetzner.com/assets/Uploads/IMG-0546-91.jpg

They do have dell servers, too. But don't expect to run on one of these nodes unless the machine type has Dell / PowerEdge in their name. This means the management capabilities are minimal (but usually enough). A fully licensed management card can do way more.

Their renting pricing is ridiculously cheap though. Hardware failure is more common than on cloud providers (see cluster hardware), support has been helpful in the past. Overall IMHO a good choice if you are price sensitive. But even then consider development vs. deployment vs. running (server) costs. Servers might make up less of the total cost of a service than you expect.

To be fair, companies that merely launch VMs on GCP constitute a small fraction. GCP truly excels when you leverage its object storage, BigQuery, managed postgres database (which starts at 7$/month), and serverless solutions for cloud-native applications. Our company operates around 500 services, with billing per second, and a significant number of them are scaled down to zero when not in use. If you need a GPU for a batch processing task involving 10,000 images, you can simply activate a VM equipped with a high-performance GPU for an hour, pay for that duration, and then shut it down. At Hetzner, you're required to pay for a whole month upfront, regardless of whether you need the GPU for just an hour each day.

Therefore, I'd argue that if you require continuous, raw computing power, Hetzner is indeed cost-effective. However, the cost-effectiveness elsewhere really hinges on your usage patterns.

> To be fair, companies that merely launch VMs on GCP constitute a small fraction.

I think you would be surprised.

On purchase price alone you are usually better off not renting. Given a server generally comes with 5 years warranty, you are about 50% better off on those figures.

However, the killer isn't the purchase price. You have to put it somewhere with a lot of bandwidth, UPS, generators and cooling. To use your Hetzner example, you can't get co-lo for less than $119. That's 14U, which is more than you need for one server of course, but then there are usually charges on top of that.

So yes, for small installations it's hard to beat renting. You need to be using several racks for the equation to tilt towards purchase.

Not really related to their GPU offering but Hetzner is such a great company. I have a dedicated server I rented where I host all of my side projects. For the same price as one shitty VPS I get 8 cores, 64GB ram and a disk of 240GB or something like that.

Just incredible, response times are incredibly low and there is no cap on data. The few times I have contacted support they've been great. The only times I have experienced downtime is when I have stopped or rebooted the servers.

Can't recommend enough, especially if you are inside the EU and care about data privacy. The fact that they're a european company is of great value to me.

If you are new on server provisioning and want to learn how to setup a new server, I wrote a small guide for how to do it for node-projects (works for pretty much any web app if you replace the node.js stuff): https://deployjs.com/ (hosted on that server)

Had a server with them for more than 10 years. Had they eliminated the setup fee, they would get more of my money for sure for small projects.

The setup fee is a one-time fee though, so if you have a server for many years that won't hurt so much. I pay around €30/month for the server I am renting but I know it will last a long time even if I get spikes on any on my many projects. For a 8 core, 16GB droplet/vps at DigitalOcean it will cost you ~$96/month. The performance is better at Hetzner for about 1/3 the cost.

Yes. I'm talking about occasional needs for a small server for a month or two.

They have VPS cloud offerings similar to DigitalOcean as well, with no setup fees.

Why not go with hertzner cloud? No setup fees and you'll be billed by the usage. Still cheaper than other cloud providers.

The VM performance tax is huge. Also, no GPU and limited/paid traffic (large limit though).

Use the server auction? No setup fee then.

They often have a couple of servers with no setup.

Ah yes, they are great if you messing about. Not so great if you have crucial business server with them.

why not?

We've been running stuff on their platform for years with very little problems. Why not?

I haven't run into a single issue for years, what do you mean by that?

Care to elaborate on that?

I've been using their services for some 20 years at various scales, and the only complaints I've had are that their shared mailservers occasionally get blacklisted.

That, and the UI of their KonsoleH could do with a bit of a refresh.

Signed up from an Asian country and during signup they asked me to upload a passport copy page.

I thought that was weird (I mean no other cloud provider asked for it but okay). I uploaded anyway because I thought it’s in EU so they at least have better privacy laws so they may end up deleting it later from their systems but I immediately got rejected. Not sure why no reason specified. So now they have my passport copy page without me having an account.

Oh well, seems like either they have an overly aggressive spam filter that gets false positives or they don’t really like accepting signups from Asia

Hetzner is pretty hostile towards foreigners.

And don't you dare suggest in their forum that foreign tech is better than German, instaban.

Fuck Hetzner

What German tech - AMD, Intel or Nvidia?

Please don’t feed the trolls.

I can see a possible scenario, in where rather your general attitude might have been the problem.

I'm from South America, got a server with them, no issues so far.

I had a similar experience very recently (albeit within Europe). Did you try their manual verification process? It was available to me after two failed automatic attempts, and it basically had me send an email to Hetzner support asking if they needed any more information. I got approved manually the very next morning.

Maybe not the most customer-friendly experience, but I figure half an hour of signup weirdness is okay for a solid after-signup service in the long term, especially as that half an hour is likely to deter a decent chunk of fraud.

Had the same experience. Contacted support for manual verification, was online the next day with Hetzner. Machine has been running without interruption for a year now.

Same story from the US. I think they mostly worry that you are trying to avoid paying VAT.

Which server do you have on Hetzner that I missed? Dedicated servers start at 37.30 Euro per month now there, the AX-41 couch is pretty good and has no setup fee. On the other side I'm paying OVH about 12 Euro per month for two small VPSes and they are even too much for me, performance wise.

Have a look at the server auction? Tbh, I would limit my dedicated server rentals to server/ddr ram setups (which is a little more expensive).

But if you just need some cpu (to transcode some video, crack some passwords, what-have-you, run a game/videochat server) - consumer cpu models give you unmetered gigabit or 30tb/month 10gbps. So can definitely be worth it vs a VPS.

I have a dedicated server that I got on their auction page for 30 euro per month. That gives me 64GB of ram, 8 cores and some disk space.

Sure, you can get a cheaper VPS, but given a VPS with the approx same perf it would cost much more.

Ditto.

Been using them since 2012 or something, and have never had a problem (that wasn't my own fault).

I've just ordered one to ditz around with. We're early in the process of deploying our own internal server for Mixtral work, but it'll be interesting to see how this performs (in raw terms, almost certainly worse, since we can roll out a 4090 without getting into trouble – but Hetzner has a better connection and handles the maintenance, so..) Order has been accepted but not deployed yet. I'll update when I have some initial inference numbers.

I got it after a few hours. You need to install drivers/CUDA yourself, but all very straightforward. Unfortunately due to having 20GB of VRAM, I'm limited to mixtral:8x7b-instruct-v0.1-q2_K but it runs fine, generating at about 40 tokens/s (65 tok/s for eval). As per official specs, it's running maxed out at 70W (being an SFF card).

(I've now tried running the Q4 mixtral which is 26GB. 18GB is on GPU, 8GB through CPU. Gets about 11 tok/s.)

To compare, running Ollama with Q4 Mixtral on my 24GB 3090 Ti locally (with 22.5GB of the 26GB model on the GPU) I get 14 tok/s generation, so the Hetzner server and the 4000 really aren't bad at all (notably my 3090 draws 70W during inference also).

I never deployed it professionally but wouldn’t getting a Mac Studio be a better run for your money if you’re only inferencing?

Possibly. It's not a bad idea! The performance is pretty good in our initial tests, though cloud platforms do scale further in terms of concurrency.

Given how quickly things are changing, using more generic hardware (i.e. PCs and Nvidia cards) or leasing cloud based services might make more business sense in the long term, so investigating fallbacks with more control, like this server, are worthwhile experiments anyway.

Further, while I've been an Apple superfan for a couple of decades now, their recent attitude towards developers stinks, and I'm loathe to give them any more money than I need to right now, so I might be migrating away from the platform in any case.

FWIW, on the server auction, you could get GPU servers since a while:

https://www.hetzner.com/sb/#additional=GPU

Looks like currently there are only GTX 1080s on the server auction, I don't know if they normally have other GPUs available as well.

Nvidia RTX 4000 SFF Ada

From the OP link, the GPU-Server GEX44 server line includes RTX 4000.

And the server only cost €184/month.

I meant if they have other GPUs normally available in the server auction. As far as I understand the servers in the server auction are mostly custom built ones for customers and once they stop renting them they land on the server auction for others to rent.

The one with the RTX 4000 seems to be a standardized line of products.

They briefly had GPU servers before. I believe they stopped when crypto messed up the gpu prices and supply. That's why you're seeing them in the auction now. But they usually go quick. This might change now since they have a brand new offering.

The old GPUs would have very litle memory by the way. Not ideal for AI.

They stopped because they had a high percentage of abuse with Morons mining crypto

Hi there, Thanks for helping to spread the news about our new GPU server, the GEX44. Please keep in mind that it’s not like we don’t want you to open an account – it’s more about safety & security. In general, we take extra precautions when it comes to verifying accounts and accepting new customers, as in these cases the potential of abuse of our ToC are higher. As a reliable hosting partner, we always take the time to listen to our customers and will do our best to find a solution for each case. So feel free to reach out to us if you need further assistance. --Katie

A reliable partner does not close accounts without any kind of recourse. Denying that under the guise of fraud prevention is just security through obscurity. I hope these practices become illegal sooner rather than later. Never before have I been treated as a criminal as much for trying to do legitimate business, and it’s quite the experience.

It looks like a good value for money, but immediately after registering, I get this reply

"After reviewing your updated customer information, we have decided to deactivate your account because of some concerns we have regarding this information. Therefore, we have cancelled all your existing products and orders with us."

Someone has a similar experience? I've tried it twice, filled everything truthfuly, valid credit cards etc. and no still the same.

I haven’t faced this issue myself, however, lots of people from India and some other countries have this experience.

I’d assume it’s something to do with collecting recurring CC payments from Indian cards. That’s inherently difficult.

That would also be my guess, but I live in western Europe.

There's a manual verification process if you contact support. Obnoxious that it's needed, but works for many people.

I got this when I first registered, contacted support and it was lifted in a day or two.

Hetzner has one of the worst customer support I've ever seen. Get really, really, faraway from this company.

The GPU product page: https://www.nvidia.com/en-us/design-visualization/rtx-4000-s...

20GB RAM, can fit 13B param models, and maybe some quantized larger models.

Is Nvidia RTX 4000 solid for training? Alternatively this card seems listed for around $1800 so at $180/month on Hetzner, it may be even cheaper to buy it, depending on the expected usage.

It's with ample RAM and the latest Intel CPU. Assuming the whole thing draws 100W, that's also including 72kWh of electricity.

I'm not sure what the exact 'rule' is here, but you wouldn't use a contraction in this sentence.

It is, with ample RAM and the latest CPU.

From https://www.yourdictionary.com/articles/contractions-correct... -

Contractions can be used in any position in a sentence; however, homophone contractions such as "it's" and "they're" sound better when followed by another word or phrase. The reason is that the sounds of "its" and "it's" and "they're" and "their" are so similar that they can be confusing unless they are used with the context of an additional word.

To be fair, the contracted version doesn't read nearly as nicely.

    Q. Is it good for training?
    A. It's with ample RAM and the latest Intel CPU.

    Q. Is it good for training?
    A. It is with ample RAM and the latest Intel CPU.

I think they are agreeing with it not reading as nicely. But you still need the comma. Otherwise it sounds like:

"It is with great pleasure..."

I don't think you necessarily need the comma but it would depend on the reader, I think. I didn't need the comma to read it correctly but I can see how you might mentally insert a pause between "it is" and "with" that makes it read weirdly.

Probably better to reorganise it as "With ample ram [...], it is."

I think it's less to do with homophone confusion and more to do with usage patterns relating the contracted part to a phrase rather than a single word [1] [2]. By the logic of the homophone explanation, the following exchange should be much more acceptable than a sentence ending with "it's" or "they're", but to me it seems equally strange:

"Are you waiting for someone?"

"Yes, I'm."

[1] https://www.youtube.com/watch?v=CkZyZFa5qO0

[2] https://en.wikipedia.org/wiki/Clitic

Why does this GPU cost so much? It seems to be like an RTX 3060 Ti/3070 with more VRAM and less power draw. Does that really justify the price tag?

Because it's "enterprise".

Like Apple, they use RAM for market segmentation.

NVIDIA's licensing. Note that the 3060/3070 is a "GeForce" whereas a 4000 is a "Quadro". Then look at the GeForce driver EULA: https://www.nvidia.com/content/DriverDownloads/licence.php?l...

> No Datacenter Deployment. The SOFTWARE is not licensed for datacenter deployment, except that blockchain processing in a datacenter is permitted.

I could never figure out why they allowed blockchain processing, given how wasteful much of it was and how it impacted card supply for years, but there you go.

Because they made a lot of money selling to miners at prices that maybe they couldn't ask to gamers?

Wouldn't the GeForce license then say "no block chain" and a different NVIDIA brand (BlockForce?) would allow it, letting NVDA charge more for large scale mining? Otherwise it looks like gamers and miners are paying the same prices.

They do. But since most miners would just ignore the license, they implemented LHR (low hashrate) beginning on GeForce 3000-series that detects the types of ops used for mining and artificially slows the GPU to about 25% of its actual speed.

They did that but with mixed success. One random link from a Google search: https://old.reddit.com/r/EtherMining/comments/t4mwxj/workaro...

Their source code got leaked and it was reverse engineered to defeat it. That said, it was all kind of late in the game, GPU mining died not too long afterwards.

I bet that now the AI demand spike will be the new crypto demand spike that NVDA latches onto.

This seems insane to me. It makes me even more confused that AMD/Intel aren't trying their hardest to go after AI compute. Nvidia is basically leaving the door open.

But I suppose that the idea could be to protect their gaming market in a way. If datacenters were buying up gaming GPUs then that could cause shortages for regular customers.

AMD is trying their hardest.

Datacenter customers don't have options, gamers do.

RTX 4000 isn't even data-center grade - it's workstation/pro segment. This is an important difference, as some server vendors (such as Dell) will not validate the workstation segment to run in their servers.

The segmentation is a huge mess from NVIDIA. Consumer grade GPUs aren't supposed to be run in data-center / compute (forbidden by EULA) and the workstation GPUs are in a weird middleground limbo, being many times more expensive, but not exactly attractive for server use for the above reasons.

You have to discount the price of a similar server without the GPU, tough.

But even so, it's still quite expensive. I have the impression Nvidia charges more from data center customers. So, for cards available to retail consumers, it's not a fair cost comparison.

With only 20GB, RTX 4000 has limited use... I was expecting they'd be offering something with at least 60GB at this point.

No, not a great GPU for training. Consider something like A5000, A6000 or their Ada variants - if you want to go full "data-center", A40, L40, L40S are solid picks for small-scale training.

Are you going to be training constantly?

TBH renting a big GPU (A100 or better) and getting the run over with quick is usually best. VRAM is everything.

It's running an RTX 4000 SFF Ada, 20GB GDDR6. Interesting that they've gone with a single-slot, low power card. Doesn't pull enough power to mess with their cooling/power design, I guess. Not a lot of graphics RAM to do much with LLMs.

That card is about $1500: https://www.amazon.com/dp/B0C2KMXQYG

It's a good price at ~0.30$/hour comparing to Paperspace's RTX 4000 with 30GB RAM at ~0.56$/hour.

Are there many other providers for such environments?

vast.ai, runpod.io, tensordock.com, ... I have just ordered a GEX44 from Hetzner. :-)

If you want to check them all out in one place, we built a platform for that at https://shadeform.ai

After the exchange rate, that's $270 CAD a month... after a few months, you might as well just buy a raptorlake and run it locally.

Welcome to Cloud Computing.

It's a GPU system, not just raptor lake.

I need some help:

I have no idea where the NVIDIA RTX 4000 SFF belongs in the hierarchy of speed and suitability for "AI" computing.

How does this compare to a NVIDIA A800 40G? (if that is still considered good, I can't keep up)

To me the RTX part of the name sounds more on the consumer side but I am probably wrong about that as well.

How is the gpu for inference or tuning ?

They had a strange setup process. I had to scan my passport and stuff. However, I did have a VPS with them that had uptime greater than a decade and that I killed only because the underlying hardware was going away. That was some 7 years ago or so.

Note: For now for existing Hetzner customers only.

A few months ago I tried to register an account, but I was denied.

I don't know why, never got a response...

I went with ovh.

It's a pity you can't commit to 3 year rental for a discount

Do we get a discount for your MITM?

Marketing still doing damage control, I see.

Does every European tech company have some kind of committe that ensures their customer facing website looks like a decrepit ASP page from 2004?

I mean what explains this? It's a billion dollar corporation and I am having serious reservations about whether I would even enter my credit card info on this site.

It's true. And it breaks on mobile when you use the menu.

Hetzner's site looks fine to me. Highly readable and to the point unlike modern web shit where you have to scroll through 5 miles of marketing animatinos.

I mean, see also AWS; for a very long time their console looked like something from the dark ages. I think they have largely spruced it up now, though I'm not sure it helped _usability_ too much, and may have actually made it worse.

I see the same thing on HN posts. 1999 vibes everywhere.

I was going to seriously disagree with you, and then I looked at their source code.

It's awful.

I would still buy from them because their reputation trumps any concerns.

Just make sure any workloads you place on Hetzner are treated as spot instances and outputs are immediately sent off network. The reliability of their systems and customer support is very low. Hetzner is a 5-star or 1-star company. Are you willing to risk your business on the flip of a coin? My own experience [2] was definitely a 1-star.

[1] https://www.trustpilot.com/review/hetzner.com

[2] https://news.ycombinator.com/item?id=39075608

I've been with Hetzner for around 10 years and didn't had a single larger incident so far. Support also answered promptly - didn't had the need to request a lot support so far. Running 2-3 servers and a few cloud instances.

Reliability and customer support are unbelievably good. 99.995% uptime for the past 10 years. Hetzner will happily customize your servers however you like, and at their low prices you can have hot standbys for everything and still save money.

My friend went onto Hetzner for the performance/price, unfortunately the machines drive died within 2 months iirc. Seems like if you're buying on auction you'll get a wiped drive and if you're unlucky it's been written to a lot already, don't remember the specifics but he really wasn't a happy customer afterwards.

I think the last sentence you have there is pertient though, maybe don't go for Hetzner unless you can also afford to have a hot-spare (but for the performance/price you probably can compared to any public cloud). In my friends case he was boot-strapping but needed the CPU so he was really penny pinching.

Trustpilot is a scam which encourages bad reviews to force companies to buy their paid plan to “manage” those bad reviews and to feed it good reviews.

Hetzner is also a fraudulent operation with zero support in my personal experience. But as you can see in this thread alone, any negative feedback is magically silenced. It sucks. Maybe we go back to webhostingtalk?

Negative feedback is silenced?

Dude the characteristic feature of Hetzner threads on HN is a bunch of people in the comments moaning about having to scan their passport or repeating the 'they just turn your servers off with no warning' line.

In fact it's so prevalent and consistent - at least here - that to my eye it looks to me like black PR. Or that they're really good at rooting out internet scumbags and that the internet scumbags are unhappy about it.

I run a legitimate non-sleazy business in the US doing mid 7 figures a year. It’s not a “line”, it was the reality of an attempt to do business with Hetzner.

I’ve never had any serious issues with other vps, colo, dedicated, or cloud providers in 25 years as an employee of many companies or as a business owner: only Hetzner.

> It’s not a “line”, it was the reality

I'm not saying it isn't true, I'm just saying that there's no way you can describe criticism of Hetzner being 'silenced' on HN; criticism is literally a defining facet of Hetzner threads.

Downvotes til grey/flagged = silenced. Which is accurate of nearly every negative comment about Hetzner on HN.

Reply on news.ycombinator.com