Hetzner Cloud Goes US (Ashburn/VA) | Hacker News Reader

Hetzner Cloud Goes US (Ashburn/VA)

twitter.com

163 pointsevilc00kie112 comments

I've had a the smallest cloud instance for a few years now, it is in fact really good and fast. 1vcpu with 2GB ram is plenty for a full stack web application + a database. IO is also very good with stable ~8000 iops write performance for 4k sync.

Compared to Azure VM instances, Hetzner offers like 100x more performance for each $ spent.

Interestingly, I have used racknerd.com from lowendbox.com deals and it works quite nicely for me. No hiccups.

They undercut linode/vultr guys like many other leb providers so you should try them as well. Leb has good deals for ultra low cost vps

Same, but there is some shady history with LEB to be aware of. Undeniable value though.

Can you share more?

It appears ColoCrossing created dozens of VPS "companies" that they pumped through LEB, sold plans, and then shut down without refunds. LEB may be/have been owned by or share common ownership with ColoCrossing.

https://www.hostscore.net/news/20-vps-hosts-shuts-down-on-de...

https://hostedtalk.net/t/colocrossing-chicagovps-hudsonvalle...

> Compared to Azure VM instances, Hetzner offers like 100x more performance for each $ spent.

So much this. I can't stress enough how awesome of a deal are Hetzner Cloud's offerings. I've also been running a personal Docker Swarm cluster on Hetzner for the past two years and I couldn't be happier with the way it performs. And how refreshing it is to actually be able to predict and control costs, and have absolutely no worry in the world regarding what might pop up in the invoice. If only this was true for any other cloud provider.

How's the development velocity compared to Azure?

Can your App scale across regions?

I don't know how to feel about this. One the one hand this is good for sites that needs to be more global. But I mainly build stuff for Europe and I don't want anything to do with USA when storing personal data. I hope they keep the seperation between Europe and US clear.

One can hope, otherwise I can see European customers move off Hetzner. We have a number of customers who do not want US staff to be able to access their servers and VMs.

I think the main thing for GDPR is that US based employees should not have access to manage EU datacenters remotely.

That surprises me (not that I'm particularly educated on this), any links to more info on why GDPR requires that?

The Schrems II judgement might be applicable. I know that in the EU-based company I work for we have strict requirement for all cloud providers to comply with Schrems II, and not send/store any personal data to outside EU.

https://www.gdprsummary.com/schrems-ii/

i was asking specifically about "US based employees should not have access to manage", which isn't necessarily the same thing as "not send/store any personal data outside the EU". You could have data stored inside the EU, but then saying no US-based employees can have access to it seems like another further requirement? Although it may be one under GDPR? But that's what I meant asking for more info about, sorry!

Why would an US employee leak personal data just because they’re in the US? It would be a clear violation of employment and probably criminal.

US court says hand over the data. German court says that's illegal to do. Complying is illegal and so is not complying.

Not Complying with US law is also illegal, now imagine who has more to say in that regard.

Hetzner should split up in two Company's.

BTW:

The CLOUD Act applies to all electronic communication service or remote computing service providers that operate in the U.S, whether those providers are established in the United States or another country.

So one concern (and it's not just a theoretical concern; it has happened) is that a US court forces the US branch to use access it has to data held by the EU branch to exfiltrate data. Companies with the highest standards on this stuff will want either a pure-EU host or a host structured such that this can't happen.

Hmm, this is definitely worrying.

Fully legal thanks to "cloud act":

https://en.m.wikipedia.org/wiki/CLOUD_Act

If you setup the server in the EU why would anyone in US have access to it?

This is such a far departure from where I was hoping the Internet would look like growing up. Makes me sad political borders are now viewed as a feature.

The internet can't just paper over the fact that there are fundamental differences on how people do things across the world.

The internet was supposed to be the last free land where nerds could be able to create without worrying which part of the floor is lava.

And it would have stayed that way if only nerds would have used it.

Now it's used by regular people, too. And our lives are much richer because regular people use it, too.

But that means that real life rules now apply.

People ruin everything. Eternal whenever is upon us.

It's a big feature for users who value freedom of speech.

It's laughable that a guy from the EU doesn't get it - their freedom of speech is shit.

This is nothing to do with freedom of speech; the issue is differing laws on data privacy between the EU and US (and repeated failure of Safe Harbour and other regulatory attempts to paper over the cracks).

> It's a big feature for users who value freedom of speech.

That would be Iceland then, NOT the US.

That has nothing todo with the internet but with US law.

BTW it was always like that, want to study the human-body in the middle-ages? Go to arabia, in Europe you get burned alive.

Because some American courts like to think they have jurisdiction everywhere as long as there's a US entity involved

The Kim Dotcom case proves that some courts will ignore nation borders altogether.

Rackspace, Indymedia, 2004. Ancient precedent by now.

They're saying:

> Hetzner's own team of technicians in Germany provides customer support for the Hetzner Cloud servers in both Europe and Ashburn, VA. See below "How do I contact support?" We hope to soon expand the hours for the support team, and we will make an announcement when this happens.

https://docs.hetzner.com/cloud/general/locations/#who-provid...

Any info on what DC they use? So far they've run everything themselves which made them slower to expand but was much cheaper to run. If they now use other DC providers it would explain why they only offer the higher margin cloud servers there. But would be curious if anyone knows where the data center is physically located. With their other ones they're also quite open about it so I was surprised they didn't mention it here.

EDIT: They do mention somewhere on their side that it's a third party DC. So probably no option to offer dedicated servers there (at least not at the same price as in their other locations). Would still be curious which one it is.

NTT/RagingWire VA1.

Here's the details about the sea shipment that was sent from Hetzner Online GmbH in Germany to the US subsidiary: https://www.importgenius.com/importers/hetzner-us

Now that is a cool database. I'm surprised it's public.

so if i didn't misunderstood that page they've had one 40 foot container delivered with nineteen "server hardware" pieces, likely racks? thats less than i would've expected, but it makes sense considering that they probably don't have any big customers lined up.

it will be interesting to see how fast they'll grow.

Maybe they only shipped some components they don't want to have assembled there or that are more expensive in the US. So theoretically they could've just shipped network equipment or storage servers (for block volumes) while other servers were assembled in the US. I can't really imagine that it's cheaper for them to assemble complete racks in Germany and ship them over, assembly cost by a third party in the US is likely cheaper and definitely much faster.

Oh wow, that's really amazing. Thanks for the info!

So they shipped a 40ft container full of servers via ship to the US? That really surprises me, I always thought this kind of equipment was done via air cargo only.

Air Freight is extremely expensive for this purpose.

It is, but these components also probably cost upwards of a million dollars and are very fragile. Esp with current shipping delays (not sure how the situation is on the route EU-US), I'm surprised they waited the extra few weeks to save a few thousand $.

Right now we're not offering any dedicated servers (as I wrote on another comment). That might change some time in the future depending on how well things go here. But I can't give a clear answer about whether or not we will introduce dedicated root servers, or an ETA. --Katie, Marketing, Hetzner Online

I've been working on bringing the managed services (which is still somewhat lacking though Hetzner has LoadBalancers) now to places like Hetzner and OVH. Right now I'm planning on starting with Redis/Memcached and S3.

If this seems like something you'd be interested in I'd love to hear from you (email in bio). If you don't have the time to write an email but maybe take a most-questions optional survey:

https://docs.google.com/forms/d/e/1FAIpQLSfyhBThpC2mYubNPQaK...

I plan on getting a small landing page up to collect people who want to be first to try it out tonight/tomorrow at the latest.

I'm most excited about Hetzner going US because I think they're going to really shake up the stagnant (high) prices for dedicated compute in the US.

Not sure about their cloud services, but their dedicated servers offering is unmatched anywhere. Unfortunately they only have DCs in Finland and Germany, would love to see it expanded. Anyone here sitting on any information if they are planning to extend their locations?

This. Cloud is everywhere nowadays, but most dedicated servers are outrageously expensive or have bandwidth limitations (I'm looking at you OVH). Servers on Hetzner's auction might be pushing 10 years, but who cares as long as they work just fine.

knownhost has a pretty good dedicated server offering in the US

What kind of bandwidth limitations have you seen with OVH? Their burstable bandwidth works great for us during peak times, I've never noticed any throttling. Though if you're running a seedbox and maxing the interface 24/7 things might be different.

Last time I checked, their midrange brand Soyoustart had 250 mbps and OVH 500 mbps. Hetzner is 1 gbps. Other than that OVH prices are much higher for more or less the same specs.

Hi there - We are currently just focusing on the new location for cloud servers. Depending on how successfully things go there, we may consider adding other products in the future. But we will discuss that internally first before making any announcements. Sorry that I can't give you a firm yes or no answer. --Katie, Marketing, Hetzner Online

No worries, the process as you described makes a lot of sense, dedicated server hosting is probably a big investment. Thanks a lot for the reply!

Same, have been a happy Hetzner customer for 8+ years because they've consistently offered the best dedicated server pricing with essentially free (20TB) bandwidth.

Unfortunately they're only offering their "Cloud" hosting packages for the time being, so it'll be a while before they start offering dedicated servers.

Once they offer dedicated servers in US and a managed RDS service I can finally look at moving off AWS for most services. But it looks like Cloudflare will beat them to it as their CEO has mentioned they're already working on a managed DB service, hopefully it will be as cost-effective as their R2 pricing.

Yes! And their support is amazing too. Doing business with Hetzner somehow always feels very different to e.g. Amazon, or the other tech giants.

I wonder if it may be related to the general business mentality (Germany vs USA), and not only the company size.

It does. Because "customer is king" mentality is very strong in the US and not so much in Europe.

Unless I'm reading you wrong, this would speak against Hetzner providing good support.

I had a terrible experience with DHL, we used them for US domestic small parcel shipments before the business unit was shut down for losing money, around 2008 I think. Towards the end they started systematic over charging. They would rate an 8oz package as 20lbs. The bills were 5-10x what they should have been. The service was never very good, they just had a good sales team. I hesitate to generalize from specifics though, it doesn't represent how all German companies do business. Maybe we should care less about national identity and more about the company's reputation.

DHL was a US company until 2002, maybe that makes a difference

Their offerings are great. The best server for the money that I’ve found, though, is Netcup.

I’m glad to see a Hetzner DC in the US, though. I hope they shake up our ridiculously overpriced cloud offerings this side of the Atlantic.

There has been a nice YT video recently showing how Hetzner operates their DC and their servers: https://www.youtube.com/watch?v=5eo8nz_niiM

Was quite surprised that they use custom built server racks with their own custom-made mainboards delivered by OEMs. They even have their own long-term Fan testing rigs and custom HDD failure check setups.

Hetzner has a seriously flawed fraud detection system. There are multiple documented review complaints. Seriously, try opening an account from a third world / low income country (non EU/US/UK/CA/AU/NZ), your order will be flagged immediately. If they don't want orders from those countries, they should explicitly say so.

Had the same issue trying to open an IBM cloud account recently. After submitting my CC info the system told me to get in touch via email, and after contacting them they wanted two official documents (Driver’s License and Passport) to be submitted via email. Big nope from me.

Hetzner will actually refuse your order after you submit your documents with no valid reasons whatsoever. At a point, you just give up and ask your EU co-workers to spin up dedicated servers for you.

We do not publish the specific reasons why we reject orders. If we did this for every customer, it would be really easy to create a fake account. Potential abusers would quickly learn what to avoid. So we don't publish this information to anyone. We understand that that seems harsh. That's not what we intend. We just want to prevent abuse. --Katie

It's not ethical to withhold this information. I'm sorry it makes it harder for your fraud departments but it's not ok to reject someone and not tell them why.

That's all there is to it. I've had this happen to me by banks before and it's shit because the customer has no recourse. They can't advocate for themselves.

This practice needs to change, industry-wide.

We know it feels terrible to be rejected. We do. But every online store and service has customers who they reject. The internet can really test your trust in people, and for good reason. Our products are not flowers, or delivered food, or books or music, which are all pretty non-controversial. Even companies that sell these things online reject new customers. Our products, like a car and many other things, are mostly for good, but in the wrong hands, they can be destructive. There unfortunately aren't international licenses for trustworthy sysadmins or an international police agency that can enforce that. So we have to try to do our best using our own systems. People who abuse servers are not always easy to spot. There are, of course, customers who accidentally resemble a fake customer. In the past, we used to tell customers why we rejected their orders because of this. And we had much higher rates of abuse. So we changed our process, and we no longer publish this information. And this has been working better. It's harder for people to create fake accounts with us now. We don't mean to hurt anyone's feelings or cause frustration. --Katie

Country of origin is not the sole reason we reject some orders. Most orders are rejected because of more than one red flag. We do have customers from "high-risk" countries. We also reject many European orders that appear to be fake, or that are far enough along the "possibly risky" scale. It is not a perfect art, but our team is constantly tinkering to improve our methods. We find that being overly cautious helps us to prevent abuse down the road. --Katie, Marketing Hetzner Online

Hetzner's definition of "high risk" orders is certainly geographically biased. If you're worried about invoice payments, consider a prepaid credits model.

On the other hand it probably allows them to offer these prices. For AWS, unpaid bills from fraudulent accounts will make up a significant line item, Hetzner can't afford that at their margins.

When we thought to rent a dedicated server for our hackerspace we initially put "Bernd Liefert" (a german in-joke) as the responsible admin down and they basically told us "lol no we know our jokes, too". We had to tell them a real, existing person as contact or they wouldn't have rented us the box.

So can confirm, they also reject European orders.

For me, Hetzner, Starlink, and Tailscale are 2014-era startups: stellar products that turn you into a zealous militant for the companies. This is a very exciting development. I am very much looking forward to their dedicated offering extending to the US.

> For me, Hetzner .... are 2014-era startups

Huh? I'm pretty sure the first time I heard of Hetzner was almost a decade before that. Wikipedia says they were founded in 1997.

EDIT: Ok, I take it you mean they are like 2014-era startups? A quick Google search tells me, Starlink is the only one of them that could be dated to 2014. But IMO still a strange comparison. With their business model and history I wouldn't exactly call Hetzner a startup.

How is Tailscale, just another VPN provider, comparable to Hetzner and Starlink?

I smell covert advertisement…

I’d put myself in the camp of zealous Tailscale customers.

I’ve used wireguard for years and Tailscale solved every single complaint I had about Wireguard.

Their ACL management could use a lot of polish but moving to them and paying for the service was the easiest decision to make after how well it worked. And the product has worked flawlessly for us.

Not a paid advertisement. Just someone who never ever thinks about wireguard provisioning anymore.

>I’ve used wireguard for years

Really for years?

Yes. For years. 2018 I think?

Same, I would invest in Tailscale if I could.

With tailscale you press one button and it magically works, no endless fucking around with OpenVPN config files.

I have no business connection to them but am an extremely happy free tier user.

I don't call Tailscale "VPN Provider". Technically it's VPN but now "VPN Provider" means it provides "anonymous" "safe" internet connectivity.

"The host systems for our Cloud instances all have a redundant 10 Gbits connection. This connection is shared by all instances on the host. We do not offer bandwidth guarantees for our Cloud servers, but you can expect about 300-500 Mbits."

This sounds pretty underwhelming and probably what's preventing me from migrating infra to Ashburn. Other than that, looks pretty great bang for buck.

I like that tweet:

>The CLOUD Act applies to all electronic communication service or remote computing service providers that operate in the U.S, whether those providers are established in the United States or another country.

Bravo Hetzner, no chance for servers with private data (for example European customers) on it anymore, Exoscale it will be then in the future.

Dang, please can we merge this and https://news.ycombinator.com/item?id=29093082 ?

As an American who'd only be using VPSes like this for personal projects, the only thing holding me back from trying Hetzner before this was that they had no US servers.

Cloud product site has been updated: https://www.hetzner.com/cloud

I changed my country on that page to US but all the prices still seem to be in euros?

It would be great if they let me see more detail about the instance types without having to sign up for an account. I'm not sure if I want to try it out, and I'm lazy, so I probably just won't now.

Their FAQ mention that prices will be in Euros.

Scroll down on that page. Specs should be there. Or do you mean something different?

I just mean prices. It's difficult but not impossible for them to do a currency conversion in-browser... just one of those things that helps convert customers, funny that they don't do it.

Here is why me having to do the conversion is annoying.

Pricing at 1 euro = 1.1579126540169 usd:

- Traffic: 20TB included, over that is $1.1579126540169 TB/month

- Instances: $0.0081053885781183 /hr, $0.0127370391941859 hr, $0.023158253080338 hr, $0.0440006808526422 /hr, $0.092633012321352 /hr

- Backups: 20% of instance price (so for first instance, $0.00162107771 ? )

- Volumes: $0.046316506160676 GB/month

- Snapshots: $0.011579126540169 GB/month

I'm willing to bet my currency conversion rate and math precision are incorrect (we all know precision is hard) and customers might get pissed if they think they're getting overcharged when they do the math incorrect on their side. A pricing calculator would help.

Currency rates are constantly changing. If they list a converted USD price when you visit the site this may be different than the rate when you are billed. If they don't want to establish fixed USD rates than providing a point in time currency conversion is likely to cause more problems than additional customers this converts.

This appears to be a product aimed mostly at European customers that want a US presence and not to US customers.

Yes, prices are still in euros. Please see: https://docs.hetzner.com/cloud/general/locations/#what-about... I understand about being a bit lazy when it comes to trying a new company. However, if you change your mind any you become more interested, you can check around and see if you can find anyone with a referral code. (We have a referral program: https://docs.hetzner.com/cloud/general/faq/#is-there-a-refer... ) --Katie, Marketing, Hetzner Online

In case anyone is interested in trying it out: https://hetzner.cloud/?ref=GxNGm9jZSUZP

(or if this is spam and unrelated to the article, flag it)

It's great so see more investment by European infrastructure providers like Hetzner, Scaleway and more recently Infomaniak.

Even though they're years behind in terms of features, their base offerings have improved a lot over time and they can at least compete on cost.

This is huge, 1cpu, 2gb or ram, 20TB of bandwidth on each vm for $3.99.

Great deal, I use OVH Starter VPS that is a fraction cheaper at $3.50 (same config) For testing and personal use I found both offers to be top-notches and only DC location could be a decision factor here

Ovh comes with a 250mb connection, hetzner comes with 2gb/s

For my purposes 250mbs is more than enough as it's just a testing )no prod) VPS

That one is not available in the US though.

It's EUR not USD, looked like $5 showdown

Naive question: what’s the difference between this and ec2? (I am already running my personal mail and website on an ec2 instance; is there any good reason to switch?)

Hetzner is a fraction of the cost of AWS. A 8vcpu, 16GB RAM instance is 22€ on Hetzner cloud or $148 on AWS. If you have substantial egress traffic the difference becomes even more pronounced.

AWS has a bigger ecosystem of services, and due to AWS's traffic cost you can't cheaply mix-and-match things inside and outside their ecosystem. That's probably the biggest reason to go with EC2.

While not exactly apples-to-apples in all scenarios, I encourage people to use Spot Instances on AWS whenever possible. We run all our web traffic on spot instances, for example.

Spot pricing really closes that gap. A m6g.2xlarge is ~220/month on-demand, $140/month with a yearly RI, and $60/month as a spot instance.

Hetzner is still cheaper for sure if you have a stateful, 24/7 workload. But if I use auto scaling on spot instances (where a good chunk of compute is only running for a part of the day) then the math starts getting much much closer.

That 22€ instance is shared cpu, and they do get oversubscribed. The dedicated vcpu ones are more comparable to AWS and at 70€ they are still a lot less expensive than AWS. Then there is the free egress bandwidth (up to 20TB per instance) and unlimited free internal bandwidth.

The main shortcoming of the US deployment so far is no HDD storage available. I hope they can offer that soon, even if no dedicated servers. Their biggest StorageBox plan (basically scp/rsync storage though sshfs works) is 40€/month for 10TB or so. The cloud servers have SSD block storage available but it is .04€/m for 1GB i.e. 10x more expensive (though a lot more flexible) than StorageBox.

Mainly the price, Hetzner is much cheaper. But you don't get all the other features from AWS. So if you only need EC2 and EBS you should probably use Hetzner and will save a lot of money.

Aws bandwidth costs would be the primary reason to potentially switch.

If you're stuff is low bandwidth there is probably little reason to move. You might compare the cost tiers of the size of system you use. Might also be reasonable to run personal stuff on spot pricing on AWS for cost savings.

Anyone know how I can invest/get exposure to hetzner as a business? Doesn't seem to be a public company.

I can honestly see these guys becoming a HUGE cloud provider.

Hetzner is a privately owned company, so it's not possible to invest in it. The owner and founder Martin Hetzner is one of the CEOs and is heavily involved in the company. Like with anything, there's some pros and cons to being privately owned. It means our fans can't help us grow by investing in us. And it means we tend to open new locations very carefully, and perhaps more slowly than our competitors. But it also means that we are not beholden to any investors. So if the CEOs decide to re-invest lots of the profits back into the company, they don't have to worry about a board complaining about lower returns. --Katie

Is there like any other reputed Cloud provider that beat Hetzner's price at the entry level?

I like hetzner but they do not support dnssec yet.

https://docs.hetzner.com/dns-console/dns/general/dnssec/

That's a good thing.

Besides, what is holding you back from setting up DNSSEC? It's not as if Hetzner doesn't support manually managing your DNSSEC-related records. Having your registrar manage your DNSSEC deployment kind of defeats the point.

I have to push the hashes (DS records of the ksk) to the parent zone (tld). I think that only can do my domain registrar.

Is there a way to see equivalent prices in USD? Also would my credit card be charged in Euro and then the card company will apply some arbitrary conversion rate ?

More Cloud providers is a good thing. But I hope more providers start following the method of an implementation of OpenStack that a customer can control. OpenStack has an API and tools which makes it much easier for a customer to start using cloud services in a standard way. Might not be good for "vendor lock-in", but it does let me ramp up on technology quicker. I've used OpenStack in a couple cloud providers so far and I really enjoyed it. (I've also administrated an OpenStack bare metal cluster, so I don't envy whoever is managing it at the cloud provider....)

Openstack: the API every customer wants, but a stack no CTO wants to deploy?

Hetzner has an API that is a lot less complicated than Openstack. It's not that big a deal to use it.

Thanks for the feedback about our API. :) For anyone who's interested, here some more info on it: https://docs.hetzner.com/cloud/api/faq --Katie

Reply on news.ycombinator.com