Render nuked my entire account with no notice | Hacker News Reader

Render nuked my entire account with no notice

228 points0xy117 comments

Recently I decided to migrate one of my side projects from AWS to Render for convenience and to manage some of my high ALB costs on AWS.

This weekend I began the process to move some of my infra to Render, including a static website, Postgres database and Redis. I switched the static website over but had some issues restoring my database from SQL backups (Render's Postgres appears to timeout connections after a certain amount of time).

After spending a few hours attempting to import all my data, I was logged out from dashboard and all my services were abruptly taken offline with no notice.

When trying to login again, it just throws me back to the login screen and sleuthing the network requests I can see I am now getting "Unauthorized" responses.

My production website was abruptly taken offline with the message "This service has been suspended by its owner." (I didn't suspend it)

For clarity, I was on a paid plan, had a credit card added, and all of these services were paid. I have received no communication, now I have to migrate back to AWS.

I can only assume they saw I was using the database heavily (normal during a restore) and decided I was abusing something? I absolutely did not violate any of their terms nor was I doing anything shady. It's just a static website and a Node service. Nothing crazy.

I just wanted to provide a warning to others because this seems egregious.

View on HN

I had exactly the same experience with Render. I was researching where it would be cheaper and easier to setup my infrastructure. So, I deployed bunch of services to Render, and boom, "Unauthorized" and "This service has been suspended by its owner". Sent an email to support, got first reply after 4 days, and then silence for a month, until I angrily asked them what's up. No need to say that I will probably never go with Render.

What was their response to your angry question?

They replied the same day saying that they unblocked my account and I should be able to login. No explanation why it was blocked or anything.

This is unacceptable. You'd be well within your rights to ignore this request, but I'd appreciate an email with your details so we can fix the underlying problem.

We need a new sub category [Support HN] when we need high level intervention from the CEO or at least a product manager. YC startups get an immediate notification if their name shows up there.

Feels like making this an official thing would immediately lead to it being abused so much that people will stop paying attention, like they do on other social media.

I have long had an idea that the best way to handle this is a community supported prioritization service.

In other words, a community (hacker news, some subreddit, or perhaps an independent site) would invite people to post problems that ought to be easily solved by customer support except that the company is failing at customer support. Because problems tend to repeat themselves, the people who spend time on this forum would be able to identify (and sometimes provide workarounds for) the majority of cases -- all at no cost to the company that was failing at customer support.

The remaining cases could be ranked by the community according to how egregious they were. Then the company that was failing at customer support could assign a tiny amount of actual attention to look at just the top few issues sorted by the community.

> The remaining cases could be ranked by the community according to how egregious they were.

It’s bad enough companies started outsourcing deciding which features to implement. I don’t want them also outsourcing deciding which cases to support.

I think it would be better to outsource it than to simply NOT fix anything for any customers OR only fixing issues for customers who manage to get a lot of attention on Hacker News.

i hope someone makes a bot then it just tweets where thread is linked, company and ceo is tagged

I know this is most likely a joke but just in case: let’s not do this, it would basically be [Shame HN], because that’s what makes it work.

Unless we find a way to turn it into some [AITA HN], so that it’s not just drama but also a bit of introspection.

We should call it [HNSHITTIFY] to bring it full circle

I agree, if this turns into something formal and compartmentalized it will be as easy to ignore as the official support channels.

Yes, these serve as a good warning for others who may be interested in exploring this service.

HN threads also rank very highly on Google so even for future users researching the service may gain some value from it.

Like I was interested in trying out fly.io for a long time but after reading so many recent threads about their mistreatment of users I've decided to defer.

These threads are really very useful.

Just stop using these kinds of funky PaaS services. Use a boring host with a good track record who is not trying to make a VC rich. For other stuff use startups but for production hosting that you livelyhood depends on don’t.

Services that are "trying to make a VC rich" are a double-edged sword. They are prone to "moving fast and breaking things", sudden business model pivots and dramatic pricing changes.

On the other hand, they are often heavily subsided by aforementioned VCs in the name of building momentum. So if you use them cautiously with all this in mind, you can often treat it as an informal piece of redistributative taxation.

Yes. Certainly. But it depends if the savings make it worth the extra hassle. The venn diagram must have a small overlap.

The other consideration is DX where these startup infras shine. They make it easy to deploy and that can be tempting. But I am not convinced it is that much easier. Learn the slightest amount of devops (a weekend) and you can script what they did for your usecase. Use github actions or whatever but not too much! Just get that to call a bash script that does what you want.

I think there is a lot of value of trying to keep off locked in paths. But it is a delicate path to tread. And for startups growth is everything. But a few seconds of thought into tech choices and not just jumping into every freemium shiny tech is probably a good idea.

Sorry to hear that. For a rock solid alternative, I use Hetzner with Coolify (https://coolify.io) installed, it's an open source Heroku alternative that uses Docker. I deploy all my apps on a 10 dollar Hetzner VPS that can handle millions of hits, and that 10 dollars gets you a much beefier machine than anything you'd get on other paid services like Render, Railway, Fly etc. They just aren't worth using in my experience when something like Coolify does the job for a fraction of the price.

Hetzner is quickly positioning itself to be the Azure/AWS replacement for a lot of shops here in the EU.

Also for privacy reasons.

Do they even offer managed database services ?

If you don't even have rudimentary stuff like message broker/database you're not an AWS/Azure alternative.

Doesn't really matter when using AWS, Azure and GCP is not an option. They only have to compete and compare with other "EU native" solutions, not the US cloud giants.

They don’t. But I’ve explored launching a EU-based business that focuses on large businesses and Hetzner was one of the logical players. Good managed services without US exposure are really rare.

I run some smaller stuff on Hetzner. Have for the last 5-10 years and never had any issues. If Hetzner added managed Postgres, something like Cloud Run, and maybe PubSub I’d be moving over a bunch more stuff.

In my experience in non-SWE related enterprise this isn't really an issue. To put it as bluntly as possible, most EU organisations don't have enough work to staff an team for their IT operations. Maybe you could have one or two people on staff, but that is realistically too few people to run reliable operations, even in Azure or AWS. So what happens is that we outsource all the parts that aren't directly related to development or IT support and operations to 3rd party companies.

Most of those 3rd party companies will happily sell you managed database service that are run on Hetzner. So this part of it is sort of a non-issue for a lot of organisations if the cost are low enough.

Heck, even if you have your own inhouse operations department going for you, why wouldn't you be able to run your own stuff on Hetzner? That's what people did in the transition from having your iron in the basement and buying it from Microsoft/Amazon. The only reason we left the "own cloud" options was cost, and if Azure exceeds the alternative. Or the EU makes it too cumbersome to use Azure, then there won't be much of a reason to stay.

Hetzner is really awful with admissions of new accounts.

I tried to register twice but got denied before I even got the chance to complete registration.

`After reviewing your updated customer information, we have decided to deactivate your account because of some concerns we have regarding this information. Therefore, we have cancelled all your existing products and orders with us.`

No support, no details whatsoever, I'm not sure if they don't want to serve customers outside EU or something else at play, since I didn't even get the chance to finish registering.

I've read similar stories on Reddit, where you needed to deposit $20, but I didn't even get to that step. They seem too strict.

I might have better luck getting hired there than open an account with them.

Same experience here. I registered as business account from EU country. In couple of days they suspended my account and asked for a proof. I submitted company incorporation documents, publicly available from our country registry. They denied to enable it back again. When I asked on what ground, they denied any information, citing that other people will exploit that information. I was bummed and opened an account in their competitor (OVH). No issues so far with OVH.

Can confirm. I wanted to switch to Hetzner, but they denied my account.

For context, I live in the EU and send them a copy of my passport when they requested it. Still got denied. I asked them what else they would need, but they said their decision was final. Oh well.

I just went with OVH instead. No hassle, and I haven't had a single issue with them.

Interesting, I haven't had issues, but I'm in the US as well, not the EU. Even still, you could use DigitalOcean and other VPS rather than paying a premium from Render et al.

Yeah I already have a couple droplets at DigitalOcean. I just wanted to give Hetzner a try.

No, just don't.

I get the same instaban on DigitalOcean, from residential ip even.

Then something is wrong on your end, not their ends. Where do you live? Just wondering if there are restricted countries that are higher risk.

Not a restricted country, the Netherlands.

Perhaps your IP is on some kind of blacklist (due to past activities of the previously assigned owner of that IP)?

Might be worth investigating.

Doesn't seem to be blacklisted, but I'm not sure what is a preferred tool to check that. I do run firefox with aggressive privacy settings, so my bet is on that.

> Hetzner is really awful with admissions of new accounts.

> I tried to register twice but got denied before I even got the chance to complete registration.

Am in EU, had the same happen to me.

Tried creating an account when I was in university, they just suspended it for no reason, even before the process was fully completed. I even sent them copies of my documents to verify my identity, yet they chose not to do any business with me.

A few years later creating an account succeeded and I've been a customer of theirs since - no idea what changed, but I'm not going to move over all of my stuff to them after that experience, or at least keep all of my backups (that I need to restore my sites on any hosting provider) local to my homelab with rsync or something like that.

Then again, maybe being stringent like that works well for them, for whatever reason. That said, their prices are better than pretty much every other platform out there, at least for basic VPSes. Maybe Contabo can compete with them directly, though.

Happened to me as well, they made me send them a picture of my ID and still denied me access, the only company where I have ever had something like that happen.

Hetzner support is garbage in my experience, which shouldn’t be surprising given their pricing.

It’s funny, if someone posts a Hetzner horror story here, of which there are plenty, I’m sure someone will recommend Render in the comments. Lots of services are cool until they decide to hit you.

I had the totally opposite experience. I have been paying ~60€/month to them for nearly 10 years for a root server and storage boxes and I never waited more than an hour on any kind of support email. And this includes a bunch of times where I need someone to physically walk to the server and manually do something.

This has been my experience as well. Though I want to make it clear that Hetzner does not provide managed services on these root servers. They will reboot your server, attach a kvm switch, and replace failed hardware. Maybe when parent says support they mean they need something else.

I love Coolify for personal projects, but it's not ready for production/business projects yet. You can't really compare it with Render, Railway or Fly.

Some reasons why I wouldn't use it for critical infrastructure:

- It doesn't have high availability options (if a server goes goes down, your service goes down)

- It doesn't handle backups

It is great for excellent for anything non critical though

That's true, I don't run very important apps on there but even still, I can create backups myself and create multiple Coolify instances behind a reverse proxy if needed. It would be useful to have everything in one service though.

I feel like this is a message for the render support team, not hn. Idk why people are going here first as if that's the shortcut to getting a response.

Hit up here if render told you to stick it. But I also often see these end up with "I fucked up but want to yell at someone"

Especially because they should get an extra-fast support as a paid customer.

Extra fast ... == LOL

I’ve been a customer of Render for a few months. The common trend of these stories seems to be they are all new accounts, before the first bill has been paid. Seems their anti spam accounts processes need some work.

I mean, I am with you mostly, but I also feel sometimes these "public support requests" help me be a tad more cautious when considering the companies mentioned. There's a few similar mentions of the exact same thing happening to other users in this thread, so I've definitely taken away a mental note to be weary when considering Render. Doesn't mean I won't use it because of a single bad comment, but it just helps addding one layer of extra diligence before switching.

Weary of Render is what OP will be if they don't take care of the problem.

Wary of Render is what you (and I) should be.

Indeed.

There should be no case where a site is taken offline, without an attempt to contact the client first. Ever.

Anything else means the provider doesn't care about your uptime, and thus, should never ever be trusted for anything ever.

I've been hearing a lot of bad stuff about Fly.io lately, so this kinda helps balance things out for me.

Tbh after all of the issues with render, fly and heroku, I went with a VPS provider with custom CI, and I am not looking back

Which custom CI do you use? I presume an open-source that one can self-host?

Mine is not open source but I heard good things about coolify so I may switch to it

> I just wanted to provide a warning to others

You might be interested by my similar service: https://elest.io/cicd

We offer a similar experience to Render/Heroku but we do it on dedicated VM instead of a giant cluster with all the customers. So this means full isolation of your workloads. And you can deploy anywhere (112 regions supported and also BYOVM)

We support 17 managed DB (including Postgres of course) and also 233 open source software.

This looks nice. Can it handle distributed Elixir/Erlang like Render and Fly?

TBH I'm not aware about their specific offering related to distributed Elexir. But we do provide load balancers that can be easily configured from the UI, from our api or from our Terraform provider.

That’s not the same thing. In the Erlang world, the machines are literally distributed - meaning they are meshed and talk to each other (share state, etc). It’s one of the big selling points of Render or Fly over say Heroku.

> Full isolation.

What hardware are you using? Asking since Intel just got hit real hard with Downfall - another round of speculative execution.

We don't have our own hardware, we use cloud providers like AWS, Digital Ocean, Hetzner, Linode, Scaleway, Vultr to provide Intel, AMD and Ampere Altra (ARM64)

So to mitigate Downfall, we rely on cloud providers infra.

It's still way more isolated than a giant cluster hosting all the apps and routing becoming a single point if failure.

I had the same experience, I tried to deploy a repository of telegram userbot in python, they immediately banned without any notice

Was your account new as well?

Yes my account was also new

For anything business critical I usually assume that any "cloud" / "provider" / "registry" may go dark on me any second. Therefore, your own DNS VMs, multicloud, your own database, no cloud vendor locking (agnostic microservices) etc etc. And if you want full control - the on-prem / colo solution is still the king.

Your own DNS VMs? Probably good enough to use registrar for most people for this job. Just pick a good one.

It's 100% good enough most of the time.

UPDATE: I got in contact with Anurag Goel, the CEO at Render and he was very apologetic and restored my account and explained why it was caught up in some anti-fraud processes. I'll be giving them another shot.

I wanted to say that this isn't the first time I've heard of something similar to this with Render on HN.

So why didn't you?

They... did? Presumably because they wanted to?

I think there is some miscommunication here. When I read I wanted to, that made me think that they knew about this before OP's problem. If they had made a warning post like warning - don't use render. It is a piece of poop, maybe OP would have saved a weekend.

Really sorry about this. Can you email me (address in profile) so we can investigate?

Keen to hear from render about this. I just joined render and so I want to know what might cause them to do this.

Always happy to hear alternative suggestions but I really just love being able to git push and leave it to CI to deploy easily.

People get smacked by the cloud and their solution is to move to…….. the cloud.

Although the Render CEO is here on HN (@anurag), you should reach out to their support and not to HN.

I think a healthy name and shame is a good thing.

Presumably they already reached out to support, but getting your support query on the front page of HN is sure to increase the resolution speed.

Do both. Render should have notified Oxy before suspending service to let them know what the problem is and how it can be rectified. They messed up first.

By posting here, at least one other person has said they had the same problem and others are suggesting competitors. It’s useful to those of us who are looking for a hosting service.

And like you say, anurag shows up here occasionally to say things like “the company is doubling down on making the cloud delightful”. If they want to use this forum for promotion, then it seems only fair for people who have been hurt by the shitty behavior of the company to come here to complain.

A day ago, in response to another failure at Render, anurag was on HN saying “it's helpful to overshare in these situations”. Well, now they have another opportunity to overshare. What looks like a pattern of bad behavior may have a more reasonable explanation.

I make a number of infrastructure choices and it’s very helpful for me to know that Render is trigger happy with bans, shows misleading error messages and then doesn’t communicate.

I have assessed Render a couple of times for work. Both. times they were just too unstable for production (in 2020 and in 2023)

I’ve been using it since 2020, very happy with it. This post worries me though.

(Aptible employee)

I help run another PaaS, and I’m very curious about your production evaluation criteria. Would you be willing to share?

For context, our PaaS has been focused on running reliable and scalable production workloads for about a decade. We don’t really do hobbyist use cases, sadly. But instead we prioritize actual businesses with production use cases.

If you share your evaluation criteria, I would love to test Aptible against it. I can share the results with you if you’d find it interesting, but frankly I just want the results for my team and me. I’d be so grateful, and happy to return the favor in any way I can. If you prefer, you can email me at henry AT aptible.com

What did you find unstable exactly?

their IaaS system. it hangs. this gives very bad CI pipelines that you have to surveil. also it makes, eg. Github actions, insanely expensive as they are billed. per. minute, and render can hang upwards of hours or unbound if not stopped.

Ouch. We'll investigate and prioritize a fix.

I understand the value of all the PaaS and SaaS, but I could never get 100% behind them, even when they started. The highest I ever went was DigitalOcean (so IaaS). I tried some layers on top, lime serverpilot (automated php hosting on your DO droplets), but the entire thing always felt too risky, like giving the keys to your kingdom so someone whose pricing, policies and features you had no control over. I decided to always stay on a layer that is so commoditized that behavior (business, pricing, features, failure modes) is basically uniform across providers. I am atrações to killer pricing and stay away from killer (read lock-in) features. If I need a platform, I setup an open source one, like kubernetes, coolify, etc.

This would have been fine if these startups grew on the merits (like DO). But they want to run infrastructure like they run a social network. No.

We've resumed all services for OP. The tl;dr is this was a new account with no billing history and on track for a relatively high monthly spend. We attempted to authorize the card on file for the projected amount, and the authorization failed, leading to automatic suspension.

This is obviously overzealous abuse management, and we will fix our systems so this kind of thing never happens to legitimate users.

To OP and others who may have been affected, I'm deeply sorry.

As someone considering migrating from Heroku to Render, the suggestion that you'd suspend an account — paid or not — without notice is concerning. You said you'd fix your systems, but didn't specify how.

• Will you pause the resource utilization, but still allow log-in?

• Will you give notice?

Yes. We will find/build a way for customers to verify their account in these cases before pausing compute.

"We attempted to authorize the card on file for the projected amount, and the authorization failed, leading to automatic suspension."

Does "render" (whatever they are) own their own infra ?

If so, this is a weird response since, really, who cares what the retail monthly spend is projected to be when, on your end, it's just some ineffable, tiny portion of your electricity and transit bill.

Right ?

Unless, of course, "render" (whatever they are) is itself hosted on an underlying cloud.

Let me guess: you don't own your own infra ?

We've chosen to run on AWS/GCP for now so we can focus on building higher-level abstractions that help devs move faster, and we've always been very public about it.

Hetzner might be a good choice, cheap and simple to use. You manage your own infra, which may be a positive for some.

+1 for Hetzner. I run my own server infra using FreeBSD. Much cheaper than GCP where I was before.

Hetzner deleted my dedicated server a couple of days after signup, and banned my account. A friend had the same experience. At least I didn't get charged anything for it (IIRC billing happens at the end-of-month).

I even went through their live-video face verification and passport scan and they didn't budge on this.

And yet in spite of this the majority of abusive traffic we receive is from Hetzner.

What’s the point of these draconian identity measures if criminals are abusing others through your service anyway?

I hope this is sarcastic, it's causal. They have identity checks because people use it for abuse.

The less strict they make their identity checks - would abuse increase, decrease or stay the same do you think?

They might be preventing new malicious actors from signing up. Maybe. Catching a lot (anecdotally, from this thread) of legit users along the way.

Meanwhile, what have they done to catch and ban malicious actors who passed verification long ago and have been abusing other services on the internet from the safety of Hetzner's network for months or years?

From what I've experienced in the few years using them is that they're pretty strict on abuse, new account or not. I think initially they would null route all ips for the server when getting copyright reports but the past few times they would just email me the report and I had to reply with the fix or else they would null route (or maybe suspend) the server after a day or so of silence.

While I do run a mail server with them I've never had any reports so I can't say anything on that but I'd assume theyre equally or even more strict with email abuse reports they get.

We are, of course, tacitly assuming that all the users on this thread were in fact doing legit things from Hetzner’s perspective when they were caught up (and, given the ID verification, that they have unremarkable pasts from a risk perspective).

Although I’d be sore too if I were turned away in the signup process, I’d just as soon learn early that they don’t want my business, before I invest more time and effort getting moved in.

Was there any explanation/reason given for the ban?

Hetzner first made it impossible for us to pay, then implied we were fraudulent for not paying, then as a special "grace" fixed the payment problem and "allowed" us to keep our newly created account.

By then, it felt very shaky to depend on such a company, so we switched server to Digitalocean.

Hetzner are cheap though.

Edit: we did also pay, but they never acknowledged that first payment. Then we somehow had an ”unpaid” invoice which we couldn’t find a way to pay for.

Sounds like classic German customer service. You are wrong or have somehow used the service incorrectly until proven otherwise.

Oh, I see.

I prepaid some invoices, they didn't activate the service because of a technical bug (if you change the bank card, services get cancelled, and not renewed even if you have prepayment, it just goes as unpaid).

They said that as a "goodwill gesture", they are willing to activate my access, and keep all the prepaid money as compensation.

Well, that's my guess, because actually they just replied with 3 dots "..." when I said it was not normal.

Btw, if you are Ukrainian, you can have the "privilege" to prepay for your services instead of getting banned.

https://de.linkedin.com/posts/dennydo_product-overview-activ...

And they are "nice", they are "not planning any sanctions against Ukraine" for now.

Fair. I'm not "planning any sanctions against Germany" for now, despite Hetzner.

Nope:

>We recently did some routine reviews of our customer accounts. We noticed some suspicious information in your account.

>We have some concerns regarding this information and we have decided to close your account.

>We do not share details about why certain accounts appear suspicious. Publishing this information would make it easier for people to create fake accounts and abuse our services.

>We apologize for the inconvenience.

They didn't even let my finish my signup. I also did the whole passport scan thing, but they just plain refused it without any further explanation. I would've happily pre-paid for a year if that would've solved the fraud risk, but I guess they just don't want customers?

If you had prepaid, you may have been into a nasty surprise (see my post below).

Did you mine CHIA on their servers?

No lol, I installed Proxmox on there because I needed a server with AVX2 support for software testing.

Also running everything on Hetzner that can possibly go there. Then I regularly see startups proudly explaining how they are wasting incredible sums on AWS when the same thing could be done 100 times cheaper with Hetzner and Cloudflare. Makes one think, dude, why??

Bragging rights.

"My AWS bill was $100k last month" is the new "I'm a manager with 10 direct reports".

In some worlds this is so true.

Got bankrupted by a runaway process? Might was well brag about it.

Mythic Beasts also good.

Could it be that some keyword triggered the spam filters? It still doesn’t justify silent deletion, but spam fighting is so painful.

That’s a business problem though, not a customer problem. And even more so now that this customer tried to move all their services, failed and has to move back to their old provider. This business will never ever get this customer back.

Try porter.run for full control over your infrastructure with Render style experience on AWS

Reply on news.ycombinator.com