Why did Heroku fail?
matt-rickard.com
matt-rickard.com
Heroku made reproducible builds and deployments a thing at a time people were used to ssh'ing in and scp'ing files around for deployment. While 12factor later become canonized it was built into Heroku since day 1. Heroku was created because you could spend a month building a MVP app in Rails, but then it'd take as long to deploy it as to build it. Deploying software was too hard. And today we're still trying to get back to that, sometimes very unsuccessfully by adding abstraction layer on top of abstraction layer. Did Heroku fail? We're still talking about it over 10 years since being acquired as a gold standard for developer experience.
Okay, fine, but it wasn't acquired for something like GitHub... While two very different companies Heroku was acquired nearly 10 years earlier. It was the first large scale acquisition out of YC. I do not know the internals of YC, but it's been commented by others that know it better the exit of Heroku helped greatly YC for the time and place it was. It was a different time. Heroku to this day generates revenue and likely it's different from what people expect.
Yeah this one is selfish, but you can largely thank Heroku for Amazon RDS. Way back in the day we had Rails devs asking for a database. We thought how hard could this be, it turns out it was much more work than we expected. We bet on Postgres because one of our engineers said it had a great track record of security and reliability (not playing fast and loose with data semantics)–it was the right choice. Years later when Amazon adding support for Postgres on RDS the team sent some personal notes that essentially said, this is because you made it so in demand.
Now... what went wrong.
Yes, several people left after the acquisition, some at 2-3 years, some at 4-5 years, some are still there and have been before the acquisition. Acquisition or not when some of the technical and product visionaries leave it's hard to replace that. Adam gave his absolute all, he was spent after. 12factor felt like his going away letter. But that void was never fully filled, some of us may have had a shot of it, but after long runs were also tired.
Was Heroku cocky at times toward Salesforce integration, at times yes. At times I don't think was always the case. Did we want to run 20GB J2EE apps when Rails/Django/Node were lighter-weight, of course not. Did we want to login to gus to open a support ticket on a VPN? No. Did we want to move into the SDFC offices that were cube farms vs. large ceilings with a lot of natural lighting?No. But could Salesforce help us make Heroku more available to customers and accelerate adoption? Sure. Could we have focused more on enterprise requirements around security and compliance to reach the enterprise audience? Yep. In retrospect maybe we should have proactively integrated a bit more, I know many of us there at that time feel that way, but there were pockets of integration. I recall hosting the Sayonara team at the Heroku office for a Friday happy hour. We made sure to have it well catered and make them feel as welcome as possible. There was good and bad in the integration, but a common metric of acquisition metrics is how many that are employed at time of acquisition are employed x years later. Heroku had a lot of us still there, 2, 3, 4, 5 and even beyond.
Was pricing and business model a factor? Maybe. But I'm not sure you can get what Heroku gives you out of a single employee. I'm excited for what others are building in this space now, but it's not about being a "cheaper" Heroku it's about creating some advancements about easier networking, packaging what defines as an app as multiple services, signed/secure builds.
Did we want to login to gus to open a support ticket on a VPN? No. Did we want to move into the SDFC offices that were cube farms vs. large ceilings with a lot of natural lighting?No. But could Salesforce help us make Heroku more available to customers and accelerate adoption? Sure.
After an acquisition products normally fail as large companies are too sclerotic to adopt or even value new ways of working, so they just destroy what they don’t understand, often after paying a huge premium for it.
And then, Salesforce didn't really know what to do with it, and growth slowed down and good leadership left.
They collectively raised $13M through the Series B in May 2010 and were acquired for $212M cash in December 2010 (technically ~$250M for the whole deal).
A ~16–19x return, especially that early, is pretty excellent.
That speaks to the product itself and its future growth potential and place in the market more than current revenue.
Let's say that a product saves you 5 hours a week of developer time. If you're paying a developer $100,000, that should be worth around $12,500 to you, right? Except that people often don't see it that way. Yes, often they're being penny wise and pound foolish - they'll try to save the pennies on Heroku while spending dollars on developer time. But this can also be a bit of an accounting thing. Developer time can be counted as R&D while Heroku would be operating costs and cost of goods sold (I think, I'm not an accountant).
When Heroku launched, it felt like a great value. Fast forward to today and it seems quite expensive. $250 for 2.5GB of RAM and unspecified CPU? That seems to be 16x higher than Google Cloud pricing. I remember Heroku being $0.05/hr for 512MB at a time when AWS was $0.10/hr for 1.7GB. Sure, you'd need 3.4 Dynos equal one of those 1.7GB AWS instances, but that would only cost you $119 which is a 65% premium. Today, you'd need 3.2 Performance M Dynos at $800/mo to equal Google's 8GB boxes that cost $50. I think that stings a lot more and feels like something people want to look for alternatives to. In essence, it feels like an unfair markup.
It also feels like a company that has lost touch with the market. If Heroku has gone from charging me a 65% markup to a 1,600% markup, have they decided that acquiring new customers isn't important? Have they decided to give up and milk existing customers until they die? You can say, "no, we think we're charging the right price for the value we're providing and the time you're saving." That might be true while it also sends a signal to the market that you aren't trying to grow - that your path to profits is via heavy markups. While cloud pricing came down over time as servers became cheaper, Heroku remained almost the same on price. It certainly didn't match the changes in cloud pricing.
There were two roads: one where Heroku tried to grow marketshare at lower margins and another where Heroku kept pricing stable as cloud pricing dropped around them. Given that Heroku would have a lot of negotiating power with Amazon (and at the very least could get committed use discounts), they could make up a lot of margin there. But Heroku now just feels expensive.
The thing is that you're right that Heroku is still kinda the gold-standard for developer experience. You folks were awesome. But if I'm going to be running a bunch of services that need the equivalent of 100 8GB servers, that's $5,000 from Google or $80,000 from Heroku. When there's the potential for $900,000/year in cost savings, you start looking at getting off Heroku - and that's just with 100 somewhat small boxes by today's standards. Plus, buying direct from a cloud provider means I can get committed use discounts. The problem with Heroku's pricing is that it only saves you time/money when you're small. Do you want to build on Heroku knowing that success is really expensive there? Maybe, but it certainly isn't the "of course" that the old 65% markup offered.
I think companies are just operating services with a lot of scale today and today's Heroku markup becomes several engineers' salaries very quickly. Yes, Heroku means you can get a lot out of one employee. When you have 50 engineers and 500 8GB box equivalents costing you $400,000/mo...maybe you wanna task 3 of them to in-house a PaaS. Even if it isn't the gold-standard, there's several millions of dollars on the table each year. What's the sales-pitch for Heroku for a company running a bill of $400,000/mo? "We make your developers more productive." Sure, but we think we can get K8s and a PaaS team running for under $2M/year, spend $300,000 with Google, and pocket $2M+.
I feel a bit wrong writing this, but I think at some point Heroku is too expensive and it's so much cheaper to build your own in-house PaaS team. I think Heroku's high markup makes that tipping point sooner than later. Fly.io is charging me $92 for 2CPU/8GB which is an 84% markup over Google's pricing. If I have 500 8GB boxes, that's $46,000 with Fly or $25,000 with Google. There's $250,000/year in the difference. I don't think 1 engineer makes a PaaS team. To get the $4.5M/year differential that Heroku hits with 500 8GB boxes, I'd need over 107,000 boxes from Fly. Even to get to a $250,000/year differential, it would be 6,000 boxes with Fly. Yea, seems totally reasonable to spend 1 engineer's cost on PaaS markup with 6,000 boxes.
With Heroku, 500 8GB boxes means leaving $4.5M on the table. With Fly, 107,000 boxes means leaving $4.5M on the table. 107,000 boxes feels like a problem for me in the far future. 500 boxes feels a lot closer.
I really feel like I'm wrong and I've missed something, but I keep going over the numbers and Heroku just seems extremely expensive. Again, when it was young and charging a decent markup over AWS, it was such a no-brainer. Sure, give Heroku that money. Either you're small using a few Dynos and paying nothing and getting the amazing no-headache PaaS that makes you so productive or you're large and you're getting the equivalent of 500 1.7GB boxes for $60,000 when AWS would charge you $36,000, but $288,000/year isn't that interesting compared to what you're getting. Unless I'm missing something, Heroku's pricing has just gotten so high that even a bit of scale and you want to move off it. But I might just be wrong and missing things.
- Impossible to add HTTPS (with your own domain) unless you pay for the LetsEncrypt plugin,
- Impossible to add monitoring unless you paid the monitoring plugin,
- etc. for all the load balancer, the DB…
It means, as a guy founding a startup (now with 1m$ ARR), the initial dyno was $35-70/month, but I was afraid the upselling for all the 12 factors would represent 10x, 50x the pricing of the basic dyno. And that price expectation was unaffordable.
Think about this alternative:
- Sure you can use your own domain if you program the LetsEncrypt thingy in your dyno… OR use your $5 addon.
- Sure you can build monitoring in your dyno… OR use our $10 New Relic addon.
You may think I’m stupid, because in the end I’ll always use the paid solution, but at least I can run with extremely low expenditure at the beginning.
Launching a startup is extremely hard. You need to squeeze prices and make do with barebone services. I used DigitalOcean precisely for this reason, with Ansible deployment. I spent $5 a month for 2 years. I tried Heroku several times but it had limits everywhere (specifically background processes, while DO doesn’t have this limitation). Then money came and I went to AWS because it was cheaper and less limited than Heroku. Now the money is flooding.
Sure, it's a bit of a work, but the number of people who could set an ok server without paying that price is not low (also you'd be paying this for something that's almost a one-off work)
I bet if we had 10x the revenue we still might've had the same outcome. When you're a relatively small part of a big corporation standard capitalism rules don't really apply.
I've spent the much of the last 10 years hoping that I once again get to be part of such a failure. It's such a rare opportunity to work with an amazing team of talented people who set a new standard for what a great developer experience is, how simple it should be to run containerized workloads, that a robust build pipeline can actually be easy to use, setting the precedent for what a tightly integrated SaaS ecosystem/marketplace looks like, review apps, buildpacks... things that improved the lives for literally millions of developers. And even a decade later continue to set the bar for what many aspire to.
An amazing legacy to have a product that has been so relevant for so long in what is such a fast moving industry. I wish everyone got to be part of such a failure at least once in their careers. It's something I'll cherish for the rest of my days.
Heroku was the biggest in a long line of developer tools. The PaaS and now FaaS models are still alive and well. A lot of developers want them. I know people are hot on containers and k8s but devs, who need to focus on code and business logic, have poorer productivity when they have to shift into thinking about ops.
Heroku inspired a lot including direct competitors (like Cloud Foundry). A lot of money has been made and people jobs make easier.
On the flip side, I keep seeing businesses not wanting to invest in developer tools that make their lives easier. PaaS is one of them. Did Salesforce fail to invest in Heroku so it stagnated? How many other places is this happening? Did Heroku make enough money and that was it?
There are a lot of unknowns.
I think we need more PaaS and FaaS. Make the jobs of developers simpler (hiding the ops complexity).
My employees are the same. They want to tick all new tech boxes with our product, and they’re right, because they exit with 35% higher pay elsewhere (and I clearly don’t underpay them - this market seems to have no upward limit, and the only thing I’d like from them is taking ownership and responsibility, and it’s not happening).
- You get $0 on day one,
- After the job is done and it hits the market, you may get a million dollars.
What employees want:
- Be paid full market rate from day one even if it doesn’t work,
- AND ALSO getting dividends if it works,
It looks like you are just advocating to give them more more more. But let’s entertain the idea of implementing your advice seriously:
- I’d have to hire entrepreneurs who understand how market moves and what the risks are,
- I’d have to sell them stock on a given FMV that we determine together (Fair Market Value, =valuation per share), so FMV=0$ when the company starts, but it’s much higher when they join 5 years later when the product has already hit big and the good ideas are there. Estimating valuation is extremely hard and they would probably feel on the hook. If I value the current company at €6m, so let’s say €2m as a rebate, they’d have to get €600k out of pocket to buy shares; If they buy only €100k because they don’t have the lumpsum, they’d get 1/36th of dividends. We’ve made 1m this year, that guy would get 20k. You see, it’s hard to estimate a company clearly. If they don’t get large dividends, they’d complain I had cheated them with a too high valuation. And programmers in France don’t have economy education in their curriculum, so they’d think I cheated them, rather than them not producing the proper output (people with no economic education tend to get angry at people around them when they take the wrong decision),
- They’d leave the boat to work somewhere else as soon as the market shrinks, and they’d still get 1/6th of our dividends without working for us, which has a disproportional impact on those who stay,
When you think about how to do your advice in practice, I don’t think it solves what you think it solves.
I’m not telling you how to run your company, or offering any solutions. I’m just trying to offer some perspective.
Love this! Had not heard it before.
I've also noticed a trend of coworkers wanting to add a lot more servers than we need for capacity and redundancy. As in they don't even look at New Relic. Reality is most of the bottleneck is database or the code needs to be refactored. Code written for a few thousand records is often not going to work when there are millions of records.
Another argument is moving parts of the monolith to a microservice because that would take load off the server.
At least it has become a running gag in our company that anyone suggesting microservices is joking. It’s a good first-step towards limiting Resume-Driven Development.
For the scale of our cloud product (1000€/month for now), we could manage everything in memory.
* They had a massive security breach where core-db was breached. Hashed user passwords and encrypted env vars were leaked. (And a hell of a lot more I bet—but they haven't admitted this)
* They're operating with probably 1/3 the staff they were at peak due to attrition with no backfill and a layoff.
* They've been under a feature freeze for going on 5 years. Check Heroku Changelog to verify this yourself: https://devcenter.heroku.com/changelog
Yes: Heroku failed.
EDIT: just because Heroku has failed in 2022 doesn't mean it wasn't also successful. It's not too different than Flash in that respect.
That's on Salesforce not Heroku. Heroku left the building already.
(Although I would argue that Salesforce was also responsible for creating most of what we love about Heroku. That acquisition happened far earlier than people realize. They're also almost entirely responsible for its downfall.)
EDIT: Craig is right below (and he pretty much always is). When I said "created" I really meant in terms of giving them enough financial resources to keep building. SFDC exerted no influence in terms of product decisions early on (so far as I know). Re-reading my post I see that wasn't clear.
GitHub is an amazing parallel, folks feel that Microsoft turned around GitHub, but actions and much of the GitHub turn around happened long before the acquisition. Those sort of things take years, you should mostly try to judge a top down influence 4-6 years after they were there.
Interesting - someone at either MS or Github told me that the Azure DevOps team moved over to GitHub to create Actions. But maybe I misheard or both versions are true.
- That Jason Warner, Max Schoening, and a brains trust of Heroku Product & Design leadership jumped ship to GitHub. After they got there they continue their run of product success and innovation and launch the next platform innovation for GitHub. Which also just happens to be yet another way to run code as a service.
- The ADO team join GitHub, and then pivot into a whole new and foreign area.
People can make their own judgements on which of these is the most likely option.
Ah Cloud Foundry. Something I drink to forget.
CF was the tool Salesforce should've bought or written. Heroku not so much.
Now everyone and the kitchen sink „needs“ their own kubernetes cluster and operations teams struggle to build complex landing zones for the underlying cloud infrastructure so teams don’t shoot their own foot off all too easily…
My recollections were that BOSH and CF exposed a ton more complexity than I wanted, and I don't remember there being a great story for deploying Rails apps. But all of these abstraction layers are just like cross-platform GUI toolkits – you're never going to approach the native look and feel or integration. And I don't like the terminology(!) it's all relatively unintuitive to me (e.g. stem cells, deployments, releases).
For contrast the Hashicorp stack for our AWS environments (tf+packer+vault) generally felt much more pleasant to deal with, and that's where the comparison to GUI toolkits falls apart I suppose.
But… look at all the integrations and enterprisey features that CF+BOSH offer. That's way more of the sort of thing that I'd expect to fit in at Salesforce.
That said I think the Hashistack doesn’t address the PaaS component. There’s no cf push with HashiCorp tools.
It’s early days but maybe WayPoint will make that happen.
The article should, in my opinion, focus on why Salesforce allowed Heroku to stagnate and grow uncompetitive.
[Disclaimer: Salesforce employee, thoughts and opinions are my own]
- Salesforce did ~27b in revenue in 2022; its very hard for a comparatively small business to get resources / mindshare in that environment, especially when you focus on a market / audience that is different than the rest of the business. From a strategy POV, for better or worse, Salesforce is not particularly focused on integrated acquired products - its interested in selling them largely as is to existing customers. Fundamentally its a financial engineering and sales exercise, not a product innovation one. (Not to say I support or think thats the best way of driving growth, but offering my POV on the structural dynamics.)
I mean:
> From a strategy POV, for better or worse, Salesforce is not particularly focused on integrated acquired products
> I heard revenue and margins weren't sustainable prior to the acquisition
What was the point of the aquisition? To let it die?
After Apex was introduced in 2006, Salesforce suffered criticism (misplaced, IMHO) over the fact that it was proprietary. In 2009 it did a partnership with VMware to create a Java-as-a-service proto-PaaS so business logic could be written in Java instead of Apex. This service (VMforce) never worked for a variety of reasons, leaving Salesforce to look for an alternative -- so in early 2011 they bought Heroku. Thats why the first thing Heroku did post acquisition was build Java support.
Turns out this wasn't a great product idea to start (the limitations of externalizing business logic, especially at that time, weren't worth the effort), so the strategic reason for the acquisition was no longer relevant. Years later there were successful integrations that had different value propositions, although those suffered from the constraints described above.
I've seen companies who build on Force.com use it for <we need a standalone application that does X because Force.com can't do it or we have teams that don't know Apex but want to do X> and then integrate it against. They have an interesting (albeit not sure if it ever got fully fleshed out) bi-directional DB sync between SFDC and Heroku Postgres, which to me is the killer app for integration.
That said, having good ideas is nothing if you can’t prove it in the market, but I do think Heroku will be around for a very long time by the nature of how sticky its customers can be, so it’s hard to use that to judge the future.
Definitely true, but SF is extremely interested in marketing its unintegrated products as fully integrated.
See for example: SF marketing cloud, where the only integration is an iframe for your SF session and a fresh branded subdomain (mc.exacttarget.com chefs kiss)
I do wonder though, do folks not have super clear plans for these acquisitions? They sometimes stagnate surprisingly. We used a smallish product (maybe 5K/year) they get acquired, they dropped a key feature and replaced it with a paid but not compatible / worse feature, then reached out with a sales person to talk about why we weren't using it and rolled our own replacement. I mean, huh? Of all the strategies this was going to be it?
Instagram a notable exception here.
Oftentimes, getting the deal done is such a rush to the finish line, mixed with celebration, that making all of these grand plans a reality is someone else's problem, time to go fix the next thing. It's not logical or rational; in reality, the euphoria of closing means the real work has just begun.
One of the hardest things to do is maintain momentum after a major accomplishment - either by buying the company you so coveted, or by selling the company you built.
Instagram was an outlier as far as acquisitions go. They had 16 employees and massive momentum. Facebook was highly motivated to make the integration happen. They cherry picked strong engineers to join the Instagram team and drastically increase their engineering capacity.
From my point of view, as a mere customer of their mass-market product (a.k.a. cedar) with some insight into operation history, I think expensive or at least detailed investment in refurbishing or adjusting the mass-market product became, for whatever reason, difficult to (internally) justify expense for.
For example: the mechanisms that make "heroku run" work could have really used a new version that could let me change my window size gracefully, and maybe streamlining some authentication, by using SSH. As far as I know, it's closely related to the first version of that feature ever written, which somewhat naively ties the client machine's tty to a remote tty via TLS connection. This is fine for getting cedar off the ground in the first release, but at some later year (decade?), it seems like it should have gotten re-do so, finally, I could change my terminal emulator window size.
Ditto the holding of extremely powerful API credentials in the plaintext file `~/.netrc`, further exacerbated by a fairly weak authorization system for Cedar. There's still a place for .netrc (e.g. so curl can pick up the credentials automatically), but at some point, use of the keyrings on various operating systems (at least) and U2F hardware attestation should have shown up.
Passing along cost reductions in inputs (e.g. AWS servers) is another long-standing example.
Somewhat related to that, IPv6 is a lot more common than it used to be. So are VPCs or similar products on other platforms. Putting all that behind a "let's have a call" process -- and cost structure -- for "Private Spaces" is not in step with the times in 2022. And probably not 2018, neither.
As a counter-example, it does seem the web/dashboard login not only grew U2F, but Macintosh TouchID support. Whoever got this in: well done. But it's a noteworthy exception. And I kind of wonder if it has anything to do with the web authentication system being more trouble than it's worth to segment between Cedar and the Private Spaces product.
There's always a temptation to put new features into an upmarket product to increase the gap in segmentation, but realizing that at least some features are closer to adaptations to raising or at least changing standards of adequacy in industry does not come naturally. I don't think this is the only problem, but it seems like a problem to me. As it comes to projects that turn on minute details, such as improving "heroku run" in the presence of terminal emulator resizing, it probably says something about the bureaucracy's inability to grapple with details of that kind. That was already present when I left.
Finally shipping something major (the Europe region) was a massive effort with a lot of internal political struggles, as was performance dynos. That was why I left, because the writing was on the wall that an organization which had atrophied its ability & urgency to ship new product was not one that could continue to innovate in a meaningful way. Not to mention that headcount was already being starved at this point, so you couldn’t throw people at the problems.
I don’t think most people realize just how small the headcount working on the core platform product was, like 8-15 people for the whole time after acquisition.
There were a handful of additional big successes like private spaces, compliance certifications, metrics, etc driven by the few remaining people with the burning will to really push stuff forward. But eventually those people moved on, and by that point the company had already lost SFDC’s interest even though revenue growth and profitability had finally come around.
Forever thankful for the core group of amazing people like you that I was fortunate to spend a few years living the dream with. I’m glad I appreciated how special it was at the time!
What CIO/CTO is going to bet everything on some magic high level platform?
The product and pricing are pretty good for small projects, but almost everyone outgrows Heroku well before they transition from "small" to "medium". The pricing model hasn't changed in many years. Dyno sizing hasn't changed or improved pretty much ever. Now that compelling alternatives to Heroku exist (Render, fly, even Netlify) there's no good reason to launch a new project on Heroku. Hell, even Elastic Beanstalk manages to give a decent experience for many folks compared to Heroku.
This is definitely false. Though some internal apps are deployed on Heroku, it is not the primary deployment target for new projects. Heroku staff work on Heroku. Salesforce has a large internal infra program and team that has no overlap with Heroku. past codename projects have been proposed to change this, but none have ever come to be.
I've heard this as a reason for leaving for almost a decade though so it hasn't been a new problem but it seems to get worse every year.
You understood it as running sfdc code on Heroku which is not happening—but was proposed as part of Shinrai (a failed project). What has happened is many if not most Heroku employees have moved away from working on Heroku and are now working on Salesforce Functions aka Evergreen.
It does feel like there continues to be a fading at a leadership level about developer experience and what it means. There may not be strategy for what makes Heroku the next Heroku whether engineering or product, but the team that is still there is a solid set of engineers, not 100% there for Salesforce internal stuff, and kudos to them for still keeping afloat what is one of the better ways to deploy an app.
If anyone at Docker is reading this, I mark “100% no” when you interrupt me with pop up asking if I’d recommend your product to a friend when I’m in the middle of working on something entirely unrelated to your product. I don’t know anyone who wouldn’t.
* The industry – suffering because Heroku holds market share that should go to competitors capable of innovating.
* Customers – suffering because their platform will die but their vendor isn't being transparent about that fact.
* Herokai – suffering because they could be doing something better with their careers, even if it is on another sfdc product.
Make sunset plans now and let customers know what the timeline will look like. "Trust is the #1 value" at sfdc, right?
As someone who said "I'm sure it can't get any worse" with heroku for many years and was proven wrong over and over, I genuinely don't know what you mean by this. Lots of stuff just doesn't work. Updated buildpacks don't get released on a useful timescale ("just make your own" isn't an answer). The platform is plagued by bizarre performance issues and noisy neighbors, and the pricing doesn't support it. In my time on Heroku, I couldn't break three nines of uptime one year entirely because of platform issues, including one incident where the DNS provider Heroku uses just stopped working in Israel (which Heroku gaslit me about for over a week). I don't know what "operating at scale" means to you, but if the service is consistently bad compared to the competition, it doesn't matter what scale they operate at. It's a meaningless metric.
Pretty sure none of these can run any JVM applications. Heroku can, very easily.
People here underestimate the cost of wasting time setting up docker containers and fiddling with configuration files. Heroku is about delegating 90% of your Dev-Ops and actually focusing on your product and it pretty much works flawlessly with many stacks.
I looked at all the alternatives that people mentioned here and none of them hold a candle to Heroku (yes, even in its current degrading state). Perhaps eventually they will, but so far, I am not impressed.
Now if you excuse me, I have a product to build :)
There are still plenty of talented engineers at Heroku.
I used to do architecture work and support for enterprise customers and I knew the whole support team. Heroku support engineers are all devs or former devs; I am surprised at how passionate they can be about their work (ex: Jason, who STILL keeps the subreddit updated in his spare time).
I do think there are situations where support can't help, but those are usually situations where Heroku is probably not the right place to host a service. They can't help you setup crypto miners on the platform, or to setup a redirect service. If you don't like how the platform works, they can drop in suggestions on what to change, but they aren't going to be able to implement them overnight. You might get ideas on restructuring your app to work better on Heroku, which I admit is quite frustrating for a customer; but limiting the problem domain is part of building a service like Heroku.
Heroku's support structure didn't have a half dozen escalations, which was good IMO.
If you run into an issue on the platform, there is usually the support engineer and then engineers who developed it. This might have changed around when I left (my team was mostly eliminated during the pandemic), so I'm not particularly up-to-date on recent changes. However, the system they had in place when I was there felt much better, as a developer who needs help, than the support I currently get from (large IAAS provider), despite myself doing work for a major customer of theirs.
If anyone is ever in a position to hire support staff and you see "Heroku" on the resume, definitely consider that a strong positive signal.
This hasn't been my experience at all. Mysterious performance issues are met with "We can't help, just install NewRelic" (the issues don't exist when running the same code on other services). Billing questions have yielded delayed and cryptic responses. Table stakes customer support simply isn't being addressed. And I don't blame the support agents, I just don't think they have the tools or resources to help.
By comparison, AWS has managed to answer almost all of my questions within one or two messages. Hell, even Amazon has admitted when there's a bug on their end. That simply doesn't happen at heroku anymore.
Yeah, there are definitely details Heroku didn't have; for example, we only had something like the past 500 (or was it 100?) lines from a customer's logs; you would need to have an add-on to retain them.
There were no internal logs to consult about app performance; the New Relic install requests were probably to see if there is an issue related to the app. If the issue went deeper, we would work with the engineering team for the feature in question; but with the volume of tickets, we would have to categorically rule out an app issue first as 99% of issues happened while I was there. I think it is fair to say Heroku's support tried to embrace the "teach a man to fish" strategy, and that wouldn't work if the problem wasn't with the app.
I can't really speak to the billing issues, but that team was tiny; two or three people. I could definitely see them getting overwhelmed, and I heard about the lack of detailed billing support.
(Out of curiousity, were you using Heroku's docker product? I could definitely see that as having caused the sort of problems you mention as Heroku's docker platform was not vanilla docker.)
It's clear nobody at Heroku cares anymore.
(For the record, I loved Heroku and still do. This makes me so sad. I'd love to see Salesforce spin it off somehow, rather than slowly letting it suffocate.)
They might lose that position in the future due to many missteps, but at this point it's like asking "why did Microsoft fail?". It didn't. It might not be what it was, but it's still a pretty successful concern, and same goes for Heroku.
Having alternatives is very cool. I did migrate an app to Render, motivated by Heroku's recent security woes. But in the process of doing so, I realized that Heroku has accumulated a lot of plugins, conveniences etc. over the years. If Heroku dropped their prices to be a bit less egregious, I probably wouldn't bother moving.
It will take quite a while before anybody can match them on convenience, and eg. Render is definitely not there yet - it has the basics down and done right, and if the basics are enough for you, it'll work. But a lot of things you get out of the box one-click support for in Heroku will be a manual setup on the alternative PaaSes, with a community wiki article to guide you at best.
A great deal of the movement to containers lies on the shoulders of the DevEx that Heroku pioneered.
There's a lot of new investment in a lot of new projects; it's cool to see. I hope they carry the torch forward and are successful in their own right. I am looking forward to deploying code on all of them.
Obviously given the hype around k8s and the order of magnitude difference between the success of the hyperscalers and the scale of Heroku, there's some mismatch btetween what they did and the potential they had. I would say the same about App Engine and AWS's various iterations starting with Elastic Beanstalk.
Overall if we had to bucket the opportunity that still exists to do better in the space, I think it aligned with wrong product and something better than "git push heroku" better than with business model, whole product, or timing (which clearly had a role). Business model is an issue but not against stuff like render, the success of which also pushes against whole product (people are still adopting something very similar).
Agree with craigkerstiens that the next iteration is about apps vs. services, extensibility, and blending different service types (frontend/container/serverless/data) more fully than Heroku (which really only has one type of app) was able to. And is also about lifecycle and SDLC completeness (which helps a lot with security and compliance, too).
As I mention on all these Heroku threads lately, I'm the cofounder of Coherence (www.withcoherence.com) where I think we're taking an interesting stab at what the future looks like here, using first principles to go deeper than other PaaS and integrate dev more deeply, along with more support for extensibility.
there was seemingly no logical reason to do any of this-- the product was at an early stage with nearly no traffic. even if the product were to become immensely successful, the traffic would probably never require more than a single box.
but none of this seemed to matter. nor did the considerable amount of time making things just work. the ceo, who was at one point a technical engineer mind you, just wanted to be able to say his stuff ran on aws
situations like this are probably more common than you would think. it's purely a brand recognition thing for them
Was the company successful after all?
The company was acquired at an early stage by another private company.
It has its weird limitations but it's... Good. (And I've gone pretty deep into aspects of the buildpacks, how it stores its git repos on S3, and more.)
Biased take from the founder of one of the next-generation companies mentioned: given how new Render and Fly are as companies and products, it's much more important to look at the growth in their adoption. Render's signups were already growing at a rapid pace, but more than doubled overnight after the Heroku incident; give us a couple more years before judging adoption!
I am myself maintaining a custom built webshop/ERP system for a local company that repairs smartphones. I barely find enough time to maintain the actual code. If I would have to dig through the weeds on AWS the project would definitely be dead right now. Instead I just "git push heroku main" and I am done. I can also provision a database or Redis through the UI, without having to worry that I made a mistake that could cost me thousands of Dollars.
- fly.dev
I'm sure there's others, these are just the ones I happen to remember at the moment.
- For languages, they support less stuff than Heroku (unless you're willing to do Docker yourself).
- For DBs, they can do Postgres or Redis on their end, other stuff is on you if you need it.
- Out-of-the-box metrics don't include basic stuff such as requests-per-sec or http status codes, so you do have to plug something external in for that.
+ HTTPS on custom domain using LetsEncrypt was a breeze and free.
+ They have "disks", which is a persistent filesystem for your app. Very useful if a normal FS is better than a DB for any of your things (image processing in my case).
Is there any innovation apart from the increase in advertising? Tiktok offers a new suggestion algorithm. Why has something like that not been invented at Youtube? Why does Youtube try to copy Tiktok a bit but does not generalize and does not offer more ways to access videos?
At their position, they could turn Youtube into a platform and let others do the innovation. Instead, they lock it down and double down on their limited user experience.
In 30 years, when something else has been established, Youtube can ask itself why it was not them. But it will take time for a competitor to establish itself and overcome the network effect.
Seems premature to assess adoption of fly.io when it's barely gotten off the ground.
The story in all of them was the same: Heroku is nice and simple to start with the free/lower plans, but gets too expensive as you grow a bit more.
Once we'd grown past the free/lower plans, we'd move on to somewhere else, that required a bit more work, but was substantially cheaper and way more flexible.
Also an issue was a reproducible environment for several devs. Using docker became super easy, but this didn't translate well into Heroku. We either needed to maintain buildpacks and dockerfiles in sync, or move to a platform where we ran docker. Not sure if this has changed since, but this was also a big reason to move elsewhere.
> for all that people worry and for all that the press loves to say, 'Ah, this happened; this thing is over,' and whip themselves up in this frenzy, it never actually kills the startup. It's a problem; people love to talk about it. And there's this schadenfreude of loving to hate on someone's perceived missteps or someone's problem. So people love to talk about it. But then they go back and they book AirBnB for their next vacation.
Econtalk Podcast: https://www.youtube.com/watch?v=JAYXUgNHHc4&t=16m25s
Heroku solved the problem very well for years but it's very costly and limiting at the same time.
Now it's time to replace Heroku with services that help you use the cloud providers which leads to substantial cost savings.
We're very soon launching https://www.utopiops.com which is an alternative for Heroku on AWS to address this particular issue.
E.g: Qovery.com is the Heroku-like experience on top of AWS
(I am co-founder of Qovery)
* expensive to scale
* developer limitations
This limited it to smaller scale projects, MVPs, proof of concepts where paying a little more for a certain range of developer productivity makes sense. Such is the case with rapid development platforms in general. When you grow you shift to wholesale-priced primitives that give you more control.
If being in every Fortune 100 company running some of the world's largest containerized platforms in production is failure, I want to fail like that
Heroku might be a bit long in the tooth, but has a solid company behind it and it a lot easier than fooling around with Capistrano, Ansible or heaven-help-us Kubernetes.
Trust me, I've tried them all, and for a fairly standard Rails app Heroku just works.
The lack of HTTP2 is a bit embarrassing though.
Any data to support this? I suspect Heroku is immensely profitable on account of its huge user base (4th largest cloud provider after AWS, Azure, and GCP), and very healthy profit margins.
What I love about HN is the exposure to new ideas and suggestions. I've seen suggested here that fly.io is a cheaper alternative that people rave about. I had a quick look at their site and so far I like what I see.
So come Monday morning I'm going to add a ticket to the dev queue: Investigate moving from Heroku -> Fly.io
We'll look at the pros and cons. Pricing. Ease of use. etc. The usual. And my gut tells me that heroku might be losing yet one more customer and the tens of thousands per year that my company is currently spending with them. We're a small team, so it was decided to spend the money on a PaaS instead of a large ops team. It's worked well enough for the past 15 years. So maybe time to move?
The selling point was easy and reproducible deployment. But there are must cheaper alternatives, and dynos productivity didn't have enough benefits for the cost.
Later there was docker and after setting up your CI/CD you had almost the same heroku experience with half of the costs.
As deployment became easier there was no need for heroku. Is was killed by Docker.
Yeah you still can use it. But sure it's pricey...