Don't solve the solved problems
blog.collections.me
blog.collections.me
"Don't reinvent the wheel." Classic advice. And it has some merit. But it's a broken idea.
Google? Not the first search company. The iPod? Not the first mp3 player. The iPhone? Not the first smartphone. Hacker news? Not the first tech oriented community site.
"Don't reinvent the wheel...
unless you think you can do much better."
Also, if you're merely iteratively improving the wheel, do so whenever you like. We don't drive around on hand-carved roughly circular hunks of wood any longer, and that's good.
The proper answer is: "it depends." But that's unsatisfying because it's too complex. But in reality you can't eliminate that complexity, it's an irreducible complexity. Just as there isn't one simple, objective system which you can consult to reliably get an answer to the question "should you go outside or stay indoors?" there isn't one for whether you should use something off the shelf or build something yourself. Look at, for example, SpaceX, who built the first generation of engines for their Falcon 9 rockets using turbopumps (one of the most complex components of a high performance rocket) provided by 3rd party companies. However, in the subsequent generation of engines (the Merlin-1D) SpaceX has designed and built the turbopumps themselves, and this is responsible for significant improvements in the engine.
Ultimately I think it's more important to take a step back and make a conscious effort to avoid making big decisions based on instinct or inertia and begin to cultivate a set of processes, elements of corporate culture, and habits which encourage thoughtful analysis and introspection about these choices. People should be more thorough about evaluating the advantages and disadvantages of doing things different ways and be mindful of when it doesn't seem to make much difference. And also mindful of the opportunity to change your mind in the future. Sometimes it can be worthwhile to rebuild a tool that already exists in the industry just for the purposes of acquiring the experience and knowledge of having done so. And sometimes even when taking something in house might easily save millions of dollars a year it still might not be worth it due to other opportunity costs.
It's a complicated question, and it's always going to be a complicated question. Instead of pretending it's an easy question people should be working on acquiring the skills to make sure they can answer the question well with some degree of reliability.
This seems like the correct way round to me for the majority of cases where subcomponents of your product could be externalised - a 3rd party should be the primary go-to choice unless a company cannot be found that fits the requirements. The issue I see in development (and have been guilty of in the past) is simply building everything when libraries could easily cover the use case - "not invented here" syndrome. That's the real problem.
[edit] Also I think it's clear that there'll always be cases where the default doesn't apply, for example when working with a new field/concept or where the 3rd party options are too far from the mark, but for the majority of development (especially web development) there's likely a library/service to match many of your requirements.
I can understand that competent sysadmins can recreate heroku's underlying hosting, but there's still a decent switching cost (especially if your whole team is accustomed to the heroku way). I've consulted with several companies that have 5-figure a month hosting on heroku that they can't migrate after all their investment.
If SendGrid changes my plan (or somehow degrades), it's a simple config change to send my email through another provider. A more apt comparison would be if your mail provider used a custom ReST endpoint instead of SMTP (say, as MailChimp does).
When you are raking in profits, you'd better use some of them to secure your business and make it independent of such risks. Which is the point the blog post seems to be trying to make.
Sometimes though you can make do with something significantly simpler if you restate the problem slightly and move the pieces around.
You might be able to do the core of the job with a cron job and some command line tools rather than the fancy SaaS platform with graphs and everything.
Building on top of the fancy SaaS APIs and then trying to scale down when you want to move away is a lot more difficult than starting with the simpler approach and then scaling up when you really need that extra functionality.
Believe me, I've done it in both directions.
The OP doesn't really talk about "solved problems", it talks about the companies core competencies.
I've spent almost a year on it and don't think I'm going to take their advice if it's all the same to them.
That's a lot more 3rd party services than most use - do you find major differences in reliability?
Is mongodb really a good solution for storing things? IIRC one concern about coinbase (YC) is that they used mongodb for storing financial information.
The Bolt-On Engineering for Dummies.