Selfhosted tech starter pack for development of new project or startup
github.com
github.com
Self-hosting all your own stuff has to be the worst thing you can do as a new startup. You need to be focused on impactful work, not dealing with critical security updates to your Mattermost install. And definitely not worrying about figuring out how to properly HA databases and your self-hosted S3.
Just use managed services.
You can save your startup thousands of dollars per month on services, with very little effort -- if you know how.
A larger group of people would struggle with the effort to do so competently.
Energy spent bringing more people into the first group is generally a positive thing.
This is the common wisdom which is correct in many cases, but not all.
If you have a decent systems/infrastructure person in your startup, the incremental cost of running a lot of these kinds of services is effectively zero. If you are bootstrapped, this can be financially significant.
Most startups do not need a decent infra person on day one, and no one would recommend hiring one for this purpose. But if someone on the team has the skills, it can make a ton of sense to leverage them.
I think it's good to help people obtain these skills which add value to (or at least save money for) their project, and correspondingly enable more people to start up in the first place.
At the start-up we ran nothing ourselves, except the gitlab runners. (Well, except for the first month where a bunch of laptops are used to prototype the app). I spent my first year writing terraform and helping design the product for low-cost scalability. So much of what we did for that whole year was in the free tier that our runway money could cover team building. 1 year in, I could deploy our entire stack from scratch into a new region in under an hour. VPC, subnets, security rules, IAM, queues, workers, Kubernetes cluster with autoscaling nodes, databases, DNS, CDN, monitoring, metrics, email (AWS SES)...
That type of fast deployability to a new region shouldn't be a late consideration after you've spent a year tinkering with in-house stuff. In fact, getting to another round of funding requires that you have that stuff ironed out. Can you support FedRAMP requirements? Yes, because that was a day one consideration. How about GDPR? Ditto. Do you have separate dev, test and prod environments which are identical apart from scale/cost? Yep!
Right now I'm working with a company that built everything in-house, from the perspective of "one single data center". It's a disaster. If you're not thinking at scale from the start, your probably not doing basic stuff like proper naming conventions, non-clashing CIDR. They're using all the right tools, but badly. Their terraform is a mess because it was obviously introduced it gradually on top of existing "snowflake" infrastructure. Their next data center was pretty much copy-paste followed by a thousand changes. And their code coverage is low - lots of stuff still manually created. (This is a known company with a decent product and lots of customers!). All of the people hired to try to make it scalable after the fact are saying things like "...it would be easier to just throw it all out and start over".
The problem with doing things badly because you think you can improve it later is that suddenly you've got customers on this thing, and you can't start over.
I agree you should use as much established products as possible but you build your stack and create that unique differentiation on the market by also learning technology as you go.
It all depends on where you land with your idea to the market. If you are selling knit figures online - you are right that you should not set this up but if you rely on technology advancements you should “own your technology” knowledge as much as possible because you will need it to solve problems along the way.
Too often I see business leaders come in and say I don’t know technology - let’s buy it all from MS or Amazon. This is not the way to go for your whole business. It’s like saying can you run my business for me. Your business is only as good as your best knowledge workers.
Another aspect is regulations, your data might not be allowed to leave the premise of your nation and these self hosted solutions at least with proper patching procedures offer this guarantee.
Managed services in all its glory to hand over the problem to someone else also hands over key learnings and if you have a complex enough environment you will need to deal with it at some point and for small businesses/startups the cost of outsourcing might be too high to begin with.
You are right that there is a time and place to start taking more ownership of those basic pieces. Doing it before you even have something to sell is not the right time.
You have to survive as a small company before you can be a mid-size company.
Revenue via leverage (managed services) once you have traction (people will pay you for what you're offering). Outsource everything except your core competencies, and optimize as throughput allows for either growth or cost reductions (judgement calls by competent practitioners close to the machinery). Broad strokes, as a startup you're attempting to grow revenue as fast as you can through value delivery to customers; you should be ruthlessly cutting anything you're spending your time on that doesn't contribute to that goal.
For sure, keep an eye towards your regulatory requirements, lock in risk (both vendor and technology), etc. That's not a call to self host everything, just what you absolutely can't grow or survive without (and in many cases, a product or service may come along down the road that allows you to refactor out of homegrown solutions). Organizational contributors who understand technology making these decisions de-risks suboptimal decisions being made.
(observations from experience at a hyper growth startup)
If something doesn't directly contribute to your competitive edge then run with a managed service / something off the shelf until you know enough about the domain to be able to make an informed decision about the trade offs.
Too often I've seen early teams that have spent months building out (and maintaining!) an area of the stack that could have been had off the shelf - freeing up engineering time and allowing focus on the areas that allow for true differentiation in the product.
Obviously off the shelf isn't free in financial or engineering time (though you can sure get a lot of free credits / on free tiers nowadays), but if your main product is doing something fancy on top of the output of a fairly generic process, do yourself a favour and find a company that has a product specialising in that process, don't build it from scratch until you have your product market fit.
What I would really like to see is an offering where anyone can just pick a suite of open source tools, all managed with the same SSO, and pay a flat rate for it. Something like https://restack.io, but not where I´d have to manage.
I am talking about a fully-managed provider of services. Something for those that want to have a set of services for their startup but do not want to worry about keeping them up and running.
> Warning: This setup doesn't provide high level of security or any high availability. You have to hire some skilled devops engineer (like me)) for close this gap after getting first round or sales.
And this is exactly why managed SaaS is popular. Pulling a repo of a popular open source project and deploying it to your server seems trivial, and you wonder why there are idiots out there paying through their nose for subscriptions. Then you realize your service has to be up 100% of the time, has to scale to more than a handful of users, you have to perform regular backups (and ensure they actually work), all user access has to be audited, you have to offer IT support to all your employees. Very soon your full time job at the company isn't building products for customers but managing an internal chat server. And then the couple hundred bucks a month other companies pay to Slack and Google start to make sense.
I think the idea of this guide is to reduce the amount of time spent setting up a self-host to a minimum, though I agree it's usually not cost-effective.
Imagine a young engineer in Bogota, or Ukraine if you want to get topical. They may barely have the hardware to accomplish anything compute wise... and your answer is "Use AWS!" Access to the cloud is privilege to the highest degree. You used to need academic credentials to use someone else's computer.
I get the idea of not wasting time, but as somebody who has used self-hosting as nothing but a waste of time that teaches me something, I can't believe so few see the value in a self-hosted starter pack.
Great architecture, OP. I challenge you to simplify the process even more... A lot of manual steps here that should be abstracted with intuitive global configuration, IMO.
Also Ukraine is very advanced in terms of dev and IT, and developers and entrepreneurs still need to maximise their productivity, so I’m not sure your idea that the technology stacks of Ukrainian startups should look different because their time is less valuable is valid.
Not everyone has constant access to a computer they own, a reliable internet connection, or the luxury of a "cloud" to use for research and development.
Good for Ukraine, I guarantee this is useful to at least a handful of folks in that country. Those who do not have access to the luxuries those of us in the developed world take for granted.
TLDR: I personally think that free paas is better than selfhosted in multiple ways, but sometime its great to have some alternaty and I hope that my pack give it for someone :)
After actually making your product these are probably the two most important factors for most SaaS startups. If you screw up availability you can't get customers - your site will fall over every time you push any marketing. If you screw up security you can't keep customers - you'll fail any kind of due diligence or you'll burn your reputation. Using third party services that do the hard parts for you is very, very sensible.
There's nothing wrong with a boilerplate starter kit if you understand the hard parts and just want to go faster. There's nothing wrong with using starter kit for a prototype. But if you actually want to build your app, and you think you need this, you should lean towards using something that manages this sort of thing for you instead.
Many people seem to forget that there are no free lunches... anyone who has ever wrestled with the learning curve of a necessarily proprietary no-code solution knows that you don't usually save time or money by outsourcing your platform, and that by outsourcing, you can easily become indentured to a restrictive black-box environment you don't control.
If you're not dealing with critical security updates to your Mattermost install, you will be dealing with something equally time-consuming and fundamental, like managing CloudFormation templates or trying to get divs to nest correctly in a WebFlow one-pager.
If your startup doesn't have the in-house knowledge/intelligence to be able to run a stack like the OP's, you might ask yourself how you plan to create a successful software company without any in-house software competency.
I don’t identify with a lot of commenters as my company does a lot of experimental work so I’d rather not pay the cloud tax, both in money and in lock-in.
I launched a product in the European space that has since fizzled but people loved the lack of cookie banners and the fact that their data was in a silo. I think your stack helps in this regard.
Awesome!
In practice, there are differences in preferences and jurisdiction that require such a solution to be quite flexible (e.g. auto-registering a domain, doing accounting, filing taxes in different countries, running payroll), but some things everyone needs (landing page, website with intranet access to secure shared collab: GitLab, CryptPad, Email etc). It is a bad idea to self-host something that isn't mature; but if there was a relatively bullet proof solution, many startups could save cost and increase their run rate this way.
If it's a learning project, or a part-time or hobby project, or if the people starting don't have a lot of money, then using a lot of self-hosted stuff will be an attractive alternative. There are lots of cases where with an ample amount of money or cash flow I'd agree with paying for SaaS or whatnot, but I don't think it's a good universal answer.
Even more important thing, than chose your docker orchestrator, - this is chose of distributed file system that you need for running stateful apps: I strongly recommend you buy efs or ceph from cloud provider, otherwise you should test next products yourself in your setup: https://vitobotta.com/2019/08/06/kubernetes-storage-openebs-...
Why some think that today tech is good tech and "self-hosting" it is a way to be free? Web tech have some reasonably good foundations but that's all, the rest is a colorful, animated bloatload mess. Modern web is needed not by us Citizens but by IT giants to rule the tech and indirectly the society, since information and communication are core elements of a society to exists.
Do we need Webmails? NO. We do need mails though, locally indexed with GMail-like powerful search (and we have them, see notmuch/mu maildir indexers based on solr), we need a modern MUA, that's for sure but why tie it to a modern WebVM a monster so big that only few giants can keep one up and no one can really know it even if FLOSS just for mere codebase size? We need mails on many devices, but that does not demand a central point.
We need notes, documents etc but definitively not a copy of Google Drive suite.
What we need are classic desktops a bit updated for modern users. Those really classic ones like Xerox Office System or more modern LispM and Plan 9 are far more advanced in tech terms than actual OSes and web-stuff. We miss the hw layer and they need a bit of changes for modern world but we can study them and we can run in that direction, a so powerful direction that no IT giants can compete.
Did you remember Popcorn Time? Why we need Youtube or Peertube when we can have a distributed app that's far superior by nature? Do you remember usenet? Yes, it base is old and crappy but it's a decentralized free place we desperately need and even if their base are crappy even today it's popular again for mostly illegal stuff (see Sonarr, Radarr, Lidarr etc) to a point that for some is superior to bittorrent. I can keep going for long. It's sure that creating for instance Retroshare or Jami seems not much useful because until enough users adopt it there is essentially nothing to share, no one to talk to etc, but that's the way in all cases.
We can't build a self-hosted better-them-{Zoom,Meet,Teams}, we can build a far superior local app connected to a decentralized network though, and while giants can easily fight back or buy with big money something web-based they can't really do nothing with such application.
It's not easy: modern GUI libraries are almost abandonware, some are active but definitively derailed, desktops is in a deeply sorry state etc BUT we still have some good environments with a good language portable enough, there it common lisp, go, python, ... with their ecosystem. They all have issues BUT issues that can be compensated by a new FLOSS community to a point of re-creating the old classic: the desktop have an OS, a single application, fully integrated, with a relevant programming language where anything is at users hands. For those who never used classic system or modern Emacs it's really hard to understand why that's extremely superior but just try to invest seriously in Emacs and you'll discover why that "alien and strange" way it's the way to go and the modern way is actually a crappy absurd idea.
Long story short: we can't compete with giants creating personal web ecosystems, we can compete on desktops, and that's why all giants do their best, and fail to completely succeed, pushing desktops to the oblivion. No one can win a stronger, resourceful enemy at it's own game, to win a different strategy, not a copycat, is mandatory.
We're running or app on linode VPSs.