Hacker News Reader

> - Easy and powerful: the cloud. AWS / Azure / DO will manage things for you, providing redundancy, scaling, and very simple setup. You may even have some say in tuning specific components (that is, buying a more expensive tier for them). Beautiful, but it will cost you. If the cost is less than the value you get, wonderful. Be aware that you'll store your data on someone else's computers though.

Easy on the magic powder. Cloud vendors manage some stuff, they mostly abstract you the hardware and package management part, but that's about it. Hosting a postgresql DB on RDS instead of a VM somewhere or on bare metal doesn't change much. Sure redundancy and scaling is easyto setup. But you still have to stay up to date with the best practices, how to secure it through network, choose and set your backup policy, schedule when to upgrade, plan the potential downtimes, what is deprecated, what is new, when price will sky rocket up because AWS doesn't want many customers to still run that old version. Same applies to many individual tech sold as "managed" by cloud vendors.

A whole lot of admin overhead is not removed by running some individual tech as a managed service.

You only remove a significant part of the management when the whole stack of what comprises your user facing application is managed as a single entity but that comes with another problem. Those managed apps end up being very highly coveted targets by black hat hackers. All the customers over the world usually end up being pwnable at the same time thanks to the same bug/security hole being shared. It becomes a question of when, not if, your users accounts will become public data.

> Cheap, easy, powerful: choose any two.

I don't think there's any reason the same codebase can't support different trade-offs here.

Maybe I'm just looking at the past through rose-coloured glasses, but it seems to me that was the norm before we standardized on distributing apps as an entire stack through a docker-compose.yml file.

Your app depends on redis for caching. If no redis instance is configured, go ahead and just instantiate a "null" caching implementation (and print a warning to the log if it makes you feel better) and carry on.

You're using minio for object storage. Is there any reason you couldn't, I don't know, use a local folder on disk instead?

Think of it as "progressive enhancement" for the ops stack. Let your app run simply in a small single node deploy, but support scaling up to something larger.

IMHO, redis is either used as a key value store (which can easily be replicated in application code) or as a central storage to synchronize tasks (like counters).

For the first case, dev should just build on SQLite or use application code. For the latter case, choose a single storage engine and use it for everything (Postgres?).

SQLite is actually substantially more capable than many people think it is. I have served 20k MAUs from a reasonably sized single node server with some headroom to spare. Yes, it requires some thinking about efficiency and not necessarily going with nodejs + some ORM, but you can take SQLite quite far, even in a small to medium enterprise

SQLite works well with 2k DAUs on a single node, even with Django's not particularly efficient ORM. You just have to be careful about what you really need to write to DB and what's okay to either not save at all, or just throw into a log file for later offline analysis.

I don't see how these guys can think about MAU/DAU to assess DB load and sizing without talking about the rest of the app/arch details. Wouldn't ops/time be more agnostic?

Agreed, number of active users cannot make sense as a generic unit across systems...

I have 2 systems where in the first (route optimization platform), 1 user would, as part of just a normal 10 minute session:

- read ~100MB from the database - utilize 100% of 32-core machine CPU (and 64GB of RAM) - resulting in thousands of writes to the database - and side-effect processing (analytics, webhooks etc)

Over a course of a day, it would likely be ~10x for that single user.

In the other system - appointment scheduling - 1 user would, in 1 day, read ~1MB of data from the database, and make 2-3 writes from a single action, with maybe 1 email triggered.

Can make sense if it's about loading and showing not-too-complicated data, but no intensive computations. Then you can compare roughly similar applications

Maybe they mean cases somewhat similar to the software this discussion is about.

I.e. load and show stuff from databases (but nothing compute intensive).

SQLite has excellent read performance.

Insert / update performance is quite another story: https://stackoverflow.com/a/48480012 Even in WAL mode, one should remember to use BEGIN CONCURRENT and be ready to handle rollbacks and retries.

You can scale quite far using SQLite. That's what Basecamp is doing with their new self-hosted chat app, named ONCE Campfire. It is designed to scale to hundreds or even thousands of concurrent users with the right hardware: https://once.com/campfire.

I wonder why it needs 2Gb of RAM even for a low number of users though.

Ruby on Rails is not known for being very RAM efficient, but this is only me speculating.

It ships as a Docker container. Docker recommends a minimum of 2GB RAM to run the Linux version of the Docker Engine, before adding constraints imposed by running apps.

awesome, thank you for the information.

It still astounds me how hard software distribution is. We've had so many generations of solutions and paradigms that attempt to solve this problem (e.g. C, Unix, Java, Docker, etc) but the best we've come up with is client-server applications and the web. Yet it's still not trivial to host a reasonably "powerful" web application.

In theory, the components for a universal runtime are pretty apparent by now. Wouldn't it be wonderful if truly portable and end-user friendly cloud computing was a thing.

I suspect that a Docker container would in this case solve the issue.

The WebAssembly Component Model is attempting to do this though it seems to be progressing rather slowly.

How big the organization has to be not to be able to deploy it on a big enough machine with SQLite?

How big is not the question. The question is how small: think self-hosting within a small non-profit or other such operation, on someone's aging ProLiant in a basement.

A big organization likely would just buy their hosted solution.

Size likely has nothing to do with it, while reliability is a much bigger concern.

If your business uses said tool for all client work, and it's hosted on a single server (be it physical or a VM), that is a SPoF. Any downtime and your business grinds to a halt because none of your project or customer information is accessible to anyone.

Does everything need 99.99% availability?

It feels like you have to be a very big business for this to be a problem that is worth the extra engineering effort vs a single instance with an SQLite file that you backup externally with a cron job

Not availability but predictability.

Most stick trading systems and even many banking systems have availability like 40%, going offline outside business hours. But during these business hours they are 99.999999 available.

Usually operation without trouble is operation with plenty of resources to spare, and in absence if critical bugs.

> - Cheap and powerful: All open-source, built from well-known parts, requires several containers to run, e.g. databases, queues, web servers / proxies, build tools, etc. You get all the power, can scale an tweak to your heart's content while self-hosting. If you're not afraid to tackle all this, wonderful, but be aware of the breadth of the technical chops you'll need.

What about lowering the number of dependencies your application uses, like only depending on a database? Running a database isn't that hard, and it also greatly simplifies the overhead of running 5 different services.

This works up to a point. Past a certain scale, running a database becomes hard(er), and you also start to need proxies, caches, load balancers, etc. to maintain performance.

I would agree, though, that many software installations never need to scale to that point. Perhaps most.

At what point do you hit that scale with project management software, though? Maybe you can't get to the point where you're managing all projects across all of Walmart from the same instance, but certainly you can run pretty much anything of reasonable size.

sqlite can scale to thousands of concurrent users.

Personally for me the issue with all these new project management platforms is that the target demographic is either: - they're so small they can't afford to self-host - they're big enough they can afford the pro plans of more established tools

Small companies can get by perfectly with the free/cheap plans of other tools, like Trello+Google Workspace. Heck if you're a one-man team with occasional collaborators free Trello + Google Workspace (~$6/mo) is enough.

A box to provision the Huly stack might come out at more per month...

> But you're severely limited in the number of concurrent users

You can easily handle 100-200 concurrent active users on a decent CPU with SQLite, if you don't do anything crazy. And if you need a project management solution that needs more than that, you probably are not too concerned about the price.

I think the point they're making is that it's not exactly cheap either, given the amount of upfront knowledge (and time investment to gain that knowledge) required, or the cost of vetting and paying people who do have that knowledge.

So "cheap and powerful" just looks like "powerful", at which point you may as well make it easy, too, and go with a managed or hybrid solution.

It's "cheap" as in "not paying $39/mo per user", or whatever other subscription / licensing fees.

Why is open-source "cheap"?

It seems like you look at the cost of things only through the lens of licensing and not the cost of people to run/maintain them.

I have nothing against OSS per se, but in my experience, the financial analysis of OSS vs paid software is much more subtle.

the cost of the people is represented by it not being "easy"