Ask HN: Best way to simultaneously run multiple projects locally? | Hacker News Reader

Ask HN: Best way to simultaneously run multiple projects locally?

37 pointssuralind59 comments

Hey HN, I often work on couple of projects at the same time and am trying to figure out what’s the easiest way to run all of them at once.

For example, let’s say that I want to run frontend and two backend services (each it’s own repo).

My current approach is to run each on a different port, but I actually wonder whether setting up a small Kubernetes cluster with Traefik as proxy wouldn’t serve me better as I could then use Skaffold or something similar to deploy apps independently to the cluster.

Basically looking for true and tested solutions. Thanks.

View on HN

Woah, so much overcomplication in the comments!

If you want to run multiple applications, how about ... just running them? It sounds like you already do that, so what is the real problem that you are trying to solve?

If it's annoying to start them by hand one by one, you could use Foreman or Overmind to start them with a single command, and interleave their output in a terminal.

I also dont know why OP wants to jump into kubernetes directly instead of just using docker...

Even Docker ... does that solve a problem OP has?

Yes, just write a small docker compose file and you can start all of the services with 1 command. Either isolated or connected, however OP wants to use them.

> jump into kubernetes directly instead of just using docker

Option A: learn k8s, use it for anything containerized. Don't use the complicated parts if you don't need them.

Option B: learn docker (docker compose I guess?), use that... then also learn k8s because you'll eventually have to anyway

I'd argue most companies would never have to touch k8s but Docker is a valuable skill to have.

I’d agree here. I thought kubernetes was going to dominate the container compute space, but ECS and similar non-k8s solution took up way more steam than I expected. I wouldn’t be surprised if k8s still dominates market share, but you can go really far with other orchestration services.

ECS, while vendor locked and less extensible, is actually pretty solid to work with.

It's not that I don't want to run them locally. It's more like I would like to

a) use HTTPS when developing services b) give each service a unique hostname c) run all my services on the same port

and there's simply no easy way to do that that I know of. Hence my Ask HN :)

I hadn't come across overmind before and searching turned up several very different projects.

I think the parent comment is referring to this one: https://github.com/DarthSim/overmind

Yes, that’s the one I meant. One thing that annoyed me with Foreman is that sometimes it doesn’t terminate all processes when one of them crashes. Since I switched to Overmind/Hivemind I never had that problem again.

What a bunch of over engineered solutions.

I run multiple side projects on my linux desktop using both postgresql and mysql, and using host entries work well enough.

For HTTP, using entries in hosts file locally allows the clients to all connect to port 80 on the local machine all using different domain names, and nginx can proxy the connection to whatever port the backend is running on.

Everybody loves their random port numbers these days, but I still prefer custom hostnames.

Just chuck an entry into your hosts file and tell your web server to sniff for it and you’re done. Run your stuff on port 80 like nature intended and never have to teach either end of your app how to deal with colons and numbers in domain names.

And you get to skip the six paragraphs of pain the other commenters are describing, orchestrating their kubernetes and whatnot.

e.g.: http://dev.whatever/

I like to combine both locally — bind to port 0 to get an arbitrary unprivileged port, and use domains in /etc/hosts for easier configuration ;-)

For those not familiar with what you are talking about, you can add things like the following to your /etc/hosts file:

  127.0.1.1  foo.local
  127.0.2.4  foo.bar.local
  ...

They will all resolve to local host, so make sure to bind your services to those IP addresses (eg. `nc -l 127.0.2.4 80` will bind only to 127.0.2.4 on port 80).

But running on unprivileged ports like 80 means you've got to run the server as root.

At least on macOS you don't even need to add to /etc/hosts, just use .localhost as the TLD.

How does that work with binding the same port to it? Does it automatically assign a new IP to every new domain resolv() call gets (and then caches it)?

This is the right answer. But use the .test domain as this is meant for the usecase.

Different ports.

Don't add unnecessary complexity unless it's strictly necessary.

Or, if you want to get a bit more fancy, use different IP addresses.

For IPv4 you have the entire 127.0-255.0-255.1-255 subnet available to you.

For IPv6 you can add additional addresses to your existing network interface.

I use the 127.xxx.xxx.xxx extensively for local development and add entries in /etc/hosts so I get dns resolution as well. It works great.

I wonder why there aren't super popular solutions for managing hosts entries in an automated way for development, since if you need to share entries with other people for a bunch of projects, then adding them manually is cumbersome and running a separate DNS server just for development is not less so, whereas using public DNS records set to local addresses seems a bit dirty.

In other words, where's:

  hostman apply Hostfile
  hostman clear Hostfile

https://github.com/2ndalpha/gasmask did this for Macs but I've found that I just don't need it. A static file is fine. I often have old entries in it that I don't use anymore but they don't harm anything being there so they stick around until I do a cleanup pass.

That’s a huge trap.

Yes, it’s simple at the beginning but it takes a lot of effort to move to non-port based solution for anything.

Cuts are small at the beginning (oh, this service should use other PostgreSQL, so lets have two - oh but I my code doesn’t specify port in an config file, so let me put on direnv - oops IDE didn’t pick up env file) but they grow quickly.

Containers are standard nowadays and allow going for Kubernetes if one wants. With solutions like Justfile or Taskfile its reasonably ergonomic.

Depends on which type of scale you are running. And it's not worth overengineering a project for a scale you hope to achieve some day. It will take away engineering resources from building something that provides value today

It’s a problem future me will always, happily solve.

If my system is large enough and complex enough, it _likely_ means the business behind it is successful. I will always solve “dumb problems from past me” for a successful business than the alternative of not having a business.

I agree to the principle in general, but non-reproduceble execution environment with free-for-all internet access is unimaginable time sink.

Business tends to topple on unreliable promises or annoying customers through the bad tech and I’ve seen a few that are gone because of it.

Unicorns won’t fall on such problems but semi-good Average Joe’s product might. No one is going to hear about it even if such products make bulk of software in global code volume.

Containers (on low technical level) are simple, performant and scale well.

Using Kubernetes can be good for your resume.

What I usually do is to use different ports on my workstation. So I can get the fastest iteration, by keeping things simple. Be careful to keep the ports straight, though.

You can put the port numbers and other installation-specific files in a `.env` file, application-specific config file, or a supplemental build system config file, that aren't checked into Git.

One way I did this was to have a very powerful `Makefile` that could not only build a complicated system and perform many administrative functions, of a tricky deployment, but also work in both development and production. That `Makefile` pulled in `Makefile-options`, which had all the installation-specific variables, including ports. Other development config files, including HTTPS certificates, were generated automatically based on that. Change `Makefile-options` and everything depending on any of those was rebuilt. When you ran `make` without a `Makefile-options` file, it generated one, with a template of the variables you could set.

Today I'd consider using Kubernetes for that last example, but we were crazy-productive without it, and production worked fine.

What is your preferred method to deploy a python api to a frontend?

can use ansible (or ssh) and copy a tarball (or python package or pull docker container)

Containers.

I’ll also recommend commercial OrbStack for Mac, because it simplifies some configuration and is performant.

That was my focus over the last couple months (and right now with customer solution I’m running tens of isolated clusters of heterogenous services and custom network configurations).

I’ve tried nix, local Caddy/Haproxy, Direnvs, Devenvs, dozens of task file runners, DAG runners etc.

Kubernetes is fine but it’s a domain in its own and you’ll end up troubleshooting plenty of things instead of doing work itself.

The simplest solution I would recommend is a task runner (Justfile/Taskfile) with containers (either built or with volumes attached - this prevents secrets leakage). Pay special attention to artifact leakage and clone volumes instead of mutating them.

I don’t recommend Docker Compose because it has low entry and a high ceiling and it takes long time to back out.

For simple clusters (5-8 containers) it’s working well. If you need to level up my personal recommendation would be:

- Go for pure programmatic experience (I’ve tested bunch of API clients and IMO it’s less time learning Go than troubleshooting/backfilling missing control flow features) - there’s also Magefile for simplified flows

- Full Kubernetes with templating language (avoid YAMLs like a plague)

- Cuelang if you want to go for full reliability (but it’s difficult to understand and documentation is one of the worst I ever read through).

https://ddev.com/ has become standard in the circles I run in (most are web devs working in agencies touching multiple projects each week). You don't have to use DDEV specifically, but it works like a dream and may provide some inspiration.

Each project gets its own Docker Compose file. These allow you to set up whatever project specific services and configuration you need.

None of your projects need to expose a port. Instead each project gets a unique name like `fooproject` and `barproject` and the container listening to port 80 is named {project-name}-web.

It all gets tied together by a single global NGINX/Traefik/Caddy container (you choose) that exposes port 80 and 443 and reverse proxies to each project's web container using Docker's internal hostnames. In pseudo-code:

  https://fooproject.example.site 
  {
    reverse_proxy fooproject- web:80
  }

  https://barproject.example.site 
  {
    reverse_proxy barproject-web:80
  }

The final piece of the puzzle is that the maintainer of DDEV set up a DNS A record for

  127.0.0.1 *.ddev.site

You could do something similar yourself using your own domain or locally with DNSMasq.

It may seem overcomplicated (and it is complicated). But since it's a de-facto standard and well-maintained, that maintenance burden is amortized over all the users and projects. (To the naysayers, consider: PopOS/Ubuntu are quite complicated, but they're far easier to use for most people than a simpler hand-rolled OS with only the essentials.)

I prefer setting up services that bind to port 0 ("get me an unprivileged port"), report that back, and use that to auto-configure dependent services.

This allows local development and debugging for fast iterations and quick feedback loop. But, it also allows for multiple branches of the same project to be tested locally at the same time!

Yeah, git does not make that seamless, so I'd have multiple shallow clones to allow me to review a branch without stopping work on my own branch(es).

Git worktrees make working with multiple branches a breeze

Nice, I wasn't aware of git worktrees at all!

Before there was Docker and Kubernetes I used to run hundreds of web sites on a single server by being disciplined about where files go, how database connections are configured, etc. I still do.

I like to run my code in containers (also during development), although your approach ain’t broken.

I would say docker-compose with traefik is definitely the easiest! You can even set dependencies between services to ensure that they load in the right order, do networking, etc.

If you're interested in running locally, a solution like kubernetes seems slightly overkill, but it can be fun to mess with for sure!

Kube slows down iteration cycle. I use bash script that encodes the port information. No big deal if you repeat this boilerplate code. LLM can apply a change to all of it simultaneously.

Simple `bin/restart`

K3s is good. Kube is also good. But for local development you want to isolate to code and have rapid cycle time on features. Use `mise` with simple run script. If deploying to k3s use Docker (with Orbstack if Mac) and simple run script.

LLMs bad at auto debugging environment means you are spending even more time on low leverage task. Avoid that at all costs. Small problem => small solution.

We use the following steps:

- Each service listens on a different, fixed port (as others have recommended).

- Have a single command (incrementally) build and then run each service, completely equivalent to running it from your IDE. In our case, `dotnet run` does this out of the box.

- The above step is much easier if services load their configuration from files, as opposed to environment variables. The main configuration files are in source control; they never contain secrets, instead they contain secret identifiers that are used to load secrets from a secret store. In our case, those are `appsettings.json` files and the secret store is Azure KeyVault.

- An additional optional configuration file for each application is outside source control, in a standard location that is the same on every development machine (such as /etc/companyname/). This lets us have "personal" configuration that applies regardless of whether the service is launched from the IDE or the command-line. In particular, when services need to communicate with each other, it lets us configure whether service A should use a localhost address for service B, or a testing cluster address, or a production cluster address.

- We have a simple GUI application that lists all services. For each service it has a "Run" button that launches it with the command-line script, and a checkbox that means "other local services should expect this one to be running on localhost". This makes it very simple to, say, check three boxes, run two of them from the GUI, and run the third service from the IDE (to have debugging enabled).

Docker-Compose is likely the simplest solution for your needs (assuming you have some familiarity with Docker).

Your docker-compose.yml file would look something like this:

```

---

services:

  frontend:
    image: localhost/frontend:latest
    ports:
      - 6767:6767
    environment:
       - ...
    volumes:
      - /home/user/dev/app/frontend:/app/frontend
    restart: unless-stopped

  backend-1:
    image: localhost/backend-1:latest
    ports:
      - 7878:7878
    environment:
       - ...
    volumes:
      - /home/user/dev/app/backend-1:/app/backend-1
    restart: unless-stopped

  backend-2:
    image: localhost/backend-2:latest
    ports:
      - 8989:8989
    environment:
       - ...
    volumes:
      - /home/user/dev/app/backend-2:/app/backend-2
    restart: unless-stopped

```

This example even includes volume mounts to your local files from you app development folder (update to suit your setup) so that any changes are immediately reflected in your running frontend or either backend containers when you save your files.

You would bring all of your services up with this command:

docker-compose up -d

And bring them down again with:

docker-compose down

If you want to go the k8s route, it’s worth checking out https://tilt.dev/.

I found that Caddy will take you a very long way if you just want virtual hosting and reverse proxying. YMMV

Docker is probably simplest. Start with one machine, with the services in it, on different ports to keep it portable. As you confirm the fit and finish, it can inform how to further architect it.

Docker Swarm can be a little heavy for small projects, there's a tool called spin that seems to be a relatively lightweight docker orchestrator that has been pretty handy.

https://serversideup.net/open-source/spin/

Kubernetes, etc, are nice too, just might be a little heavier and more complex than a simple set of tools needs in the beginning. I understand it's interpretation and preference, most of the time if I'm solving a problem, I want the most of it to be invisible as possible.

Do you actually need to just run them, or do you need to bring them up and down through automation?

If you just need to have them running, nginx and /etc/hosts are your friends. You can script those as well, put a magic comment or two in hosts and split on that and change the inbetween programmatically, add a sites-available that maps port to name in hosts.

If you need to spin up, tear down, spin up, tear down, then containers is a more convenient option. Name your containers to make them discoverable, don't get bogged down in kubectl or similar tooling.

If you’re happy enough to write a little bit of C# and install the .NET SDK then try Aspire.

Here’s and example of orchestrating a node.js app and a Redis dependency.

https://github.com/dotnet/aspire-samples/blob/main/samples/A...

I’m the developer of https://canine.sh — I’d humbly ask you to take a look! It’s basically built to solve exactly this problem, and it’s totally free + open source.

The backend uses kubernetes, and the frontend integrates with GitHub, dockerhub, etc, for doing builds and deploys.

I’ve been able to host 2 rails apps, 1 postgres and 1 redis in a $10/month Hetzner VPS.

Visual Studio gives each project a (static, included in what gets committed to version control somewhere) random port.

For stuff I'm playing with at home, I'll either pick a port or put it in k3s. Half the reason for the latter is an excuse to practice with a kubernetes since it seems like a good thing to learn, both in general and for some specific things at work that it would probably be a good match for.

My suggestion would be Process Compose[1]. Just write a simple config telling it what commands to run; it will launch them all and give you a nice TUI to read the stdout and control each process individually.

[1] https://github.com/F1bonacc1/process-compose

If you want to avoid containers/complexity, you could just have a shell script (or choose your pet language) that starts the apps via a single command on your machine (e.g. $> start-my-app).

That would handle the actual startup on your specified ports (it could even randomly discover open ports and use those if you don't care).

Github, nodejs, and PM2.

I can deploy to my local machine AND to EC2 with a single deploy command using PM2 configuration facilities. This works quite well for side projects.

I also have nginx and postges running on both locations with local (to the server) database connections without opening unnecessary ports to the world.

not answering the op's question, but an alternative approach to multi-service development. I have been fixated the idea of a single process that runs all my code until an optimization is desperately needed. imo adding N number of processes to development slows down iteration by N^2. instead of reliably having one place to check for an error it could be any of the services, and having all team members have the context on where classes of problems could be is challenging. there may be a compelling reason a multi-service architecture is needed (different languages, legacy code) but i personally heavily weight development iteration in my decision making. slowing down experiments to test hypotheses is the death of productivity and morale in a codebase.

I'm using Pulumi to generate a Docker-compose on-the-fly, so that I can reconfigure the services I want to have up, and whether to run them / discard them / mock them

eg "when I want to debug the app, I want to mock the authentication server, and run the API" etc

Docker compose, possibly with VS Code dev containers if you're a VS Code user.

Makefile at the root of your code directory with target per project for building and running and a command that runs them all.

Using Kubernetes for your use case is like buying a Bugatti as your first car.

I constantly run multiple projects without issues. Usually it’s both front- and back-end. I just assign random ports.

Same goes for databases, I just map it to a different port.

pm2 could work https://pm2.keymetrics.io/

To throw yet another option in, you could consider an LXC container per project, if they’re small and you don’t find you need Docker. LXC containers are basically multiprocess containers, unlike Docker’s single process containers, making them feel more like VMs and giving you a great dev experience.

docker compose, technically other clients support the former also.

Buy mini computers and learn to manager a cluster

Vagrant

Reply on news.ycombinator.com