Hacker News Reader

This looks great. Definitely a few idioms I will have to explore further.

I can use Docker in a basic sense, but it is amazing to me how much black arts still exists for what has become a cornerstone of modern deployment. Lots of conflicting/dated advice about best practices. Unsure which advice is still required/applies to podman, etc.

> but it is amazing to me how much black arts still exists for what has become a cornerstone of modern deployment.

What?! Unless you are using complicated networking, docker is simple. Dockerfiles are essentially just annoying-syntax shell scripts.

We have plenty of black arts in computing. Docker isn't one of them.

So, am I still supposed to specify UID and GID? Should I be using Alpine or Debian? How do I handle loading certificates? Should I use an override config for development? In this thread, someone was indicating that the naive volume option does not work over SSH. Also in this thread was a request to pin by tag and not hash. Do I still need to worry about Docker blowing a hole in iptables?

Maybe not black magic, but there are a lot of subtle optimizations for which there is much conflicting guidance.

This is how people end up using Kubernetes, despite its warts.

[edit: added second clause]

Kubernetes doesn't answer any of the parents questions though, does it? You still need a Dockerfile for k8s and all those questions still apply.

> So, am I still supposed to specify UID and GID?

If you run your container as a non-root user and create a user without setting a UID / GID it'll default to 1000:1000, so unless your Docker host's user isn't 1000:1000 then things work out of the box. A lot of this is general Linux knowledge around user / file permissions, not so much Docker.

> Should I be using Alpine or Debian?

Debian, no contest in my opinion.

> How do I handle loading certificates?

You can volume mount them or deal with SSL certificates in a way where Docker isn't involved such as running nginx on your Docker host directly or putting a load balancer in front of your app and handling SSL there.

> Should I use an override config for development?

You can use the same docker-compose.yml file in all environments and tweak things with environment variables. The compose file supports variable interpolation. Docker Compose profiles also let you control which services to run in each environment, it's even configurable by a single env variable.

> Do I still need to worry about Docker blowing a hole in iptables?

If you use -p 8000:8000, yes this will publish the port in a way where the outside world can access it. Likewise without Docker if you edit iptables to allow that port it will too. I wouldn't classify this as blowing a hole in iptables. This is "user configured application to make a port open to the world".

> Maybe not black magic, but there are a lot of subtle optimizations for which there is much conflicting guidance.

In the end my example apps address most of these issues for you. You're on your own with certificates since that varies on your deployment, but everything else is fully set up and ready to go and it protects yourself from blowing a hole in iptables since it only publishes the port to localhost by default, not 0.0.0.0. This would let nginx or another web server access it directly on your Docker host but no one else.

'black art' maybe not, overcomplicated mess? Yes!

I've tried to use the docker compose command to limit the maximum number of CPUs used, I failed, even though it's possible: the support team managed to do it, but why did I fail? Because the doc and the design aren't great..

These are excellent, thank you.

I maintain similar Django and Flask + compose stacks on behalf of the startup studio I work for so it’s fun to compare notes.

For our Django stack, for instance, we have also settled on Postgres, on celery+redis, and on whitenoise. black/flake8/isort also seem universally agreeable. We also throw in pyright and generally make extensive use of type hints.

For the front-end, we’re currently Create React App with TypeScript and Tailwind. I’d love for us to move away from CRA, so your use of esbuild is helpful to see. (I’d personally be happy using HTMX or Turbo/Stimulus but for the moment a JSON API backend with a React SPA front-end seems more comfortable for more of the CTOs who hop on board.)

We also supply some minimally opinionated glue at the API layer. On the back end we have a base View that provides a few helper methods for transmuting invalid Form instances to nice JSON replies that the TypeScript API invocation code works with gracefully. (We used to push DRF but have lots of feedback from older startups than ran with it and had regrets down the road.)

no offense, but man, the amount of tooling you guys are using sounds to me insane. How is a person able to oversee and understand everything. The older i get the more i feel distantiated and disconnected to these modern practices. I am afraid that if I ever have to find a new workspace i wont be able to succeed because of this.

I definitely relate to this but frequently challenge myself and then overcome it. You just have to bite the bullet and carve out some time to try them out.

The key thing to understand is that all these things are shrouded in incomprehensible jargon and alien sounding names that make it incredibly intimidating to get started. But the fact is, it is a wide but shallow pool of jargon sitting on top of the same old computing fundamentals that have been around since the 1970s.

You will find if you know your fundamentals, then the jargon is far less difficult to overcome than it seems. You just need a bit of exposure and if you spend any amount of time playing with the tech it just starts to happen by osmosis. (If there are fundamentals you aren't solid on, treat it as an opportunity to bed that in - even these things are not generally super complex in the end).

I'm so-called full stack developer so I have to be on top of it and it's hard. I'm constantly behind so I spend quite a lot of time catching up whenever I'm getting chance. It's possible (e.g. I think that I can one-handed build an application with react frontend and java/golang/node backend, build a k8s cluster and deploy everything, following best practices), but it's lot of things that you'll forget eventually.

I think that one sane way is specialization. Become an expert in writing nginx-ingress yamls in some big corporation. Another sane way is to throw away modern tech and stay with old tech. You can deploy perl cgi to OpenBSD just like you could 20 years away. Another way is to carefully select technologies which were proven for 7+ years. I'm trying to follow this way.

No offense but that list of tools is miniscule and you should pause if you think it's anywhere near "insane". For comparison a regular handyman with a basic set of tools easily has 50+ if they just started. It's definitely possible to understand all of that and way more. Unless you're fresh out of uni and never worked on a professional company with more than a couple of people you'd see many more.

I'm not GP, but I used to write firmware. I had a C compiler, a debugger, and emacs.

IIUC, if you write a web application in the MS world, you have Visual Studio, ASP.Net, and IIS.

There doesn't have to be piles of libraries.

No offense taken. As a fellow old person (whose first computer as a young kid was a TI-99/4A) I definitely feel this pain.

But because I am also an old engineer who works with a lot of other old engineers we are lucky to have a shocking number of human-years’ accumulated experience not only selecting a pile of tools and frameworks but also sticking with them from zero to $BIGCO. Put another way, I’ve shot myself in the foot more times than I can count with these tools and I do it less often and in more esoteric ways these days.

I can’t claim the same degree of experience with all these tools, of course. Pyright is bleeding edge. Tailwind is still the new kid. Relative to (say) Python and Django or Flask, React is new too. Most of the pain and learnings come from these newer moving parts.

Is it “better”? That depends on the axes of evaluation. As a startup studio where the 70% case might be “SaaS that takes a back-office process held together today by Excel & email and makes it way better” the answer is: sometimes, unequivocally yes. Not just because of the tools and the potential velocity they can confer, but because of the kinds of teams we can build around them. Sometimes, plain old Django with nothing added is a clearly better choice.

In the end, every startup is its own snowflake. We try to select “starter stacks” that balance industry familiarity with our ability to offer meaningful operational perspective. We definitely don’t think of them as the final world.

I think it was a subtle trend of making everything in tiny blocks while javascript / html5 evolving a lot, it created a jungle. But deno/bun/esbuild minded tools are more integrated, faster and leaner it seems (hot reload, typechecking, high perf all in one)

Offtopic, but I've googled Tailwind because of this comment. This seems absolutely crazy to me, in the worst sense imaginable. So, there was (and still is) "style" attribute in HTML. It leads to lots of repetition and yadda yadda, so people started using classes instead to write their CSS. There are dozens of schools of "best practices", or meta-frameworks, or actual frameworks to make it all more manageable, but now typically each element still has like 5 different classes specifically to define styles. So, finally someone got tired of it and came up with an ingenious solution: JS framework that dynamically defines CSS-classes with CSS-property-like names to write styles right in the HTML again, but to use class attribute instead of style attribute?! What's even the point?!

Hah! This was pretty much my response when I encountered Tailwind for the first time.

This post by Tailwind’s author gives some more perspective: https://adamwathan.me/css-utility-classes-and-separation-of-...

I had the same concern, this article perfectly captures the problem and gives perspective to the solution Tailwind provides - Thanks for sharing!

Classes are not perfect, they introduce relative meaning (or meaninglesness) and in prototyping its often a huge waste of time.

Other than that I agree.

Taking a quick look and nothing I saw manages ssl - do you even package that or do you have that as entirely different steps?

Not the same guy, but many cloud providers take care of ssl in their load balancers.

It's not included with these projects, as someone else mentioned in a reply to you, it can be handled in a number of different spots.

For example, a load balancer running 1 level above your server or perhaps nginx running directly on your Docker host without Docker.

A while back I wrote up why I prefer running nginx outside of Docker at: https://nickjanetakis.com/blog/why-i-prefer-running-nginx-on...

Your paragraph on SSL is basically what I have come to understand so thank you, I just thought I must be stupid or something.

Awesome repos, starred. Do you have any preferences for next steps, such as deploy/management? And if you’ve seen it, what do you you think of https://github.com/mrsked/mrsk?

Thanks.

Yep I've seen mrsk and even briefly chatted with DHH about it a day or 2 after he open sourced it.

I have mixed feelings:

On one hand I think it's fantastic Rails is starting to take on deployment officially. This is going to lead things to a better place in the long run. Generally speaking DHH has a really good track record for making things that feel good to use.

On the other hand, I think the project is trying to reinvent too many things that already exist and doesn't account for Docker Compose. For example, for literally the last 7 years I've been deploying any Dockerized web application with Docker Compose to 1 or more servers with about 15-20 lines of YAML using Ansible to set up the server and git to deploy the code. It doesn't matter if it's Rails, Flask, Django, Phoenix, Node, Go or whatever. It's all the same. Those 20 lines also include everything from taking a blank slate Debian / Ubuntu box to production ready, complete with self managed system updates, locking down SSH, iptables, various server configurations / optimizations, nginx, HTTPS, database backups, sane logging and everything else you'd expect.

I don't have anything ready yet but I've been slowly working on https://nickjanetakis.com/courses/deploy-to-production to assemble all of this into a course.

Thanks! I will use this.

Does this start rails app and postgre db on the same server?

By default yes but you can change a single environment variable to not run Postgres or Redis through Docker Compose. You can also very easily choose to run Puma and Sidekiq on different servers. It leverages Docker Compose v2 profiles.

I'd suggest watching and reading: https://nickjanetakis.com/blog/a-guide-for-running-rails-in-...

It's a full end to end walk through of the example Rails app and how it all works when it comes to Docker and Docker Compose. I just recorded it a month ago, it covers everything with up to date info vs the code in the repo.

Any golang?

Nope, I haven't run any Go code in production that was running in Docker.

Go's ecosystem is also pretty fragmented so it would be difficult to create a solution that lots of folk would be happy with. I get the impression most folks who use Go want to pick everything themselves (not a bad thing, just something I noticed). Plus I'm only 1 person doing this in my free time with no income sources attached to these projects, I don't have the capacity.

With that said, most of the apps are very similar when it comes to the Docker bits. You could take any of the examples and replace XXX with Go. I'd suggest basing it off the Phoenix example because that one covers using a 2nd Docker build stage to create a release. With Go being able to compile a self-contained binary that pattern of using a build stage to copy that over would be similar.