Hacker News Reader

Comment by MuffinFlavored | Hacker News Reader

> A sizable fraction will be concerned about 'container overhead' (and "scalability issues") when asked to move workloads to containers. They are usually not able to explain what the overhead would be, and what could potentially be causing it.

For what it's worth, one of the biggest "containerization" recommendations is to not run your database (example: Postgres) in a container, correct? Due to I/O performance decrease?

View on HN

No. Docker volumes aka bind mounts have little or no overhead. You don't want to run a database in an ephemeral "cattle" container without some kind of HA because you'd lose data.

    docker run --rm \
      --name postgresql \
      -e POSTGRES_PASSWORD=postgres \
      -d \
      -p 5432:5432 \
      --cpuset-cpus="0-1" \
      --cpus 2.0 \
      -m=1024m \
      --mount type=bind,source=$HOME/docker/volumes/postgres,target=/var/lib/postgresql/data \
      postgres:12.2

native results on a digitalocean VM:

    $ pgbench -c 100 -j 2 -T 60 postgres
    starting vacuum...end.
    transaction type: <builtin: TPC-B (sort of)>
    scaling factor: 1
    query mode: simple
    number of clients: 100
    number of threads: 2
    duration: 60 s
    number of transactions actually processed: 17660
    latency average = 342.761 ms
    tps = 291.748482 (including connections establishing)
    tps = 291.791293 (excluding connections establishing)

in a docker container:

    starting vacuum...end.
    transaction type: <builtin: TPC-B (sort of)>
    scaling factor: 1
    query mode: simple
    number of clients: 100
    number of threads: 2
    duration: 60 s
    number of transactions actually processed: 13014
    latency average = 466.928 ms
    tps = 214.165822 (including connections establishing)
    tps = 214.199201 (excluding connections establishing)

214tps in docker, 291tps outside of docker

26% decrease, with a bind mount on ubuntu 18.04

Docker introduces all kinds of performance issues. I for example noticed higher latency even with just pgbouncer. It was very visible when running integration tests, which normally took 15 minutes, but as a container it was 45min - 1h.

Funny, I looked into papers or articles about performance issues with containerized RDBS _binaries_ and didn't find anything relevant. Of course you want the data mounted outside the container so it's not ephemeral.

I ran some casual tests using and found out there is a performance hit in using db binaries inside a Docker container due to Docker networking (different for different types of networking).

I would be more concerned about writes going through additional fs layers and about abrupt termination of a container.

You generally are trusting a database to keep your data safe, so those things will contribute to data loss.

Remember the freakout about PostgreSQL not handling sync() correctly on Linux due to ambiguity in the man page? Having a networked filesystem + additional abstractions (like layers) etc only reduces data durability.

Except you never do that (having your DB directory be in the container rootfs). Nobody does, because then (beside the performance/reliability impact you mention) if the container goes (docker rm) the data goes. You're always gonna use a Volume for this kind of cases, be it K8s volume or Docker volume, and these, as neighbour message mentions, are just bind mounts (or actual mounts in certain cases), so no layers, no overlay, nothing of the sort.

check out my response here: https://news.ycombinator.com/item?id=22809524

looks like docker with a bind mount has a 26% decrease in performance versus native

I think it is more because containers should be stateless and you cannot make a database stateless.

We do run databases that do 100k`s ops/s in containers but we don't run them in kubernetes. We just mount the VM hard drive in it.

A DB container may and should be stateless, but when configured correctly the volumes specific to the storage engine are persistent. I've been running production databases in Docker since 2014 without any data loss, it makes a lot of system-level administrative work much easier.

With a healthy understanding of how the individual storage engines commit to disk, upgrading, backing up, etc. can be done in parallel and without impact to a running production system thanks to the power of overlayfs.

Reply on news.ycombinator.com