Hacker News Reader

Comment by michaeldwan | Hacker News Reader

I really want to like Alpine, but we (Fly.io) have seen so many DNS issues with customer images that we’re now recommending Ubuntu or Debian slim. The extra ~50mb is a worthwhile trade off to avoid hard to debug musl-libc issues.

eg https://www.linkedin.com/pulse/musl-libc-alpines-greatest-we...

View on HN

The person responsible for musl's DNS support thinks broken DNS is good UX and purposefully didn't add tcp upgrade support to it.

Note that some DNS resolvers do not provide truncated UDP results apparently so that might explain some of the weird DNS issues people see https://twitter.com/RichFelker/status/994629795551031296

Yep. I had an (immutable) JVM ECS service that worked fine for a year, then started failing DNS in just one of several AWS AZs and for just one of various host names--one with something like 24 A records, a few but not an outrageous number. Occasionally forking a process to run 'dig' on the same name made it work for a little while.

AWS support's only advice was "don't use Alpine"; annoyingly, switching the containers to a Debian base cured it, even though this would appear to make absolutely no sense with respect to it failing in just one AZ.

Failing in one AZ probably means they changed something with DNS servers for that region. We had a similar issue recently when we rolled out a new DNS server that returned longer SOA records which broke python's mysql driver in only some regions during the rollout. Debugging nightmare fuel.

Yes, my theory was also some DNS variance in the AZ, but AWS stolidly refused to supply any information to that effect. Adding 'dig +trace' to the equation only deepened the mystery.

Interesting!

Thank you for sharing. Did you notice that some language are more affected than others?

For example Go vs. Node.js?

Node, Ruby, and Python come to mind. Oddly enough the alpine versions of haproxy and nginx on Docker hub had issues too.

Any insight on Laravel/PHP? I've been seeing DNS resolution issues with Alpine as well, but they seem to mainly affect PHP services and not Go.

Also, isn't Python problematic with musl-libc [0]?

[0]: https://pythonspeed.com/articles/alpine-docker-python/

What about busybox? It still has a glibc version doesnt it?

Reply on news.ycombinator.com