Dora: A open-source Rust DHCP Server | Hacker News Reader

Dora: A open-source Rust DHCP Server

github.com

126 pointsckdarby36 comments

I have a datacenter with so many machines (blade computers) in it that dnsmasq couldn't keep up and we ended up having really bad UDP storms. That wasn't fun to debug at all since we had no idea that dnsmasq would have scaling issues like that. Even worse is that the logging output is abysmal and made things even harder to diagnose.

Switched to another DHCP server and the storms went away, but I am not entirely happy with it because of the way it handles leases, it doesn't guarantee that each MAC keeps the same IP over time.

Now, I'm always on the lookout for a good DHCP server... tried another one that was golang, but it wasn't working on my network for some weird spec reason, I didn't have time to debug it and the authors didn't seem that interested in helping out.

We've done some basic benchmarking at +5k leases/second.

If you want to go down the rabbit hole with us of looking at using Dora and giving us feedback I'd be glad to chat further cdarby[at]bluecatnetworks[dot]com

Thanks. I've saved your project and if we are ever in the mood to switch, we will definitely give you a try first. Keep up the good work, you're solving a real problem.

If you don’t mind sharing/are able to share, what are your lease times?

We don't expire.

It would be great to have a better dnsmasq with sane config file, better logging, a simple statistics page and some more.

Did you try Kea[0]? There are a variety of ways to ensure the same MAC addresses get the same IPs even after the initial lease expires.

I'm not surprised you had issues with dnsmasq - I wouldn't use it for anything other than a run of the mill /24 (or maybe a few of them on VLANs). Don't get me wrong - it's perfect for what it was originally intended for and I use it in at least half a dozen places. Just not for an application and network like yours.

[0] - https://www.isc.org/kea/

Kea is what we ended up with. It has a few feature differences from dnsmaq that we had to work around a bit. Not the end of the world, but not fun when you're under pressure.

Now that I've studied the source code of dnsmasq a bit more because of the issues we had, I have to agree with you. We had run it with about 3k blades and it was fine. It was fine at about 9k blades too... but once we hit around 10k, it just fell over.

The frustrating part was that it just didn't log what was up. We used netdata and discovered the UDP queues were filling up. Tried mucking with tuning the sysctls and that didn't work either.

Switched midstream to Kea, which was a giant pain because we had a hard time dealing with duplicate IP addresses (imagine rebooting 10k+ PXE booted blades during a UDP storm).

Frankly I'm AMAZED you made it to 9k leases with dnsmasq. Not only from a performance standpoint but the overall logging, lack of instrumentation, limits in configuration, etc. As you say (from reviewing the source) it's really not intended for anything other than small home/SMB networks, memory and CPU constrained devices, etc. It's perfect for that and it's my go-to in those scenarios. A 10k host network is a completely different animal and from the down-votes my comment received it doesn't seem like many here on HN have been in scenarios like these either.

Your comment about the logging is spot on as well. I think dnsmasq and I think "small flash device where the logs will never get stored anywhere and no one can or will view them anyway". As you can probably tell I've had issues getting meaningful logging and debug out of dnsmasq as well...

I don't envy your situation in the slightest!

Note to mods - I'm not complaining about downvotes. I'm using the downvotes as a sign there's little understanding on HN of network ops at scale. People may love dnsmasq on a home router (I do) but plenty of network admins like OP and I have watched it fall over in other scenarios.

Both myself and my super talented devops coworker hadn't ever run with that many individual machines in a single location. 4-5k was both our maxes. It worked fine before, we didn't expect it to fall over.

dnsmasq has some really nice features that kea doesn't have. just finding a single blade in the datacenter, means a naming system, and it was easier with dnsmasq, and the config was easier to create. We ended up having to use ansible to generate the kea config file because we also keep machines subnet separated based on physical location.

HN has all sorts of people, I never try to quantify it. We don't know what we don't know.

On IPv6 this would be a solved issue. With SLAAC the router just broadcasts Router Advertisements and all the machines just automatically configure their addresses, which are fully predictable based on their MAC address (if you have privacy extensions disabled). The router has no state to keep and as such the network can be scaled up "infinitely". Just pick a random /64 prefix and set up radvd (or systemd-networkd) to send the router advertisements.

Sure. The thing is that we also have a physical location based naming system which is somewhat tied to the IPv4 address and we use that system across multiple data centers. We also have a single daemon running on each box that helps with the naming. It would complicate things (ie: more code to write) to have one dc (where we had these issues), on IPv6. We hadn't had issues before, so it never really made sense to use IPv6 other than for the novelty of it. IPv4 works surprisingly well and the issue really wasn't at that layer. Who knows too.. maybe radvd or systemd-networkd also have other issues we don't know about.

Hey, I'm from BlueCat (though my opinions are my own). Will lurk the comments if there are any questions around the project or general DHCP.

Looks cool, what was the motivation for writing another DHCP server - just for learning Rust or something else?

FYI /u/leshow is the core contributor of this project.

We've been doing Rust before this. We've rewritten some of our core components over to Rust. Discussed a bit in this post: https://medium.techatscale.com/open-heart-surgery-from-java-...

In terms of writing a DHCP server there was research looking through Github, and the conclusion was that there is a gap in the DHCP space. DHCPd was approaching EOL (and now is EOL). Others in the DHCP industry were calling for options, and it looked like Dora could potentially fill some of those asks: https://ns1.com/blog/isc-dhcp-is-eol-what-will-replace-it

I dunno about "another". After some issues with dnsmasq I went looking and there aren't actually a lot of open source DHCP servers. There's like dhcpd, dnsmasq and kea.

(I actually contemplated writing my own dhcp server in Rust while debugging dnsmasq but felt like it's one of those real PITA kinds of projects. Managed to workaround whatever the dnsmasq problem was after a few hours.)

Depending on your use case CoreDNS may also be an option.

is there a plugin to serve DHCP I've missed?

https://github.com/nextdhcp/nextdhcp

was currently writing a rust cli that's called by an incantation of dnsmasq --script my_bin but it's a nightmare to debug.

A couple of notes:

If you can, add the support for CIDR style netmask (/8, /24 etc). Nothing wrong without it, but when you hop between them constantly mistakes happen.

In the example.yaml you state what option 3 is required - but it's not. It is absolutely a valid way to run some network without providing a default router.[0] If this is somehow is really a requirement anywhere than that requirement should be removed.

One of the things what I often use in MS DHCP is the server wide options, so some basic option values are configured on the server level and I add/override the needed ones on the scope level.

PS while the name is amusing, it is a possible IP conflict there, especially since you clearly state you know about Dora.

[0] And you can still provide the routign info through the option 121. Even for 0/0

> One of the things what I often use in MS DHCP is the server wide options

The config format was created primarily to be easy to generate & consume programmatically, so the format is fully 'flattened' with no real abstraction. But this may be something we add later. It's also why the option format is the way it is. I'm sure we could make some improvements there to make hand writing easier.

I appreciate your other suggestions, thanks for taking the time to look through things. I will have a look at both the router option and the CIDR input.

I didn't look at the source code. I like dnsmasq for its flexibility. I understand it. It's great. But the learning curve is very steep. Which might be because of the options I wanted in a certain way.

I'll look at this and see if it's more consistent.

Also, if anyone knows a good device to use at home with router and switch (>4 ports) and ap that runs open source firmware... let me know. I don't like SRM on the Synology, and UniFi has gone the Apple way.

I use pfSense on a small firewall appliance that has 6 intel ethernet ports and then a AP from TP-Link that is only an AP (no dual function).

Several people have recommended GL iNet routers (OpenWRT) but I haven't tried their gear myself yet.

I looked into it some time again and the shipping to Canada was at a significant premium, like half the cost of the device itself, and the models which were available on Amazon was a small sliver of their product range :(

Hard to go wrong with opnSense, it's open and very powerful and configurable. I run multiple VLANs, wireguard and OpenVPN, DHCP with some static IP mappings, dynamic DNS for my on-prem services, LACP for link aggregation to my core switch, multiple ISP connections with policy based routing (it's not as good as SD-WAN here but it's the best I've found that's open). Worth checking out, at the least.

Whats the motivation behind this and how does it compete with ISC DHCPd and Kea? I am operating a mid sized infrastructure using ISC DHCPd and plan on the long run to migrate to Kea for its API, Stork interface and convenience tools.

ISC DHCPD is end of life [1] and unlike Kea this server is written in a memory safe language (Rust). So from a security perspective alone that's enough motivation to start a project.

[1] https://www.isc.org/dhcp/

I touch on this below: https://news.ycombinator.com/item?id=33895978

Wouldn't say Dora is competing. The project is in the early days and isn't feature parity to either. Some in the industry have voiced gaps & concerns that I reference in my comment.

Something that is sorely missing from the FOSS world is a DHCPv6 server that supports RADIUS. The only one I am aware of that is considered production ready is Kea ($$$). FreeRADIUS has one in development but I do not know when it will be complete.

nice, there's definitely a space for some stateless DHCP server backed by netbox or something with a local cache for speed. selfishly hope to see it move. That way or at least facilitate.

my gut says DHCP<>netbox<>coredns backed by vault for x509 could be pretty sweet

Anyone try this on Windows Server?

I haven't, but I am currently running this on a raspberry pi using Ubuntu.

dora is for:

Discover: first request broacasted by client in order to get lease (@ip and others configurations)

Offer: response bradcasted by server with the lease (@ip and others configurations)

Request: broacasted by the client in order to confirm the usage of this lease(from this server, others server can release other lease)

Ack: ack from server

Yup. Although, to be pedantic, the offer/ack is often unicast. Here's the relevant bit from the RFC.

> If the 'giaddr' field in a DHCP message from a client is non-zero, the server sends any return messages to the 'DHCP server' port on the BOOTP relay agent whose address appears in 'giaddr'. If the 'giaddr' field is zero and the 'ciaddr' field is nonzero, then the server unicasts DHCPOFFER and DHCPACK messages to the address in 'ciaddr'. If 'giaddr' is zero and 'ciaddr' is zero, and the broadcast bit is set, then the server broadcasts DHCPOFFER and DHCPACK messages to 0xffffffff. If the broadcast bit is not set and 'giaddr' is zero and 'ciaddr' is zero, then the server unicasts DHCPOFFER and DHCPACK messages to the client's hardware address and 'yiaddr' address. In all cases, when 'giaddr' is zero, the server broadcasts any DHCPNAK messages to 0xffffffff.

Reply on news.ycombinator.com