adamch

I was aware that other DNS clients existed, but just using one of them would have taught me as much!

Oops, thanks so much! I forgot to check 'em. Fixed now.

Does anyone know why Axum is so much faster than the basic "spawn a new task to accept the connection and write the response" async webserver? I'm curious what they're doing under the hood. They're both using the Tokio runtime, but Axum is based on Hyper. So I guess it boils down to how Hyper uses Tokio to efficiently accept/serve TCP connections?

EDIT: Actually I'm guessing a lot of it comes down to encoding the HTTP responses, now that I think about it.

My previous blog post about Nom tries to do that: https://blog.adamchalmers.com/nom-chars/

That is exactly why I ended up learning bitwise Nom, and the inspiration for this blog post :)

I agree, it's surprisingly usable given how complicated parsing can be! I think the key to its ease of use, at least for me, is that you can unit test the individual parsers. You can program very iteratively, building it piece by piece and testing as you go.

This is much easier than trying to design the entire parser on a whiteboard/paper and implement it in one big go, I think.

That's really neat, thanks for showing me. This looks very straightforward for parsing a straightforward format.

I think Nom might be more useful as the format gets more complicated... the combinators can really save you quite a bit of time. Having things like many0, or delimited_pair, or length_count already tested and documented saves time compared to implementing + testing it yourself. It's cool seeing how many different approaches you can use to solve the same problem :)

Oh, interesting. I've used Bitvec for serializing binary data, and I can see how you can use it for parsing single bits or u8s... do you have an example of how to parse, say, a 3-bit binary number with Bitvec?

Cloudflare | Data loss prevention team | Remote (must live in same country as a Cloudflare engineering office, including USA, Australia, UK, Portugal, Japan, Singapore)

I am the sole engineer on a new team at Cloudflare, and I'm looking for a coworker! It'll be 90% Rust, 10% Go. Prior experience with Rust, distributed systems or network protocols is a big plus, as is working in a US timezone (currently both myself and my manager are based in the US). We're building ambitious software that will keep customers' data safe at high scale.

Full-time work only, no part-time/casual unfortunately.

JD: https://boards.greenhouse.io/cloudflare/jobs/3812886?gh_jid=... Please email achalmers@cloudflare.com

Oops, yes

Serde is good for self-describing formats like JSON, YAMK, TOML etc. At least that's what I use it for. Nom isn't necessary for them!

Nom is good for protocols where you need to know the schema in advance. E.g. a binary key-value store, where the binary in the key and value has to be deserialised into some special rules.

Nom can also handle bit-level parsers very easily. It's easier to use the parsers in nom::bits than remembering all the bit-shifting tricks I haven't used in five years.

But I'm not an expert on Serde, I suspect it can do bit-level too. I've never written my own Serde library, I just use crates like serde-json.

Hi Josh! "Parse, don't validate" is one of my favourite programming blog posts of all time. Between Nom and Serde, I find it very easy to use that approach in Rust.

Interesting! The quote "sometimes, nom can be almost as fast as regex" has been removed from the Nom docs, so it's probably become faster since 2018 when that Cloudcity article was written. In fact, Nom now claims to "outperform many parser combinators library like Parsec and attoparsec, some regular expression engines and even handwritten C parsers" [1].

Someone used Nom for Advent of Code last year and found "The regex approach benchmarked at about 1ms while the parser approach benchmarked at 145 nanoseconds." [2] Maybe I'll try benchmarking Nom vs. regex for a follow-up post.

I find regexes are easier than parser combinators for simple tasks. If the problem is small, parser combinators are overkill. But if the problem gets complicated, I think parser combinators are more readable. You can break a parser combinator into small, well-documented, well-tested parts more easily than a big regex. But I still use regex more in my work, because most complex parsing I just throw into serde.

[1] https://github.com/Geal/nom

[2] https://www.christopherbiscardi.com/advent-of-code-2020-in-r...

Very happy about the speed and privacy improvements. I'm excited to install this.

Yes, this is just a simplification of the real problem I was solving: docs.rs/restartables

Rust 1.51 stabilized a lot of const generics use-cases, but there are still some advanced parts which are only on Nightly. For example, doing math with const generics, like an array [String; WIDTH * HEIGHT], still requires nightly.

This is, literally, the exact same reason I use const generics. I was so excited when I started using them on Rust beta and could shave off some allocations!

Unfortunately I can't answer because I never learned C++. By the time my career required me to learn a low-level systems language, Rust was available, so I didn't have any need for C++.

Last night I was writing a blog post about how to use const generics. I included instructions on installing Rust Beta, and setting your directory to use the Beta compiler instead of Stable. I started to write a disclaimer like "By the way, this will be stabilized in a future version of Rust, probably on..." and realized that const generics were being stabilized in less than 9 hours! So I removed all that stuff.

If you're interested in some concrete examples of using const generics to write faster code, my post on them is here: https://blog.adamchalmers.com/grids-2/

Wow, awesome code. I really respect the 2000 line match, it's a really neat abstraction. Keep some state, have an exhaustive list of everything users can do, and define exactly how each possible user action can change that state. Beautiful.

My job isn't to write a program that shares objects between threads. My job is to write a program that responds to web requests or stores user data or responds to queries or checks the state of certain services.

In some languages and frameworks, I might need to share an object between threads to do this efficiently. In Rust there are other options, e.g. using Arc or a reference or moving the object into a task. In my years of programming Rust I've never wanted to share some object on a thread, because there are frameworks and libraries (Actix-Web, Tokio, Rayon, async-std, others) that give me a higher-level abstraction for doing this.

I can understand using this as a point of comparison if you're not very familiar with the language. But I have to confess that in my three years of working with Rust, I've neer had to worry about threads. I've been able to use a library to solve my concurrency or parallelism problems so that I can go back to worrying about design or reliability concerns.

This won't be everyone's perspective, after all, someone has to write those libraries. But it is mine, and I think users of such libraries far outnumber authors of such libraries.

This is pretty overblown. I write async rust every day for my job, just fine, with no real problems. Probably because I'm consuming other libraries, I'm not trying to write my own. I use well-tested libraries like Actix-Web or occasionally Tokio. I've migrated multiple projects from futures to async/await once the syntax came out. '

The problems the author is describing might apply more to library authors, but for the many Rust programmers like me who are trying to solve problems, not write a framework, async is easy and practical.

You can't abstract over it in the same way that you can in Haskell, because you have to manage ownership. You can't abstract away ownership as easily, because the language is designed to make you care about ownership.

I write Rust code for my day job, and I frequently use map/reduce/filter. IMO if I can write all my collection-processing code using primitives like that, it's got first-class functions.

Wow, higher-kinded data. I've never heard of that, but it would absolutely be useful. E.g. for converting JSON inputs, where any field might be missing, into a version of the data which definitely has values for all the fields. Or a version of the struct where the values are actually read from a cache/database.

How do you "own" something that everyone else can consume and replicate? Owning a physical item implies a degree of control over it. Owning a public digital item doesn't seem to grant any kind of control.

I feel like this is a classic case of "tree falls in a forest, does anyone hear it?" where the confusion relies on the word "hear" being used in two ways (objective sound waves travelling through air, vs subjective perception). Is the word "own" being used in two ways here?

ripgrep's src/ contain both library code and unit tests. The grep source code appears to use different tests/ and src/ directories.

ripgre's source code files appear to be between 1/3 and 1/2 unit tests, in the few samples I checked. For example, https://github.com/BurntSushi/ripgrep/blob/master/crates/sea...

Perhaps the developers had a success metric other than LOC.

Yes, this was my immediate thought too. I love writing Haskell, but Rust has become my daily driver, both at work (Cloudflare) and personal projects. It gives me the excellent type system, better domain modelling, great ecosystem (serde, actix-web, rayon, reqwest, diesel, tokio) and performance.

I've basically traded away some nice abstractions (functors, monoids etc) for the ability to debug error messages more easily, and not have to convince people to learn/support an obscure language. Good deal IMO.

It's predictive of a forthcoming legally-significant event.

At a general level, you may work with users, and it's important to not have fallacious thinking about users, e.g. "if I can dispel belief X their behaviour will change to Y". Instead, Talib would bet their behaviour will still be like others in the X-believing community. When tech is currently doing a lot of soul-searching about the social effects of e.g. Facebook it's useful to have a theory to explain why predictions of user behaviour were wrong.

At a more specific level, I think this is very useful for thinking about programming language communities and how they affect behaviour.