The kernel has 6 separate ASN.1 parsers | Hacker News Reader

The kernel has 6 separate ASN.1 parsers

twitter.com

The Linux kernel is one of the world's largest collaborative projects, and it shows. The strength of Linux is that tens of thousands of people work on it, and it can be expected to support virtually all technology you might want to apply it to. The disadvantage is that coordinating tens of thousands of people is nearly impossible, and while Linux has central leadership, they're mainly concerned with organizing development and less with providing a unified vision or architecture for the kernel. Each subsystem has a great deal of soverignty in their affairs.

That being said, if a group took it upon themselves to be a general-purpose "refactoring" team, without a focus on a single subsystem, they would probably be able to address these problems, albiet only if they have a strong stomach for bikeshedding. But Linux is made of volunteers, so you, dear reader, need to step up if you want this done.

> That being said, if a group took it upon themselves to be a general-purpose "refactoring" team, ...

I pretty much agree with you, but I'm not sure if it really needs a special cleanup task force. Stuff is being refactored and cleaned up on subsystem level all the time already.

If, for instance, you do a `git log fs/squashfs` (an example I picked because I'm somewhat familiar with it, and there is comparatively little feature development) and skip past the maintenance fixes, there are a bunch of refactoring patches where common filesystem or block layer code was changed.

On the mailing lists I'm on I somewhat regularly see Christoph Hellwig submitting larger cleanup patch sets for block layer stuff (even when you don't count the big cleanup spree during the first 2020 lock down).

In the MTD subsystem there was a multi-year effort (earliest discussion with Boris and Miguel about this that I remember right now was around 2016) to refactor the entire subsystem, pull out a generic NAND framework, etc...

But those are just some of the instances I can think off of the top of my head, from subsystems I'm personally somewhat familiar with.

Regarding the ASN.1 parsers, Linux does have a `lib/` directory with common algorithms and helper functions in it. In fact one of the parsers being linked to is from there (https://git.zx2c4.com/linux/tree/lib/asn1_decoder.c).

Somebody(TM) would "simply"(TM) have to patch the code in $SUBSYSTEM to use that instead, delete the old one and submit a patch set to the appropriate mailing lists. Of course it's a lot easier and less risky to simply post a mocking tweet and walk away.

EDIT: In fact, this has already been done for one of them: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/lin...

> Of course it's a lot easier and less risky to simply post a mocking tweet and walk away.

This tweet seems to be from Jason Donenfeld, who worked for years to refactor linux kernel crypto APIs as part of upstreaming wireguard, apparently with some success. (Random link for some context, though I haven't followed the whole arc of that story: https://www.phoronix.com/scan.php?page=news_item&px=Crypto-A... )

If you're aware of that context and think that Donenfeld is too critical of the kernel and hasn't done enough to fix it, then OK, but from an outside perspective it sure looks like he's put a ton of time into fixing things and earned some right to point out remaining issues.

Thanks for the info! I only saw the twitter thread doing the usual point and laugh game and didn't bother looking further into at, given how often you stumble across those.

Not being a Twitter user but tangentially following Wireguard development, the name Jason Donenfeld does ring a bell, in contrast to some Twitter account named "Edge Security".

Interesting to know that the tweet from 2017 pointing out code duplication in the kernel is from the same person who, 1 year later, kicked off a brief flame war by wanting their own crypto library merged into the kernel alongside the existing one[1], before finally giving in and porting Wireguard to the existing API[2].

EDIT: Also thanks for pointing at the patch set for the crypto API. I have yet to look into that, as the last time I touched the kernel crypto API was a few years prior and I agree with some of Jasons criticism, although I can see the point of the kernel API being designed for asynchronous workloads, trying to explicitly support hardware accelerators, which I was also playing around with back then.

[1] https://lore.kernel.org/lkml/CAKv+Gu9hOW0pOjDjrvg1q-v85aiwSZ...

[2] https://lists.zx2c4.com/pipermail/wireguard/2019-September/0...

> Linux is made of volunteers

This doesn't seem to be true to me:

https://lwn.net/Articles/780271/

The vast majority of contributors seem to be paid. Am I misunderstanding?

In those cases it would be the company that is volunteering, the contribution is still voluntary, and the company is paying someone to volunteer on their behalf.

The company is paying an employee to work on a product which their business probably uses in return for publicity, bigger voice in the direction or prioritization of bugs and features, and to ensure stability of the product they depend on

What does volunteer even mean? Does it just mean not-slavery?

No, it is a common myth that Linux, or many other bazaar-based projects are made by a thousand small, unpaid, helping hands, — perhaps because it sounds romantic.

Most of Linux is indeed the fruit of paid, professional labor.

> The vast majority of contributors seem to be paid. Am I misunderstanding?

Correct.

I would question the sanity of this refactoring team after 1 month of reading code written by so many people ^^

Joke aside, they would not even need to code IMHO, just be able to ask questions like:

  - why is it done this way?
  - we found common code in those subsystems, do you think we could generalize or do you have other concerns?

yeah especially since the whole point of ASN.1 is the agreement of the format of data structures so this means they never get the value out of ASN.1 that they potentially could.

A big problem IMO is that it's hard to find a good implementation of ASN.1 compilers that will work across (sub-)systems and that as a FOSS project gets properly maintained. If this can be solved it would open up incredible possibilities such as automated grammar based fuzzing[0].

[1] https://blog.doyensec.com/2020/05/14/asn1fuzz.html

Oh that link is awesome! Thank you!

That assumes the correct number of ASN.1 parsers in the kernel is "1". I would strictly prefer my kernel to include "0" (zero, nil, none) ASN.1 parsers. It's a trash spec designed by committee that probably needs its own bytecode machine to be parsed safely (that certainly is the track record of ASN.1 parsers in the wild).

How would you suggest that the kernel interoperates with systems that need ASN.1 such as X.509 encryption, CIFS filesystems, or BER encoding in the H.323 ITU standard?

For example, by having a userspace helper for it. Pretty similar to the same way that the kernel itself doesn't do firmware patching, but relies on userspace to handle the opaque blobs.

Firmware patching happens rarely, but decoding packets in the firewall would be far slower and more fragile if it required calling out to a userspace process.

I think there was a recent kefuffle in the python community because they wanted to depend on a Rust library to parse ASN.1 and some more obscure platforms weren't ready for that change.

Not in core Python, but in a widely-used library, yes. https://github.com/pyca/cryptography/issues/5771

Which is 6 too many. This garbage technology from the failed OSI project should have been buried with it.

Now that Linux is starting to be more receptive to rust (drivers only for now, but perhaps in kernel down the line), these parsers should be replaced by implementations in Rust asap.

Yes, someone should just drop by kernel mailing list and ask a very logical question "Have you considered rewriting it in Rust?" That would help remedy the situation fast.

Apparently they did,

https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-n...

They went above and beyond :). I was thinking small and talking only about parser.

Your snide comment is putting words into the mouth of nynx and then arguing against those words. A classic web forum tactic.

Applying Rust to harden a parser is exactly where Rust is being used. The parent didn't not say Linux should be rewritten in Rust, only that it could be applied to a parser that takes untrusted input.

As mentioned in a sibling comment, Rust support in the kernel is being worked on.

Okay, yeah that wouldn't go over well. Despite that, i think it makes sense to replace very dangerous sections of the kernel with a safer implementation. Parsers are very difficult to make safe in C.

Actually it went over fine [1]. Rust support in the Linux kernel however is still very nascent, & only suitable for drivers. The reason is that rustc only supports the more popular systems (hence drivers for that system are suitable). There's work ongoing to expand the utility of Rust in the kernel on multiple fronts, but for something like this that's built on all platforms, it might take longer. ASN.1 & other crypto format parsers are ripe for being rewritten in Rust & no one would really blink an eye I think (provided the other pieces of support are available to support all the architectures Linux does).

[1] https://lwn.net/Articles/829858/

This kind of mentality is going to make people resent Rust, it's not really the culture we need.

While it's technically true that Rust encourages safe practices, it's not a blanket fix-all for everything.

You are criticizing nynx for something they didn't say. As mentioned in another subthread, Linux is working on Rust support but only for drivers at this time.

Nynx did not say rewrite linux in Rust. They said apply Rust for making a secure ASN.1 parser, which is exactly where Rust should be applied to maintain safety properties.

Rust is being applied to multiple commercial operating systems at the driver level to harden them against known issues. Windows is already running Rust code in its USB subsystem.

Rust does not have tier 1 support for all the platforms that Linux supports.

That should not happen until Rust has support for all the platforms that Linux supports. As far as I know, Rust support is still missing on plenty of platforms.

Reply on news.ycombinator.com