Hacker News Reader

Comment by agapon | Hacker News Reader

Great blog post! Something from personal experience. OpenZFS on FreeBSD feels mostly like a port of illumos ZFS where most of the non-FreeBSD-specific changes happen in illumos and then get ported downstream. On the other hand, OpenZFS on Linux feels like a fork. There is certainly a stream of changes from illumos, but there's a rather non-trivial amount of changes to the core code that happen in ZoL.

View on HN

This is because Martin Matuška of FreeBSD has been focused on upstreaming changes made in FreeBSD's ZFS port into Illumos. At present, the ZFSOnLinux project has had no one dedicated to that task and code changes mostly flow from Illumos to Linux. This is starting to change. A small change went upstream to Illumos earlier this year and more should follow in the future.

That being said, there are commonalities between Illumos and FreeBSD that make it easier for the FreeBSD ZFS developers to collaborate with their Illumos counterparts:

1. FreeBSD and Illumos have large kernel stacks (4 pages and 6 pages respectively) while Linux's kernel stacks are limited to 2 pages.

2. In-kernel virtual memory is well supported in FreeBSD and Illumos while Linux's in-kernel virtual memory is crippled for philosophical reasons.

3. FreeBSD and Illumos have both the kernel and userland in the same tree. FreeBSD even maintained Illumos' directory structure in its import of the code while ZoL's project lead decided to refactor it to be more consistent with Linux.

Difficulties caused by these differences should go away changes made in ZoL to improve code portability are sent back to Illumos.

FWIW, if there's anybody interested in learning about the ZFS code base, we'd love help porting patches from ZoL into Illumos and vice versa. That's a good way to get a new developer integrated with the code and process surrounding each platform.

I'm interested in point 2, can you clarify how and why Linux in-kernel virtual memory is crippled or provide a link?

There are two issues:

1. Page table allocations use GFP_KERNEL, even when done for an allocation that used GFP_ATOMIC. This means that allocations that are needed to do pageout and other things to free memory can deadlock. There is a workaround in the SPL that will switch it to kmalloc when this issue occurs. There is also a new mechanism in Linux 3.9 and later that should render this unnecessary.

2. The kernel deals with kernel virtual address space exhaustion in vmalloc() by spinning until memory becomes available. This is not a problem on current 64-bit systems where the virtual address space used by vmalloc is larger than physical memory, but it is a problem on most 32-bit systems.

Reply on news.ycombinator.com