Comment by IsBisu | Hacker News Reader

Comment by IsBisu | Hacker News Reader

You say in your bench that Dokan support does not support memory mapped files but it does as I can see with fsx (memory mapped read/write enabled) running on their CI https://ci.appveyor.com/project/Maxhy/dokany/build/1.0.1-48/... It is your implementation that doesn't support it.

You add lock (that will slow down the result) on Dokan implementation only https://github.com/billziss-gh/memfs-dokany/commit/03ebaebb1... After that you see that dokan is faster to list files (https://github.com/billziss-gh/winfsp/commit/595a77bd2e45347...) You improve YOUR MEMFS winfsp implementation https://github.com/billziss-gh/winfsp/commit/f7ca9f05221040a... but not on dokan ?

You also improve memory allocation on winfsp memfs implementation and again not on dokan https://github.com/billziss-gh/winfsp/commit/d12234bb01b3620...

And last point...running it on VirtualBox O_o ? Even running it 5x time and take the best time. You cannot be sure that virtualbox has not corrupt your test. For me a benchmark on a VM is not reliable at any point.

My point is even if Winfsp is fast on your test with an unfair FS, I am pretty sure there is not difference with real applications on a non-virtual machine.

> You say in your bench that Dokan support does not support memory mapped files but it does as I can see with fsx (memory mapped read/write enabled)

It was me that ported FSX to windows so I am quite familiar with this test.

Here is the code that fails in fsbench:

    Mapping = CreateFileMappingW(Handle, 0, PAGE_READWRITE, 0, FileSize, 0);

I would be happy if you explained to me how Dokany/MEMFS fails this code.

> You add lock (that will slow down the result) on Dokan implementation only https://github.com/billziss-gh/memfs-dokany/commit/03ebaebb1....

The lock exists on WinFsp/MEMFS. It is just taken in a different place:

    https://github.com/billziss-gh/winfsp/blob/master/src/dll/fsop.c#L60

> After that you see that dokan is faster to list files (https://github.com/billziss-gh/winfsp/commit/595a77bd2e45347...) You improve YOUR MEMFS winfsp implementation https://github.com/billziss-gh/winfsp/commit/f7ca9f05221040a.... but not on dokan ?

The WinFsp/MEMFS had indeed a problem which I fixed, but it was not WinFsp related (it was MEMFS related - aka my test user mode file system). I attempted to apply the same fix to Dokany/MEMFS, but I could not because Dokany does not pass an Offset parameter to its FindFiles operation.

> You also improve memory allocation on winfsp memfs implementation and again not on dokan https://github.com/billziss-gh/winfsp/commit/d12234bb01b3620....

Actually that is incorrect.

    https://github.com/billziss-gh/memfs-dokany/blob/master/memfs-dokany.cpp#L88

> My point is even if Winfsp is fast on your test with an unfair FS, I am pretty sure there is not difference with real applications on a non-virtual machine.

No. WinFsp is faster, because it is faster. Feel free to send me any pull requests that you want for Dokany/MEMFS and I will incorporate them into the project and re-run the tests.

>Mapping = CreateFileMappingW(Handle, 0, PAGE_READWRITE, 0, FileSize, 0);

So it is more a "case" not handled by dokan rather that all memory mapped not handled ? Is this made in purpose just to remove dokan from the bench ?

>The lock exists on WinFsp/MEMFS. It is just taken in a different place:

It changes nothing on what I said. You added it on dokan side without reason and not on winfsp memfs.

>send me any pull requests that you want for Dokany/MEMFS

A pull request will not change that a real application on a non-virtual machine will not see the difference.

> >Mapping = CreateFileMappingW(Handle, 0, PAGE_READWRITE, 0, FileSize, 0); > > So it is more a "case" not handled by dokan rather that all memory mapped not handled ? Is this made in purpose just to remove dokan from the bench ?

I do not understand this.

Are you familiar with how memory mapping works on Windows? CreateFileMappingW is it. If that API fails you do not have memory mapping.

Furthermore I challenge you to show me how Dokany/MEMFS makes this API fail. If you find such an issue I promise to fix it and re-run the tests.

>The lock exists on WinFsp/MEMFS. It is just taken in a different place: It changes nothing on what I said. You added it on dokan side without reason and not on winfsp memfs.

I added it on Dokan side, because I needed to protect the Dokany/MEMFS data structures against corruption. On the WinFsp side this lock is taken automatically as a convenience (although one can remove it if the file system can cope by using a custom "operation guard" (FspFileSystemSetOperationGuard) -- it is not needed for WinFsp operation).

The exact same lock is taken on both sides. There is no unfair advantage to WinFsp. If you design a file system that takes no locks and works on both WinFsp and Dokany, I will be happy to run the tests against that file system.

> A pull request will not change that a real application on a non-virtual machine will not see the difference.

All the "real application" has to do is to open a file and issues some I/O. Just because Dokany does not cache I/O it will be 9x-10x slower than WinFsp.

> The exact same lock is taken on both sides. There is no unfair advantage to WinFsp. If you design a file system that takes no locks and works on both WinFsp and Dokany, I will be happy to run the tests against that file system.

Alternatively you can show me that there is a lock already taken within Dokany for this purpose (protection of the data structures of the user mode file system) and I will be happy to remove the lock and rerun the tests.

BTW, here is how FSX calls CreateFileMappingW.

https://github.com/billziss-gh/secfs.test/blob/60f9f44b87731...

I do not see a big difference to the fsbench call. It may be that fsbench attempts to make a mapping that is 4MB long. Now you can fix that problem in Dokany ;)

Reply on news.ycombinator.com