Hacker News Reader

Comment by ec109685 | Hacker News Reader

Interesting they would discuss system call overhead of opening a file, reading from it and closing it. Seems like in almost all cases the open and close calls would be overwhelmed by the other operations.

View on HN

For lots of small files, that might not be the case.

(I worked on a FUSE filesystem that had these issues.)

It seems more straightforward to fix your data-in-files layout than to implement a novel in-GPU filesystem, though.

I think the main benefit here is not having to do memory copies through the CPU, which frees up memory bandwidth for other things.

There are plenty of cases where you can't just change the file layout. And the GPU filesystem is being implemented by someone else, so the choice is: migrate your data to another filesystem OR fix the data-in-files layout, even though the files may come from completely different source than your application, the layout may be a standard or other applications may depend on it, or you can't easily change it for another reason.

If you can get the data into the GPU-native filesystem, you can change the data layout at least as easily. The point is there is some sort of data ingestion pipeline involved.

> It seems more straightforward to fix your data-in-files layout than to implement a novel in-GPU filesystem, though.

You can improve file-open overhead in conventional filesystems, too. Including the FUSE one I was working on.

Sure!

Reply on news.ycombinator.com