Show HN: Goofys – a faster s3fs written in Go | Hacker News Reader

Show HN: Goofys – a faster s3fs written in Go

github.com

40 pointskhc23 comments

I appreciate any project that makes its strengths and weaknesses very clear and documents its consistency semantics.

Reading around[1], it looks like close-to-open consistency means that all writes must be flushed before the close() call returns, and all subsequent open() calls to the file must see the changes. IO may otherwise be buffered or cached by the FS.

1] Best source I found were these lecture slides: http://www0.cs.ucl.ac.uk/staff/B.Karp/gz03/f2011/lectures/gz...

The caveat is that goofys is subjected to the underly S3, which if you are using Amazon is only eventually consistent (http://areweconsistentyet.com/)

I applaud the "Filey System" clarification. Confusion between things that are very close to 100% POSIX compliance and things that don't even have that as a goal but adopt similar interfaces or structures has been a real problem. It's nice to see someone being up front about the distinction, and it looks like a useful project too.

Thank you! It's not a general purpose filesystem but I don't think that has to be the case to be useful. I am lazy and I want to do as little POSIX as possible.

That's wise. There are parts of POSIX that even I (a notorious purist) think are outdated, unclear, or just not worth the trouble. Appending and random writes sure would be nice, but I appreciate the difficulties involved. Maybe if I get a chance to learn more Go I'll be able to contribute in some of those areas.

I started learning Go a month ago and wrote goofys in it as an exercise. Looking for feedbacks from Go best practices as well as usefulness of reduced-POSIX filesystems.

riofs (https://github.com/skoobe/riofs) seems much faster than s3fs and deserves a spot in any benchmarks.

Yes I looked into it (I only found out about it after I started working on goofys). They have a stub flush() which does nothing (https://github.com/skoobe/riofs/blob/master/src/rfuse.c#L104...), so of course they are faster and the benchmarks won't be meaningful.

AWS S3 doesn't expose a flush()-like call in their API, hence what is the point of exposing something that the underlying service doesn't support?

data on s3 is durable after a successful PUT (or a complete multipart upload), so their flush() is implied.

exactly, so why do you need a flush then?

I looked at the code some more and they do handle release(), so much of my point above was invalid. I expect riofs's streaming write performance to be comparable to goofys because we both use the same implementation strategy.

Fun fact: the S3 API actually makes no such claim.

http://docs.aws.amazon.com/AmazonS3/latest/API/RESTObjectPUT...

Nope. Let's try the developer guide.

http://docs.aws.amazon.com/AmazonS3/latest/dev/storage-class...

Nope again. Let's try the FAQ.

https://aws.amazon.com/s3/faqs/

They claim S3 is "designed for" 99.999999999% durability, but there's neither a guarantee nor a clear definition of when an object starts to be covered by one. While it's both intuitively obvious and conventional wisdom that an object is durable at the end of a PUT, as far as I've ever been able to tell Amazon doesn't come out and say so.

That's one of the problems with company-defined "standards" BTW. This kind of issue would surely have been noticed and discussed in any kind of open standards process. It's what makes those processes so tedious. De facto standards can be turned around a lot faster, but there's a necessary sacrifice in precision to go with that.

Not surprising that s3fs is slower, it's implementation quality is not very high. Goofys looks much better on first sight. It would be nice if anybody could do a benchmark between goofys and riofs (without cache). But honestly, if you have some proper request handling there is not that much to tune. The biggest performance gains can be achieved from a good cache implementation and make such a system useful in a production environment, that's what riofs was written for.

Disclaimer: I initiated and supervised riofs.

first off apologies for the criticism about flush(), I only skimmed the code and flush() is usually where I look for first.

Anyways goofys at this point is just a toy project. There are more optimizations that I can potentially do (proper read prefetching) it's mostly good enough.

I've updated the benchmark to include riofs

This is cool. Any reason you chose not to cache reads locally (detect md5/mtime changes for a new read or something similar)?

Many reasons:

* I am not working at the moment and have some free time (shameless plug: resume at my profile), so I want to bound the amount of time I need to get something useful

* many archiving/backup workloads are WORN (write once read never), that and many streaming workloads (data processing, media streaming) don't really benefit from cache. (Unless your cache is as big as your data, but that's usually not why people use S3)

* for the use cases that cache can help, I think you can just use another layer of caching filesystem. I intend to write one if one doesn't exist already. I wonder if you can use cachefs with fuse filesystems? Let me know your use cases and I will think about it some more.

As far as I know, FS-Cache does not yet work with FUSE, though we (in Gluster) and others have often considered adding such support. You can actually get some caching within FUSE itself, but I think you have to go down to the low level (inode-based) interface to get any control over it.

the go fuse binding that I am using only has low level interface anyway. That said I think the parent meant on disk cache not vfs cache.

I want to see benchmark comparsion with the Rust version.

I see what you are doing there :-P goroutines do make certain things easier but there's nothing you can do with good old pthreads. One of the motivations for this project is for me to understand better what this Go hype is about.

Congratulations then! Hope you enjoyed writing code in Go.

Reply on news.ycombinator.com