LizardFS – Reliable and Scalable Distributed File System | Hacker News Reader

LizardFS – Reliable and Scalable Distributed File System

github.com

36 pointsturrini16 comments

This is a fork of http://www.moosefs.org/

I don't see what lizard does that moose doesn't.

Is this just a re-branding of moose in hopes to sell consulting for it?

It seems that's the goal: https://lizardfs.com/

This is interesting, especially the native windows client, but it's based on the same tired architecture as HDFS, and has all the same limitations.

1) Single Master for Metadata

One of the biggest weaknesses in this model is that you can only scale the master so far. For metadata responses to be snappy, everything in your working set has to fit in RAM. That imposes hard limits on your number of files; 100M is achievable, 1B can even be done, but more than that and you're in for hurt.

2) Chunkserver to chunkserver replication

As the moosefs write diagram shows, you're waiting for chunkservers to walk a linear tree, and you get the slowest performance of each one. Which, as it turns out, is on average pretty bad. If you have a 99th percentile latency of 10x your average, you lose 20% of your speed with three replicas. Not to mention the fact that your 'single point of failure' is actually any of the three of those chunkservers, with a long retry stall if one of them errors. LizardFS supports a xor mode, so it may be avoiding this by having clients talk to more than one chunkserver directly. Combined with proper queueing and retry logic, this reduces the effect of the long tail significantly.

Last time I investigated about a year ago... I saw some MAJOR as in "DO NOT USE, UNSAFE, WARNING, DANGER WILL ROBINSON", issues in this project... Silent data loss/corruption type stuff, and I did not see sufficient activity on a project of this complexity to make me feel at all comfortable using it.

I've used tahoe lafs before, but not many other distributed file systems.

There doesn't seem to be much information provided in this link, and their website doesn't help much either. Does LizardFS do full end to end encryption too? That's one of the more important things to me.

The MooseFS reference will help you out:

http://www.moosefs.org/

You might also find this list useful:

https://en.wikipedia.org/wiki/List_of_file_systems#Distribut...

So, no reasoning for a fork? What was the itch? Just to use GPLv3 instead of GPLv2, can GPL be "switched" just like that?

Looking at the MooseFS source files, I don't think they can. The license header of the files says:

   /* ...
    * MooseFS is free software; you can redistribute it and/or modify
    * it under the terms of the GNU General Public License as published by
    * the Free Software Foundation, version 2 (only).
    * ... 
    */

I think the reasoning was to add missing features, mainly HA, which moosefs did not have at the time and currently provides it in its paid version.

One thing I never understood though is why when forking they decided to rewrite everything in C++. Wouldn't be better to just write it from scratch?

I´ve used MooseFS which LizardFS is based upon and its very stable in production!

Does LizardFS support atomic rename?

Reply on news.ycombinator.com