LF: Fully Decentralized Fully Replicated Key/Value Store
github.com
github.com
But he should be working on getting it into people's hands, instead of creating new tech.
Seeing this leads me to believe that zerotier wants to do this whole global network thing. This is not what I want from zt.
This probably came from questions like "how can we trust you as a network controller". And the answer is to create some tech. But in that case, why not simply offer an AMI on aws for a controller?
All assumptions of course. Upon further checking:
From https://news.ycombinator.com/item?id=20329060 and https://www.zerotier.com/lf-announcement/ :
> We are going to use it to fully decentralize ZeroTier and for some specific enterprise customer projects, but others can use it for other stuff.
> Our interest in decentralizing the roots wasn’t just philosophical. Over the years we had numerous requests by policy-bound or very (justifiably) paranoid customers to run their own infrastructure that could be logically separated from ours as much as possible
I dunno. I'll have to see what it turns into. But it surely explains the silence and lack of updates on zt/zt-one
So yeah, defocus is a legit criticism but not because of this project.
This was also a two birds with one stone project in that it is also useful on networks with arbitrary connectivity graphs and unreliable connectivity. An enterprise customer of ours funded some of this R&D for that purpose and we combined their objectives with our root server decentralization and network robustness improvement goals. I wish I could talk about the enterprise customer's project. It's quite cool but we're under NDA. All I can say is that it's industrial and involves mobile ad-hoc networks.
There is admittedly quite a lot behind this that is not apparent from the README, and from the comments here its clear that people have trouble understanding this because it is not yet another cryptocurrency and isn't solving exactly the same problems. (There is some overlap but this is different.) We built this to solve a problem nothing else solves, but I figure it will take the appearance of use cases for people to get what that problem is. The people bringing up etcd and consul are closer than the people bringing up coins.
Beautiful solution. Adds so much to what ZT can provide for borderless networking. First thing I'd implement is distributed DNS for usability. Then probably the config backend for the network layer.
This issue -- of creating equivalent proofs of work -- is what I was most interested to see if there was any novel art (which it appears there isn't).
Bitcoin's difficulty adjustment is the only well-functioning algo that deals with the economic coordination issue around how to -- without any third party coordinator -- equate the value of a proof of work made by computer's of differing computational capacities (a phone, an ASIC). All PoW consumed by Bitcoin has marginally positive contribution to the system. Sacrificial PoW (as in LF, hashcash) only has value if reorgs happen. It also results in a system with tenuous game stability properties ("When will it be uneconomical for an attacker to reorged these records out of the DaG?" "Can't say, blocks' PoW requirement is arbitrary") vs bitcoin ("When will this 50 BTC txn be uneconomical for the most adversarial miner to reorg out?" "well there's about 14 BTC of block reward per block so about 4 blocks ~40-60 minutes"
> adamierymenko 478 commits 403,913 ++ 148,611 --
Impressive productivity.. Worried about the Sybil attack. The documentation uses the "foolproof" wording, but the overlay might not be attack-resilient and creating a million fake identities would be sufficient to dominate the system. Even a loose connection to the main DAG might be sufficient to hurt LF.
> Oracle nodes document low reputation records in commentary records. When a node sees a commentary record it extracts this information and caches it internally in an indexed table.
That could be an attack vector. Broadcasting negative information can often be exploited in my experience. Slandering attack with a 100 Mbps VPS might cripple the network. Could this be altered into positive re-enforcement loop?
> Casual users can choose to trust nodes that are generally trusted.
How will central server "lf.zerotier.com" handle copyright claims by Hollywood (people will try to upload also large files)? (disclaimer, creator of Tribler, we're designing systems like this within our university lab for 14 years and 8 months)
Depends on what hardware the attacker has available. A GPU offers at least an order of magnitude increase in efficiency over a CPU, while FPGAs and ASICs offer further orders of magnitude increases. The particular PoW used here (birthday collision) aims to be memory hard but falls to a sublinear memory-time tradeoff.
That's assuming you need a reorg to adjust values. The original purpose of hashcash was anti-spam. That lends itself to allowing variable work factors and having the recipient's classifier regard higher work factors as a stronger not-spam signal, with the baseline set by the recipient administrator, or the mail daemon packager setting defaults for this year's distribution, as a trade off between false positives and false negatives.
Meanwhile a sender default of e.g. calculating hashes for ten seconds per message would cause the amount of calculation done to automatically rise over time as people buy faster hardware, but someone with slower hardware can reduce the false positive rate for their messages by increasing the time spent per message. Or someone with a high message volume but widely known and sterling sender reputation could reduce computation per message and rely on their good reputation instead, while less well known senders avoid spam classifications by doing more computation.
It works because it's all probabilistic anyway and having non-zero false positives or false negatives was already the status quo, so something that gets you closer to true is valuable even if it only solves 99% of the problem and not 100%.
Not sure how I feel about this, and I'm sure someone's going to figure out how to remove that 'feature' in a fork... Does kinda seem against the nature or p2p and blockchain
From https://www.martinfowler.com/bliki/TwoHardThings.html :
"There are only two hard things in Computer Science: cache invalidation and naming things.
-- Phil Karlton
Long a favorite saying of mine, one for which I couldn't find a satisfactory URL.
Like many good phrase, it's had a host of riffs on it. A couple of them I feel are worth adding to the page
There are 2 hard problems in computer science: cache invalidation, naming things, and off-by-1 errors. -- Leon Bambrick
There are only two hard problems in distributed systems: 2. Exactly-once delivery 1. Guaranteed order of messages 2. Exactly-once delivery -- Mathias Verraes
there's two hard problems in computer science: we only have one joke and it's not funny. -- Phillip Scott Bowden"
So... What's the realistic use case for this? (In both public and private network variation) Zerotier's network seems like one of the very few projects where this makes sense.
(From the tech point of view, this looks really cool though!)
Think of it (as the readme says) as consul for decentralized open systems.
Not someone trying to overwrite existing records, just constantly inserting new records.
BTW "high resource use" is in reference to things like tiny embedded devices. You won't be putting this in a fridge magnet or a smart light bulb. This will run fine on a $10/month VPS or a Raspberry Pi 3 or 4, though if the data set gets huge you might need to attach some storage.
Nice! Thinking a bit about this - this sounds like it has real potential vis-a-vis traditional CA. With a (many) globally trusted db - it sounds quite feasible for eg Mozilla or Debian to simply store (or "bless"/trust) any number of root certs - and record that in lf.
And the same could be done for signing binaries (whether via x509 or some other scheme - "all" that is needed is some way to "discover" trust of the signing key after all - be that a gnupg key or a x509 cert etc).
LF looks like a well-reasoned project with a solid direction that is unique and applicable.
I do disagree with some of its design choices (DAGs aren't very scalable, full replication versus partial/sharded, etc., and GPL instead of MIT - which my decentralized DB, http://gun.eco is MIT/Apache2), but for the problem it is trying to solve, it seem like you've made the right choices for LF and its use cases.
Congrats and keep it up! Excited to hear about this over time. :)
In some scenarios, you can rebase/snapshot to clean up history, but these usually require a type of centralization or consensus, which defeats the point of using something like git.
As a result, DAGs can only be used to replace a subset of apps/tools out there.
The most important/used apps, though, are indexed lists. Things like:
- Google rankings
- Reddit homepages
- AirBnB listings
- Ubers nearby
If you're updating a geo-index 100s of times a second, like in the case of cars' GPS locations, then you're just wasting resources with a DAG that you'll never use and wind up bottlenecking the system, preventing scaling beyond a certain threshold. I've dealt with this in practice, and it was no fun.
We switched off DAGs, and our biggest production deployment now has been with 15 million monthly users. This is far far far beyond the scale of any of the previous systems.
There aren't any externally packaged drivers with a different licence, and you can't include their native code in your commercial projects without having to share the code due to the inclusion of the GPL3 licenced code.
https://gist.github.com/rjhansen/67ab921ffb4084c865b3618d695...
LF isn't a wrapper around SQLite at all. It just uses SQLite for meta-data and index information because it happens to be a very capable database and because the ability to do relational queries saves a ton of time vs. hand-rolling such queries on top of something like LevelDB.
The last issue is something I've never understood about cryptocurrency code bases. Why do they insist on implementing a slow ad-hoc relational model on top of a simple KV store when they could just use SQLite? I assume it's because they assumed SQLite would be slow, but it's really not unless you are doing massive numbers of writes.
>> "The European Patent Convention states that software is not patentable. But laws are always interpreted by courts, and in this case interpretations of the law differ. So the European Patents Office (EPO) grants software patents by declaring them as "computer implemented inventions"."
...but it's interesting, ie. what would it mean exactly if bitcoin was built on something patented? It's completely distributed so nobody to sue. Would the repo be banned on github? Let's say it would and the development would move to linux kernel like development, then what? US based exchanges would probably have some premium fee and that's it?
Caught a demo of a blockchain product performing at over 6,000 tps on a single shard spread over about a dozen geographic regions on a test net last week. Expected to be more performant than anything out there today when at scale.
# openssl speed ecdsa
Doing 256 bits sign ecdsa's for 10s: 494208 256 bits ECDSA signs in 10.00s
Doing 256 bits verify ecdsa's for 10s: 161277 256 bits ECDSA verify in 10.00s