IPFS, Again
macwright.org
macwright.org
It pains me to see a great idea like the Interplanetary File System still not working. I had similar experiences with IPFS and yes, we do need a project like this, only without the broken incentive structure attached to it. Why a "Filecoin"? We already have a tested and working native internet currency: bitcoin.
I've long believed that a browser itself should evolve to be both viewing visited websites but also to host said website for an x number of minutes/hour/days. A bit like webTorrent aims to work. This way a website that gets visitors gets decentralised hosters as well :-)
Edit: I'm not affiliated with them other than being a customer.
100 bucks!? I host a dozen unique domains with static content (and a few other services) on a 1/1 VPS for 3 euros/month. At Hetzner, not at a hyperscaler.
There are other providers with comparable offers, I mention Kimsufi because for some time I've had one of their low spec units running a couple of simple tasks.
You are not going to be getting brand-new unused high-spec kit with that sort of deal, of course, but you pays your money, you takes your choice!
If one of those people doesn't mind running as sys-admin and help-desk for the rest...
Though your $100/m is rather higher than it needs to be for that level of service so you won't need the same number of people for that cost/person, someone is still taking on a part-time job doing admin for others.
That’s why alternatives exist in order to solve these “problems” that the BTC community either choose not to fix or simply won’t. It’ll be interesting to see what crypto people would intend to use for something like IPFS.
IPFS is basically BitTorrent with magnet links - hashes to address content, if you want to be sure your obscure content will stay around then you have to seed it, etc. There's some fancier stuff on top, but if you know that's how it works, all else follows.
You can tell how "blockchain" IPFS is from the way that advocates seem chronically unable to tell "could" from "does" - stuff that doesn't exist and/or doesn't work is routinely talked about as if it's here and working in the present. This leads to disappointed posts like the one linked.
You make a really good point about communicating readiness - the author of the post is expecting more from pre-alpha software than it is ready to provide.
Building a platform is not like building an app. And a platform that changes significant parts of internet infrastructure is different from just building any platform. I worked on Firefox for 13 years... the internet is a harsh and fickle thing.
That said, people are using IPFS with millions of users today.
I very much agree about the rhetoric. The homepage is polished, and speaks to a bunch of features that are possible to varying extents in varying environments, but not easy to demonstrate. Bandwidth savings is a good example of this.
This seems like the thing they should lead with: what are those users doing and why would be a great way to explain the value.
There is an option in there to seed visited content as well as to host your own stuff.
https://macwright.org/2017/07/20/decentralize-your-website.h...
Hasnt written about it again though.
This is exactly what the http://metacurrency.org project aims to bring to life: http://holo.host/learn/ and http://ceptr.org
The distributed pattern they have created combines Git local source chains with cryptographic signing and BitTorrent’s DHT to create easily evolvable apps using http://holochain.org. Peer validation enforces rules defined in the logic of each app. Apps run locally instead of on a corporate server. If the current tech ecosystem is Platform Capitalism, then this pattern is Protocol Cooperativism.
I haven’t come across anything like this.
Scuttlebutt has been awesome for me to see that a social network can be completely hosted by it’s users.
I think Holochain and Ceptr are here to help us evolve how we work together and do our accounting.
What I love about Holochain's pattern is that it essentially provides a fully distributed, immutable and peer-to-peer crypto accounting framework for barter transactions/sharing work and resources. It does not focus on creating and managing artificially scarce coins (as is the case in the current financial system, and also in Blockchain). It is trying to show us that we are using Industrial Age tools (interest-bearing-debt national currencies), and that instead we could focus on creating peer-to-peer Information Age currencies [1].
Holochain's pattern enables communities to make their own judgements on what value something holds and how they want to account for it. It allows us to build that into a flow language - a currency. This means instead of using a fiat 'medium of exchange' backed by US imperialism and it's military might/control [2] - we create, redeem and destroy currencies based on asset-backed mutual-credit principles that mimic the way Mother Nature works. I think I read somewhere that today over 90% of money in the money system is speculative, and interest-bearing monopoly debt-money has no roots in the real world.
An example of a mutual credit currency could be a currency for seats on a train. When I buy a ticket, my ticket is tied to a certain seat on the train. If I went to Mars, my ticket would be worthless. The ticket is only valuable because the ticket is tied to the train system - to one of the seats available. This makes it asset-backed. Another example would be farmers being able to pre-sell their produce locally, without going through expensive middlemen. They can create currencies that represent their produce (the asset).
The MetaCurrency Project is bringing us a into a world where we have more agency when interacting with the world, using evolve-able peer to peer protocols that can be forked and re-forked, again and again, to better meet the needs of it's users.
They are giving us the tools to see the world with new eyes, and to allow multiple perspectives - instead of one rigid, centrally enforced truth.
Moving towards using mutual credit systems, together with reputation currencies (imagine your ratings on Uber, eBay etc. was hosted by your peer-to-peer community), allows you to build trust together in a way, un-intermediated or hindered by a corporate app and UI. These reputation currencies already exist, yet are contained and held hostage by our current paradigm. Other examples of reputation currencies I can think of are ‘FairTrade’ or ‘Organic’ labeling.
Holochain is BitTorrent + Git + Cryptographic Signatures + Peer Validation + Gossip [3] - making it agent centric instead of data centric. It is not anymore about the database in the middle, but about the relationships between peers and their balances/interactions, and the agreements we make together on what actions we are allowed to take in our different apps and networks. Together we can see and improve the app logic and I can bridge between my apps, as I now have complete control over my own data [4].
HoloREA is a project that is using Holochain to build a modular, cooperative and transparent economic supply chain framework. It uses the Resource Event Agent accounting model to enable users to steward and follow flows of economic actions. These relationships are today tightly controlled in corporations. HoloREA aims to create a cooperative web of interactive flows of people, actions and resources.
Essentially, the vision is to reclaim The Commons and give us better tools to steward the world together more peacefully and compassionately.
If you’re interested in this paradigm, a few keywords I've found useful: Protocol / Open Cooperativism (currently we have Platform / Surveillance Capitalism), mutual credit, myth of barter, buddhist economics, social darwinism.
I also like the work of Elinor Ostrom, Bernard Lietaer, David Graeber, Douglas Rushkoff (Life Inc.), Charles Eisenstein, E.F. Schumacher, Silvia Federici, Mariana Mazzucato and Anand Giridharadas.
For technical material: http://ceptr.org
Footnotes/links: [1] https://prezi.com/xmzld_-wayho/new-economy-new-wealth/
[2] “Nixon floated the dollar in order to pay for the cost of a war in which, during the period of 1970–1972 alone, he ordered more than four million tons of explosives and incendiaries dropped on cities and villages across Indochina—causing one senator to dub him “the greatest bomber of all time.”7 The debt crisis was a direct result of the need to pay for the bombs, or, to be more precise, the vast military infrastructure required to deliver them. This was what was causing such an enormous strain on the U.S. gold reserves. Many hold that by floating the dollar, Nixon converted the U.S. currency into pure “fiat money”—mere pieces of paper, intrinsically worthless, that were treated as money only because the United States government insisted that they should be. In that case, one could well argue that U.S. military power was now the only thing backing up the currency. In a certain sense this is true, but the notion of “fiat money” assumes that money really “was” gold in the first place. Really we are dealing with another variation of credit money.” — David Graeber: ‘Debt: The First 5000 Years’
[3] https://holo.host/faq/what-is-holochain/
[4] https://medium.com/holochain/holochain-reinventing-applicati...
What if I don't want a train ticket? What if I want to sell my widget-coin for some brussel-sprouts-coin. Now I can't go to any farmer because he's only got carrot-coin or broccoli-coin, I have to find the brussel sprouts farmer-and not only that, but the brussel sprouts farmer who wants to buy widgets.
And if you say "nonsense, you can exchange any coin for any other coin", how is that different from what we have now?
The MetaCurrency Projects’ Holochain is the next generation P2P no-middle-man accounting system which open sources and decentralizes the rules around money creation and usage. It allows us to evolve the rules together and build mutual credit, interoperable, open and unencloseable carrier’ currencies. Currencies are backed by real world commodities, leading to a rich network of flows between a community.
Your comparison shows me that you might not yet have had the privilege of critically exploring the underlying patterns and stories that have made Industrial Age money the tool it is today. For me ‘The Future of Money’ by Bernard Lietaer, and David Graeber’s ‘Debt: The First 5000 Years’ are very paradigm shifting books and helped me to challenge the underlying stories of economics that I believed. These stories stopped me from seeing a more beautiful future.
Because it’s a lot easier to raise $300 million on a bit of hype selling your own coin.
(Also Bitcoin micropayments aren’t quite there yet, but I’m not sure any other truly decentralized cryptocurrency really is either)
IPFS is a great idea but it is very hard to implement something like this. It was tried before (hello Wuala) but no success. I am not sure if this idea can be implemented at all.
Filecoin's promise is to be tightly coupled with file storage, to set up contracts that are automatically enforced by the network. Bitcoin does not do this.
I don't know much about it, but maybe Ethereum could be used to do that, though
Maybe the "proof" section of the FAQ will explain the difference: https://filecoin.io/faqs#what-s-the-difference-between-all-t...
If you've already read that FAQ and still disagree, are you saying you can embed the incentives to store terabytes of others' data and also the proof of that storage into existing Bitcoin blocks?
(I'm not saying IPFS "proof-of-storage" scheme will work. I'm just saying Bitcoin's proof-of-(cpu)-work doesn't seem to easily map to verification that the peers' harddrives actually has the data they claim to have stored.)
The same principle goes for Namecoin, incidentally. The token embodies the value of the resource. I actually think this kind of thing is a much more stable base for a currency than Bitcoin's "it's valuable because we say so" system.
To the degree that bitcoin is valuable today, it's because speculators think somebody else will pay them more for it in the future.
True decentralization at this level is why it is valuable
If anything, it means that economies of scale will favor larger organizations. In fact, that's what we see, with a few large Chinese mining syndicates controlling a majority of the hashrate: https://www.buybitcoinworldwide.com/mining/china/
Compare bitcoin to a real decentralized system like bittorrent, or hell, email.
We need P2P stuff that works.
Like https://github.com/webtorrent/webtorrent and https://github.com/amark/gun
They're both run in production, at scale (millions of users), and do NOT require any tokens.
You're looking at it from a pure technical perspective of pushing bytes around in a decentralized way.
What the folks pushing "tokens" are trying to solve is the game theory of financial incentives to store & serve those decentralized bytes. In contrast, things like Bittorrent/Beaker/webtorrent/etc depend on others' "altruism" to host and serve files.
Because altruism doesn't scale, that's why nobody wants to seed my 100 gigabytes of personal vacation photos. Sure, they'll be happy to seed & disseminate the latest cracked copy of Adobe Photoshop or a bluray rip of the latest Marvel Avengers movie. But my personal files are uninteresting to the current decentralized web.
(But I'm not claiming Filecoin actually solves the incentive puzzle. I'm merely pointing out that the "problem" Filecoin tries to solve is at a higher abstraction level (the economics) than webtorrent (the protocol).)
You must solve the technical scaling problem first, then sure, heck, add tokens if you dandy.
WebTorrent/GUN/etc. do scale. Add economics to that.
Preferably, add something that is time-scarce so people do not have to lose money (they don't pay FB or Google! If they have to pay Filecoin, they'll still choose free FB), something like BAT or Pirate Booty ( https://hackernoon.com/hollywood-crypto-behavioral-economics... ).
Consider what it would take to replace DNS with a distributed system. In a global namespace, names have value. You can't just operate on first-come first-served - there needs to be a system that ensures that names go to whoever wants it most. In other words, the challenge to be solved there isn't the technical one of having a DHT - that's a mostly solved problem - it's how the hell to make the names cost money, and who the money goes to. (And if you don't think the names should cost money, how do you propose allocation should work?)
Incentives don't exist on a one dimensional scale from pure greed to pure altruism.
People have lots of contextually dependent reasons for storing or sharing information. Those reasons are dynamic and super diverse.
In your case, why would you need lots of people to seed your 100 GB of personal vacation photos? You can seed them just fine! But don't use bittorrent, use the right tool for the job...
As a matter of fact, for the past 4 years I've been using btsync/resilio/syncthing to backup my personal media across different devices, as well as 70GB of audio projects. It's been working great.
That seems like a relatively small scale. And I'll bet some of those photos would be interesting enough to family and friends that they'd seed them.
From my naive perspective, the information itself is much more raw of a "currency" than any abstract economy built on top of it. Its "value" is determined by its relevance to one or many people, and p2p sharing, I believe, has the ability to accommodate this across many scales.
Why did public trackers fail? It wasn't because people didn't want to seed the content, it's because seeding the content became legally dangerous (and the trackers got shut down). To me, that's not a failure of "altruism", it's a failure caused by greed and an inability of industry to adapt.
And also by the reality that the tools are prototype level. Bittorrent and bitcoin are two incredible examples of applied cryptography and network science. They've showed us what's possible, but they're only the tip of the iceberg.
For many applications, lack of commodification is a feature, not a bug. What's holding us back is that the tools aren't built yet, but that's a work in progress!
I think you're missing some context for the motivation of Filecoin. One of the elevator pitches is that it can disrupt cloud storage like Amazon S3.[1][2]
People do put personal files (e.g. via a cloud backup service) on cloud storage like AWS S3. Instead of a paying a centralized Amazon, Filecoin claims they have a way for money to go to the decentralized owners of harddrives.
Therefore, if "backing up my personal vacation photos" to Amazon S3 is a use case, that means the decentralized peers hosting my uninteresting personal files is also supposed to be possible.
>But don't use bittorrent, use the right tool for the job...
Yes, exactly. That's what my reply to gp was explaining: his suggestion of webtorrent/bittorrent/gun is a protocol that solves a different problem than the one Filecoin tries to solve.
[1] https://www.google.com/search?q=filecoin+amazon+aws+s3
[2] deep link of Juan Benet of Filecoin mentioning the centralized cloud services: https://youtu.be/6h2WNxEV8q4?t=482
was to sell tokens. Take an existing idea, build an incomplete implementation with a token attached, talk about features that don't exist as if they do - ICO success!
By design, the Filecoin Network stores data on Filecoin nodes — not IPFS nodes. While we want to make these two networks even more interoperable (https://github.com/filecoin-project/specs/issues/143) — so that, for example, you could choose to supplement IPFS nodes with Filecoin nodes — we want to leave the choice up to users. There is a big opportunity to have a distributed marketplace for ensuring IPFS persistence, and multiple solutions optimizing for different use cases will likely coexist.
As far as I can remember that's one of the features of https://beakerbrowser.com/
Bitcoin manages low 6 figure transactions per day, using as much energy as the entire country of New Zealand to do so. That’s far from internet scale (or even “single Raspberry Pi”-scale) and, because it was designed around gold bug economics, the protocol says it can never improve because the whole point was to scale the overhead as a function of network size.
Fundamentally, the reason why none of these have worked comes down to economics: if you’re storing things, you want provable access times and reliability but the costs of doing so in a decentralized system are much higher than in a centralized model because you need more copies to balance out the lack of trust. That also hurts on the other side of this: if you’re considering hosting, you need to charge more to cover your risk of getting involved in legal proceedings and there’s a fairly high threshold where that risk just isn’t worth it.
I have also in more context spoken recently with someone and convinced them why they should use gaia over IPFS: https://forum.blockstack.org/t/cannot-find-ipfs-driver/6147
and they brought up some other usability issues not mentioned in this critique, particularly about file pinning to keep files alive, and in general the waste behind the idea of data duplication by not separating out the auth of the user owned data with the duplication of the data itself in the DHT network, for which Blockstack has "Atlas".
While we still have a long road to go w/ IPFS usability, it’s worth pointing to what works today -- there are millions of end users benefiting from IPFS, 100Ks of libp2p nodes, we see PBs/mo of traffic in the infrastructure that we run, and millions of daily requests to our gateway. Look to fully decentralized applications and systems like OpenBazaar, Textile, Dtube which use IPFS and paving the way. New technology -- especially new technology that seeks to build new platforms from the ground up -- is hard and extensive to build, and to polish. We’re working on it, and though nowhere close to where we want to be yet, there’s a lot of utility already provided.
Re Filecoin incentive structure -- this comes from recognizing that computers are not all the same -- there is huge utility to be had in dedicated infrastructure running 24/7, well maintained, high performance spread out around the world. P2P systems of the past have failed to match the reliability, uptime, and performance levels of centralized systems. Cryptocurrencies enable the creation of Open Services (like Bitcoin) that run public infrastructure w/ high uptime and reliability guarantees. With Filecoin, we aim to bring that to file storage and distribution. More here: https://www.youtube.com/watch?v=6h2WNxEV8q4
From far away, DAT looks smaller and better documented (perhaps less ambitious, too?) Apparently the best IPFS overview is the 2015 paper [2] which looks pretty daunting and does not seem to cover any practical considerations.
1: https://docs.datproject.org/docs/faq#how-is-dat-different-th...
2: https://github.com/ipfs/papers/blob/master/ipfs-cap2pfs/ipfs...
Essentially, dat:// behaves like BitTorrent but the torrent data can change.
The only downside for both protocols I can think of is that the integration story outside the browser and CLI tools is very poor (there is no FFI/C lib Ic an bind my Rust app to)
Datrs development should be unblocked again soon, starting by moving the fs layer over to async IO. And then tackling the network layer.
Language choices in both have set off my "this is doomed to obscurity" alarm bells for precisely that reason. You don't write the reference implementation for a new Internet protocol—especially the core library for it, and especially a very complex one—in a language that can't easily be included in most other languages. So, probably C.
Dat in particular seems great but ain't no way I'm relying on a large JS project for anything I don't absolutely have to, on my own time, especially if it deals with my files.
Not to mention you can call into Go funcs over shared libs from C.
Rust is a great language definately, but not using it is a completely sane design choice. C has been a source of a number of security issues due to memory management, that type safe languages solve.
Not to mention, there are various parts of the IPFS/LibP2P stack that are being written in Rust by other teams.
So Swarm vs DAT vs IPFS.
- https://www.arweave.org/ - https://github.com/ArweaveTeam/arweave
IPFS is essentially run by a very well funded (300musd) private company.
For this reason alone I think DAT is the more likely to succeed. It seems hard to reconcile the longevity of a truly distributed protocol with the need of a private company to retain control.
For efficient file transfer protocol between peers, Bittorrent protocol have multiple independent implementations that is working right now. They should build on top of that. Instead, both DAT and IPFS try to implement their own protocol with dubious additional features. For IPFS, it even relies on traditional DNS. What are they thinking?
So IPFS might not theoretically rely on DNS, but it seems that it does practically rely on DNS if you actually want to use it.
I have a hunch that IPNS is just broken in its implementation (manifesting as "unusably slow") but I haven't yet had the spare time to investigate this theory..
From what I can tell, most "serious deployments" of DAT and IPFS are made by people directly using the loose underlying collections of libraries that implement each of them. These people often end up putting together application specific transport and discovery layers that work for their specific application.
Seriously. IPFS decided the standard URL format wasn't good enough so invented something worse, which was pretty funny/sad. I never saw reasons for it that made any sense.
See:
https://github.com/ipfs/ipfs/issues/227
[EDIT] here's the original, deeply "LOLWUT?" justification for it, quoted in this comment on another issue. Other justifications were given but this was the motivation. Oh man. Wow.
https://github.com/ipfs/go-ipfs/issues/1678#issuecomment-139...
[EDIT] farther down, same author as the quoted text in the above comment: "wish unix:// wasn't taken by unix sockets." Oh FFS, guys. Hahaha.
https://github.com/ipfs/specs/issues/29
> I would add that if TLS can be used without all the X.509 nonsense, and with our own choice of pubkeys, including using different signing public keys (not the DHE keys) in the same conn, then we can consider breaking our "not TLS pls" stance.
TLS fit their requirements all along, they just… decided to reinvent it instead of reading about how to use it.
That is news to me - I've thought they were a scrappy startup. The issues mentioned in the post like glitches in the docs are excusable for a project that relies on volunteers who prefer writing code to polishing docs, but if you have $300M in funding then wtf? Just, you know, hire good project management and docs people.
We're still paying down years of documentation debt but there has been quite a bit of progress:
* Expanded https://docs.ipfs.io/ with a concepts section. * Tutorials https://proto.school/#/ * A ton of work on libp2p specs (https://github.com/libp2p/specs/commits/master) along with a full time documentation writer.
tl;dr: I feel that IPFS is the smaller more tightly scoped project that fits better into the existing web ecosystem. Dat has its own browser and versioning and other stuff bundled in, and IPFS works with normal browsers (https://news.ycombinator.com/item?id=20162972) in a way aligned the web's graceful degradation principle.
Dat doesn't use immutable addressing (addresses stay the same when content changes) while IPFS does.
Dat at the lowest layers is stream-oriented, allowing stream-oriented services and applications that are near-real-time. IPFS is static blob/object oriented.
IPFS has a better developed "discovery" network at present (if you use Dat today you are typically in your own island whereas with IPFS you're part of "the" IPFS network). This is being worked on however.
I think it's possible to view Dat and IPFS as two different layers of a stack that can interoperate and each solve useful problems at their layer. For example, Dat has more UX focus and high-level abstractions making generic app development smooth and easy (an area IPFS is weaker - though https://medium.com/textileio has been working to make this much better), while IPFS has the benefit of global name-spacing and content-addressing primitives that enable deduplication across identical datasets and validate the content is what you asked for (used by tools like Qri to do dedup within a data commons: https://qri.io/faq/). I've seen demos of projects using both together, each for their unique strengths - but there's still a ways to go to make interop easy.
If you were looking for nice IPFS overviews, I'd recommend: - https://hackernoon.com/understanding-ipfs-in-depth-1-5-a-beg... - https://medium.com/textileio/whats-really-happening-when-you... - https://docs.ipfs.io/introduction/overview/ (see the concept guides for easier to parse explainers on CIDs, Pinning, etc)
It's clear that those who control the funds aren't always the best at judging tech talent. Big tech corporations have had years to settle down and build a reputation to attract top talent. Crypto companies tend to attract greed over talent and it shows in practially all projects.
Its improving for some projects but not others.
I don't think it's particularly novel in 2019 to say that a successful app is built off of the feedback of its users. Any app, regardless of intentions or engineering prowess, is going to struggle if it doesn't have a sizeable user base providing feedback (and devs who listen to that feedback).
Crypto people say this like it's an argument in their favour - whereas it actually means: even for its one and only genuine consumer use case, crypto is utterly trounced by conventional currency.
I can see an argument that enabling crime under oppressive regimes is moral. Or even enabling particular crimes that you feel shouldn't be crimes.
I have trouble seeing the legitimate use case for cryptocurrency in first world countries. If the main use case is crime, money laundering and speculation, it will either be squashed or remain niche.
Most people in the world don't live in a first world democracy. Today, many of these countries are stable and providing currency for their citizens. But ask a Venezuelan if they can see a reason why we might want an alternative to a state backed currency and I feel like the answer is self-evident.
Vitalik saw this use case and talked about how disheartened he was by the ICO boom because it was use cases like this that he believed in, so it isn't at all an afterthought.
The massive amount of cryptocurrency funny money they raised makes this problem worse, not better. Lots of money leads to too many cooks in the kitchen. Large organizations require immense discipline to avoid scope creep and runaway complexity growth. They need someone to say "no!" 95% of the time and relentlessly exterminate features and not allow things to be released unless they are ready and actually solve a problem.
Most large organizations don't have this, which is why most "enterprise" software is a mass of twine and chewing gum. In the case of enterprise over-engineered bloatware you usually have some corporate requirement forcing its use. No corporate requirement forces IPFS or any of Protocol Labs' other sprawling mega-projects to be used, so nobody is going to use them.
We even created a plugin (https://github.com/almonit/almonit-plugin - unreleased officially yet!) for websites that combine IPFS with ENS (a decentralized DNS). Our repository contains a list of decentralized websites using this method. We found about 20 so far.
Technically, IPFS work well for us, but we made sure to have a server seeding our website at all times (where we suffer similar problems like the author of the post describes).
Even then, we still had to make sure that the website is available in all main gateways, since most people don't run their own IPFS daemon. The strange part is that sometimes content is available in one gateway, but not in others. Or sometimes it's available in one gateway, but we can't get it on our local IPFS node.
I understand that IPFS is not a blockchain, so I can't expect all the nodes to have the same content. But I do expect the main gateways to communicate more directly with each other.
Conceptually, IPSF websites are a bit like sending your website to someone via an unreliable slow mail; i.e., it's not that attractive. You can make it somehow more dynamic, using what they call IPNS (it allows you to update your content). But the result is so slow, that even the most devoted monk would lose his patience eventually.
A workaround is using a decentralized name system, like ENS.
This works very well, but the result are still static websites. No comments or anything really interesting happening.
Those websites are censorship-resistance and very robust, you don't have to worry about ddos attacks. But then again, how many people worry about such things?
That said, I still like the concept of IPFS. We are exploring a few options to add dynamic behavior to that now, where the dream is to mimic existing services in a decentralized way.
Surely, they won't work as well, but the pro would be that it will be controlled by the users, and that they will be able to survive financially with no ads.
> We even created a plugin (https://github.com/almonit/almonit-plugin - unreleased officially yet!) for websites that combine IPFS with ENS (a decentralized DNS).
This is great! We’ve been wanting something like this for a while-- and there are a bunch of utility in bringing ENS and IPFS together.
A lot of the problems you mention are known, and being worked on. Let me describe a bit more in each.
> Technically, IPFS work well for us, but we made sure to have a server seeding our website at all times (where we suffer similar problems like the author of the post describes).
The IPFS content model requires somebody with interest in the data to keep serving it. So for now, yes you need to keep some ipfs nodes with the content around. For example, we serve all our content (all our websites are distributed w/ ipfs) using ipfs-cluster, which connects to the gateways. We’re working on ipfs-cluster and filecoin as the ways to solve this. Ipfs-cluster for when you want to run your own infra (or a community gets together), and filecoin for when you want to hire someone else to serve it for you.
> Even then, we still had to make sure that the website is available in all main gateways, since most people don't run their own IPFS daemon. The strange part is that sometimes content is available in one gateway, but not in others. Or sometimes it's available in one gateway, but we can't get it on our local IPFS node.
This is a big problem that we’re working on right now. Many of our recent releases aim to fix / reduce this problem. The nature of this comes from content-routing scaling. We’ve detected this getting worse as the network grew orders of magnitude, and we’re working on fixing this right now.
> I understand that IPFS is not a blockchain, so I can't expect all the nodes to have the same content. But I do expect the main gateways to communicate more directly with each other.
Yes, definitely. We agree-- on it.
> Conceptually, IPSF websites are a bit like sending your website to someone via an unreliable slow mail; i.e., it's not that attractive. You can make it somehow more dynamic, using what they call IPNS (it allows you to update your content). But the result is so slow, that even the most devoted monk would lose his patience eventually.
IPNS key names are just not working well -- and getting worse with scaling. We’re working on fixing it, but lower prio than other more important problems. For now, we’ve been directing people to use DNS -- check out https://docs.ipfs.io/guides/concepts/dnslink/ -- these should be fast for you. But yeah, figure you know about this and may just be avoiding DNS in favor of more decentralized tools (ENS, IPNS key names).
Also check out https://dev.peerpad.net/ -- this is an experimental tool w/ dynamic content (give it 10-20s -- unfort takes a while to “get online” -- this is the dev pad). Once peers are connected you get a pad that’s distributable across ipfs nodes.
> A workaround is using a decentralized name system, like ENS. > This works very well, but the result are still static websites. No comments or anything really interesting happening.
+1 for ENS. And, you could use ENS to point to something like the content hash of peerpad above (see the latest content via `ipfs dns dev.peerpad.net` or `dig TXT _dnslink.dev.peerpad.net` == /ipfs/QmWbsqqqG9YpNYDt5afp6HY8TrKMtCtdGUtUfgkS9fRYeH -- and this would be a _static_ html5 bundle that gives you a _dynamic_ local app. The hard part is backing up your content beyond your own browser-- pinning it to an ipfs-cluster or other tools, etc.
Anyway, the picture is clearly not there and usable yet, but the implications are big: you can get fully distributable p2p apps w/ fully dynamic content. (you could even encrypt the app bundles themselves, but this needs an extension to decrypt browser side) -- this feels like this: http://www.accademia.org/wp-content/uploads/2014/01/slaves-b... -- the form is emerging, but much work to be done.
> Those websites are censorship-resistance and very robust, you don't have to worry about ddos attacks. But then again, how many people worry about such things?
A lot.
* https://en.wikipedia.org/wiki/Block_of_Wikipedia_in_Turkey and https://ipfs.io/ipfs/QmT5NvUtoM5nWFfrQdVrFtvGfKFmG7AHE8P34is...
* https://en.wikipedia.org/wiki/Censorship_of_Wikipedia#China
* https://firstmonday.org/ojs/index.php/fm/article/view/9402/7...
> That said, I still like the concept of IPFS. We are exploring a few options to add dynamic behavior to that now, where the dream is to mimic existing services in a decentralized way. > Surely, they won't work as well, but the pro would be that it will be controlled by the users, and that they will be able to survive financially with no ads.
It takes a long while for all of this kind of tech to come together. Keep at it-- you make significant progress YoY, and it adds up. What we can do now in 2019 is vastly superior to what we could do in 2015. Big innovation leaps take significant time to develop -- but the good news is _a lot_ changes decade over decade:
* https://en.wikipedia.org/wiki/History_of_the_Internet
* https://en.wikipedia.org/wiki/History_of_hypertext
* https://en.wikipedia.org/wiki/History_of_personal_computers
* https://en.wikipedia.org/wiki/History_of_operating_systems
* https://en.wikipedia.org/wiki/Cryptocurrency#History
* https://en.wikipedia.org/wiki/History_of_self-driving_cars
Peerpad looks awesome, and is exactly in the direction of stuff we're interested in. Ironically, a friend sent me its link literally 5 minutes before I saw your comment. Great minds think alike.
We've been playing a bit adding a p2p network in the background of static ipfs website (using webRTC). That way if a website has enough visitors, they could mimic to each other many functions that normally a server does. I did similar things in previous projects I was involved in. But there it was native softwares, where here there are extra challenges caused by the (relatively) limitations of web technology.
It may have failed at adoption, but the problem of decentralizing DNS has been solved.
I am unconvinced that a solution nobody uses counts as a solution to a problem anyone has.
It might just never happen. Until then the "worse is better" rule applies.
I have used ncdns in conjunction with dnsmasq in the past and I am happy that there is a solution out there that works just in case the other fails.
If the very unlikely case occurs and bitcoin stops being a thing there is always the possibility to change the POW algorithm. Until then Namecoin is a solution that works ultra reliably since years and most importantly unlike other solutions it works today.
In the mean time merged mining is the best compromise between rock solid chain security and environmental impact.
Do you assume that if Namecoin were to become super popular, the price of its coin would become very high? I don't see why. Can you explain this assumption?
Namecoins are not Bitcoins, they're not supposed to be used as a currency, so the logic that the popularity of Namecoin would create a huge raise in its price is not straightforward.
If blockchain technology were to become popular and widely used by ordinary people, its already worrying environmental impact would skyrocket, assuming the technology is capable of scaling at all. We need to bring the energy consumption of blockchain technology down to a manageable level, and that means abandoning proof-of-work as far as I can tell.
I keep telling my friends hyped by bitcoin that I do not believe in its future as a currency for real daily exchanges because of this energy consumption problem.
Is there any serious track to address this issue?
Unlike most technologies, I do not see the traditional efficiency gains when the technology gets more mature. It seems inherent to proof-of-work and cannot be improved in this paradigm
Assuming a solution is found, is it possible to "update" Bitcoin?
At a larger societal level, we should definitely be worried about tackling emissions for climate change. For this, very few people are advocating getting rid of energy consumption in our societies. Yes, in the short term, people recommend being mindful of the emissions impact of your lifestyle (e.g. a flight has a large carbon footprint) but in the longer scheme of humanity's development we'll always have greater and increasing energy needs.
With that in mind, a lot of focus has been on using non-emissions producing energy (solar, wind, nuclear, etc.) to mitigate climate change. Seen in this context, bitcoin is agnostic to the source of the energy, and its Proof-Of-Work protocol is fine.
The future for ethereum is proof-of-stake. This project is called Casper and will be in Ethereum 2.0. For Bitcoin most transactions will be off-loaded to layer-2 networks but PoW will remain on the main chain.
and their concept of gaia hubs is the association of immutable identities with user owned storage without the need for IPFS pinning or otherwise data duplication: docs.blockstack.org
You can see the most recent conversation about it here: https://forum.blockstack.org/t/cannot-find-ipfs-driver/6147 in which I convinced someone looking at IPFs to use gaia instead
full disclosure, I am an Engineer at Blockstack.
What I'm always wondering about: Someone needs to foot the server/bandwidth bills. Who? If you run a Pi on your home network and serve requests with your 5MB/s upload: That's fine. More power to you. We need more of that. (and in that case, you are paying your ISP)
But the transformation envisioned and the bandwidth and compute required doesn't come for free. Someone's gotta pay. In things like dollars. How can this work?
I'm puzzled by your comment because in the second paragraph you provide the answer to the question your third paragraph is about. If people are paying their ISPs for bandwidth in things like dollars, what's the problem? It's true that bandwidth is more expensive when you buy it retail from ISPs rather than wholesale in data centers, but only by a factor of two or three, so this is only an issue for the most bandwidth-intensive applications, such as Netflix streaming.
Bandwidth and computation are extremely cheap now, so I don't think this is really an issue. As a point of comparison, when I started browsing the web in a graphical web browser in 1993, I was running it on a ≈100-MIPS RS/6000 with a couple of dozen other people. There were nearly a thousand web servers (in the world, not on the RS/6000), but each one could only serve about one or two hits per second, mostly on machines similar to that one. Shortly after that, my university, with a few thousand students, upgraded its shared connection to 45 megabits per second, as part of a contract to manage a supercomputer center in another state.
My cellphone runs about 10,000 MIPS, and would have no trouble handling 5000 hits per second; this is more than the entire WWW at the time and well into 1994. Most homes in rich countries have more individual bandwidth than I was sharing with hundreds of other active users at the time.
We don't have a shortage of bandwidth or computation. We have a shortage of community, a shortage of innovation, a shortage of cooperation, a shortage of imagination, and a shortage of freedom. But bandwidth and computation are abundant in a way that was unimaginable in the days when we built the WWW.
One ugly hack to get around one problem (IPv4 address scarcity) has single-handedly transformed the structure of the Internet from a mesh to a top-down monopoly-driven medium. NAT is like literally Satan.
It wouldn't be quite as evil if it weren't so often symmetric, but for some odd reason symmetric is what many vendors implement. I can't for the life of me understand why symmetric NAT exists when the same scalability can be achieved with port restricted cone NAT that falls back to symmetric-like behavior if port preservation is not possible due to resource exhaustion. That would yield working P2P >90% of the time instead of <5% of the time.
NAT is merely a tool. Using it to restrict user activity is bad. Using it on your home network to preserve your privacy (ie how many devices you have and what each is doing) is good. There are also other use cases (both good and bad) that I'm omitting here.
NAT doesn't help privacy either. There are a million plus one ways to fingerprint or track a web browser without knowing anything about the end user's IP. A modern browser presents a ton of surface area. It's also quite easy to track by externally visible IP or IP prefix (e.g. /24 or /64 in IPv6) if you make certain nearly-always-valid assumptions about the timing and sequential nature of user behavior.
I think a lot of people just don't grasp how easy tracking actually is. Think of how obscure and sophisticated hardware-level attacks have become: Spectre, RAMBleed, etc. Now imagine the surface area presented by a browser. It can and does get that clever.
The only way to really prevent tracking is to use a sandboxed browser with strong anti-tracking features and redirect your traffic... or use a P2P decentralized system!
If we didn't have NAT, we'd probably have much more privacy-respecting P2P alternatives to the centralized web and we'd also have easy to use P2P systems for anonymizing requests by bouncing them off other peers. In other words without NAT our privacy tech would be better.
I'll be more accurate than saying NAT is Satan. NAT is a massive piece of technical debt. It's a "simple" hack that breaks something fundamental about the Internet, namely namespace unity. That in turn makes a ton of other stuff exponentially more difficult.
Well at the ISP level, yes, absolutely!
Regarding privacy, your response seems very focused on web browsing. I agree that NAT plays no role there - it's at a lower level. I was referring only to the potential for masking devices - with NAT, there's no way to tell how many (or how few) devices are producing a given stream of traffic. To the extent possible, I'd rather external observers (particularly my ISP) not be able to tell what's going on inside my network or how it's configured.
In case this isn't making sense, imagine a scenario in which your ISP equates simultaneously active IP addresses to number of active devices and then adopts a fee structure based on this. Or perhaps just tries to profile the types of devices that are active on your network in order to sell that data to third parties. For example, identifying how many Alexas or smart locks or whatevers that you have, correlating that with how many devices are streaming or browsing, correlating this with customer demographics, and so on.
NAT is a useful tool for engaging in namespace shenanigans - consider NAT-based load balancing, for example. It can also be used for ill, and I agree that the current state of affairs is unfortunate.
I also still think you're not fully enlightened as to just how easy tracking can be. It's not just web browsers. Every single distinguishable characteristic of a client forms one bit in a hash that can be used to track it, and thus distinguishing precision is 2^N where N is the number of bits of information that can be gathered.
Your IPv4 /24 already provides 24 almost always unique bits, so that's a good start for any tracker. Now start correlating /24's over time using clustering algorithms. Now start TCP fingerprinting, keeping track of pinned certs, measuring anything and everything that can be measured about a client. Pretty soon you're up to something like 32 bits which is one in four billion.
I use web browsers as an example because they're just embarrassingly easy to fingerprint.
Fair enough, I don't know enough about usage of different NAT types to debate such things (and never intended to).
Regarding tracking, I'm still not sure that we're talking about the same thing here. Are you saying that fingerprinting could be used to accurately extract per-device data from the aggregated stream? That is, if an entire network is hosted behind a single external address via NAT, are you suggesting that the carrier could reconstruct the separate fingerprints from the aggregate data stream that they have access to?
If so, that would be news to me. Obviously they can make some educated guesses (a single device probably isn't originating simultaneous Netflix and YouTube video streams), but I'm assuming that aggregated (TLS encrypted) data streams are going to be fairly difficult for an external observer to tease apart.
We're pushing hard on getting libp2p relays up and running to help get through this. The idea is that we can use a relay to make the initial connection, then ask the remote peer to try dialing back (assuming that both peers involved arent undialable).
P2P systems historically have dealt with the problem altruistically, or with limited tit-for-tat, which both work in many cases, but have so far failed to work for large scale long-term resilient systems.
This is where Bitcoin managed to do something remarkable: achieve high uptimes typical of the best centralized systems, through a very clever, but still open and permissionless, economic incentive structure. Markets have been shown to be extremely useful to create a robust Open Services. More on these ideas here: https://www.youtube.com/watch?v=IfLIoOr4p0A -- we think this kind of thing is going to lead to extensive, global, public utilities run w/ internet-native money.
But it will take a while-- this stuff is extremely difficult to build right now-- it feels similar in nature to very early Web, or pre-unix systems. Lots of hand-rolled primitives, many with the capacity to cause serious failure (not very old cryptography, and complex security questions). Perhaps better programming languages will help us build these systems dramatically faster/easier. For now though, you can see the entire blockchain space wrestling with these problems.
The author insisted on doing this on bare IPFS but I think this is (vaguely) analogous to building a website based on IP addresses and port numbers, not URIs. To the point: the semantics just aren't there.
I could imagine an IPFS based web site being built with a local URI resolution map as part of the bundle of objects that is the web page. The sub pages would refer to symbolic URIs like before but the browser would also download a site map that links URIs to actual hashes of the latest revision, and resolve references like "foo/bar.html" or "/root/foo.html" based on the map. Or a proxy could do this transparently, translating URI requests to hashes and fetching data directly from IPFS, then serving it back to the browser as if it was downloaded from "/root/foo.html" instead of "ipfs://50ad443758222efea0286f3a94db2c25".
The top-level entry to the web page would basically be this URI resolution map which, as content-addressable, would effectively refer to a single revision of the web page. This could be implemented as a separate URI scheme, like ipfs-uri://bb9f6cbcc28829b57dd25102f67b9d37/main/news.html where ipfs://bb9f6cbcc28829b57dd25102f67b9d37 would point to the URI map and the URI handler would resolve the relative URIs such as /main/news.html based on the offered mappings.
But all this does require extra lifting from the browser or a proxy. I don't think IPFS as designed is feasible to replace something like HTTP which was explicitly meant to work with URIs.
https://gateway.ipfs.io/ipfs/QmYXYP1J346Pic6UHV5AkUy1UgabtJV...
Normally that link would retrieve the content via a public IPFS gateway, but if you install one of the IPFS browser extensions you can have URLs like that automatically redirected to a local IPFS client.
As long as you stick with relative paths, it's not difficult to host a static website on IPFS. You can even configure a CNAME to point to a public gateway with special DNSLink TXT records so that visitors don't need to know the hash of the root directory. The end result looks just like a normal HTTP URL.[1][2]
[1] https://docs.ipfs.io/guides/concepts/dnslink/
[2] https://developers.cloudflare.com/distributed-web/ipfs-gatew...
> but I think this is (vaguely) analogous to building > a website based on IP addresses and port numbers, not URIs.
IPNS is a naming solution designed to address this:
https://docs.ipfs.io/guides/concepts/ipns/
It's not super fast right now, but there's some work happening now to make it much faster.
ENS, the Ethereum name system, is also an emerging way of doing this.
> basically be this URI resolution map
IPLD is a data model that works with IPFS to address the use-case you're describing, where you have a permanent reference to a mutable set of data: https://ipld.io/
From IPNS: You point the IPNS entry to something nonsensical and then by default it will be gone once the TTL of the old IPNS entry is reached (kind of like DNS).
There are "youtube competitors" that offload the high bandwith requirements via webtorrents running in javascript.
You can not host your full webpage with it but you can reduce bandwith costs drastically.
A good alternative to DNS is namecoin. It already works flawlessly and I honestly wonder why it has not been adopted more widely.
I would say that Namecoin appears to solve a problem that nobody actually has in practice.
I know advocates are pushing Ethereum Name Service, which does the same thing - but again, what are the lessons learned from nobody bothering with Namecoin?
(Namecoin is interesting, as it was literally the first altcoin. It hadn't occurred to anyone to fork the Bitcoin code and make their own coin until then.)
Check it out. Make up your own opinion:
This can be very useful in certain situations and projects! But it means that it is not a replacement for a centrally-hosted website... and the IPFS site does a very poor job of conveying this limitation.
(I can't take Filecoin seriously as a 'solution' here either - as far as I can tell, it suffers from the exact same problem as every 'blockchain-backed storage system' I've seen before... it's unable to reliably verify that some peer is actually storing data, without a local copy to verify against.)
Filecoin can actually do this. I'm planning on doing a blog post about how this works soon (in all that copius free time), but a good summary is here: https://github.com/filecoin-project/specs/issues/155
Is this an admission that IPFS is unworkable, and they don't have any prospects of making it usable in the next few years?
You can check out our roadmap
> IPFS is unworkable... making it usable in the next few years
Absolutely not.
The OP brings up a lot of great, useful feedback for us, and we'll respond to it.
But the OP is also simply wrong in saying it's "not usable". There are millions of end users benefiting from IPFS, 100Ks of libp2p nodes, we see PBs/mo of traffic in the infrastructure that we run, and millions of daily requests to our gateway. Look to fully decentralized applications and systems like OpenBazaar, Textile, Dtube, and others.
Beyond that, we're well aware of the many shortcomings, and working on them. We're unfortunately spread thin across a lot of projects (IPFS, libp2p, filecoin, ipfs-cluster, etc), but each is seeing significant growth and improvement.
You can see the long-term IPFS roadmap here https://github.com/ipfs/roadmap
P.s I think there is a typo at ‘ipfs pin add’ where it’s used twice instead of ‘ipfs pin add’ then ‘ipfs add’ I think?
It blows my mind that almost every decentralized system mentioned in the comments here relies on unsustainable proof-of-work systems. Blockchain technology is possible without proof-of-work, and decentralization is possible without blockchain. We can do better, folks.
Last I saw, the IPFS devs seemed to be pretty excited by a pub/sub mechanism they were building into the system; potentially to replace the workings of IPNS.
Is that a stable or useful alternative for indexing changing content like a blog? How decentralised is the pub/sub (i.e. do new subscribers need to contact the publisher, or are messages persisted for a time in the network)?
There is also some work to make IPNS work over PubSub independently from the DHT. That work is being tracked at https://github.com/ipfs/go-ipfs/issues/6447, and should significantly improve IPNS performance as well as add in features like allowing users other than the author to keep IPNS records alive (so no 1 day expiration issues).
As for the decentralized nature of pubsub it is essentially an opt-in system. Random people on the network are not holding or forwarding your messages as they would in the DHT. However, anyone who has subscribed to a topic will propagate messages for you to other subscribers. This means subscribers do not need to directly contact the original IPNS record publisher to get a record, but instead can get it from anyone who has it and is advertising that they do so.
(following the link to 2017 post)
>And even if I used the base tag to re-route that link so that it points http://localhost:8080/ipfs/QmR96iiRBEroYAe955my8cdHkK4kAu6SA... ... It wouldn’t work: the correct URL for that Recently post is, instead, http://localhost:8080/ipfs/QmTbJ6RSLZDmVYy8dgdoeQLCtKya7UrNT.... So IPFS content links are fully content-addressed. I suppose to make my site fully IPFS, I’d have to build each individual page and then construct a home page that linked to the generated hashes for those pages. That leaves an open question: how could two pages link to each other? Adding a link from one page to the other would change its hash, so wouldn’t it be impossible for pages to reference each other? This might be a lack-of-coffee problem on my part.
They've misunderstood that. Both links of these links would be valid. You would have all the pages be under http://localhost:8080/ipfs/QmR96iiRBEroYAe955my8cdHkK4kAu6SA... and refer to each other by relative links.
The author's point of relative links or a base tag being necessary for the site to work on the ipfs gateway URLs is valid though.
(back to new article)
>So, links don’t work. I posted an issue detailing this issue, and while I got an encouraging response that there’s a real solution planned, there’s no real solution. People use specific plugins just for IPFS, like this one for GatsbyJS, to get it to work. I ended up writing make-relative, a script that rewrites my built site to use relative links. This is where the story about IPFS being useful here and now for web developers breaks down a little. I’ve done enough HTML-mangling and path-resolution in my decade in industry that writing this script was straightforward. But the knowledge required to do it is not all that common, and I think this is where a majority of web developers would call it quits, because IPFS’s ‘website hosting’ story would look broken.
You can either use relative links, a base tag, or keep absolute links and only support your domain with dnslink (and eventually when the web gateways support the hash in the subdomain field, or extensions support the ipfs:// protocol, then your site will also work through that). I'm hoping for the hash-as-subdomain support which will make things simple.
>IPNS
Yeah, it's awkward. This is because of multiple reasons (zooko's triangle and hosting of the names) that can only be addressed with a centralized service (see DNSLink) or a blockchain. Thankfully there's the DNSLink option too. I wish the IPFS docs wouldn't push IPNS as much because it's not ready / it's not what most people want.
>That was an incorrect assumption. IPFS-based websites do update their DNS records every time that they update their website, so that they can avoid using IPNS, because IPNS is just too slow. This was a tough discovery, because it works against everything I know about DNS – a system that isn’t particularly designed to be fast or scriptable.
IPFS isn't for frequently changing websites. You're not going to host a forum or other dynamic site on IPFS. (Or if you do, it's going to work by being static content hosted on IPFS that contains some javascript that talks to some regular external HTTP(S) servers or maybe even a WebRTC swarm for all of the dynamic content.)
>Unfortunately, once I started getting this set up as a ‘pinning device’ [server], the fun stopped. I tried running ipfs pin add with the hash generated earlier from ipfs add -r, but it just ‘hung’ - outputting nothing at all. After a while I realized that, like ipfs pin add, IPFS doesn’t communicate very well when it’s having a problem. So I figured out how to turn logging information all the way up, and then… I was never able to get past a ‘got error on dial’ failure, despite trying all potential configurations of the IPFS daemon, enabling logging, upgrading to the newest version, and so on. There are about 63 similar issues in the tracker, 21 of which are marked as bugs.
I think the "got error on dial" errors are just it complaining that it can't connect to some random people in the p2p swarm. Like people who may have recently turned off their computers, etc. You can't expect to be able to connect to everyone in the p2p swarm. IPFS is just trying to connect to tons of people in the swarm until it finds people with the blocks that you're trying to pin, and it's reporting errors that it can't connect to some of the people in the swarm. The real problem is that it should be giving some kind of logs of how it's trying to find some content, it's successfully connected to a bunch of people, but none of them it's talked to yet have it. Also of course it should eventually make a connection back to your computer and successfully get those blocks. I'm not sure why that's not happening and that's disappointing it's not working. I really hope IPFS improves in things like this because the concept of IPFS seems great.
Using another part of crypto ecosystem (Infura)
It's still not straightforward.
BTW... A few weeks ago I saw on HN an automated tool to publish static website on IPFS. It was pretty sleek...
https://github.com/agentofuser/ipfs-deploy
Makes it super easy to push static sites on IPFS.
BTW as a proof of concept their website is already hosted on their own IPFS cluster.
https://redbot.org/?descend=True&uri=https://www.shiftnrg.or...
They’re using DNSLink so basically what we’re seeing is that using DNS and a major CDN makes your site reliable, which does not seem like a particularly novel advance.
> For journalistic integrity, this post hasn’t been edited or reviewed by anyone. Hence the typos.
I'm not native in English but, for me, it would seem to be nearly impossible to mix these two words even in a moment of carelessness. They mark greatly differing meanings and thus they practically live in different slots in my brain. Even a blind typo won't explain it because 'a' and 'e' are not too adjacent on a qwerty.
I would guess that those who have learned the language have a more logical, grammar based structure and write more deliberately.
We have our own native languages with their own idiosyncrasies and we don't make mistakes like "could of" or "you're/your" in those languages. There's just no excuse for those simple mistakes...
Greek has 5 very common and 1/2 rare spellings for the sound 'i' (as in 'kit'): ι, υ, η, οι, ει /υι,ηι
Imagine if people who can't tell you're/your and write 'could of' had to face this reality... They are lazy ignorant people and the other native speakers need to stop enabling them and making excuses.
<edit>
I'm not actually sure if 'ηι' exists, google is showering me with irrelevant results.. in any case, furthermore, there's
two ways to spell 'e' (as in bed): ε, αι
two ways to spell o: ο,ω
two ways to spell the av/ev sound: αβ/αυ , εβ/ευ
two ways to spell the af/ef sound: αφ/αυ , εφ/ευ
and more. Maybe the difference is that all this crap makes you actually pay some attention to the language if you don't want to embarrass yourself
But the next question to wonder is whether this is English specific or person specific?
Is it that native English speakers form their language primarily through phonetic patterns and translate back to spelling from there, or is that some people simply crunch spoken language and other people crunch written language in their heads?
I definitely start from letters and words, and I'm often lost at how to pronounce something because, in English, you can't deduce pronunciation from the written form. In many cases you just have to "know". But I'm not a native speaker so can't tell if this is just because of how my brain works or because I learned English later in lafe.
I can't backreference to my native language, Finnish, either. Finnish is spoken exactly as it's written. If you know how to say a word you know how to write it, and vice versa. So the phonetic and written forms do not differ.
If anyone native to English could verify that what they keep in their head is "written English", that'd be a valuable counter-point to suggest it just depends on the person.
Still, I think that the author's criticism is valid, since the IPFS project does seem to encourage the view that it's suitable for Web hosting, rather than just "a nice consequence". The author's struggle with IPNS, and recommendation that it be de-emphasised in the docs, is in line with my own experience.
EDIT: Not looking to argue about whether leetcode filters for good programmers.
EDIT 2: Self taught developers at random companies can be amazing, but for a company "evolving the web" and 300 Million in the bank, they have hired almost no nationally recognized experts, and theyre greater developer base is not made up of people with 20 years experience. But rather a bunch of developers who have been coding for 2-4 years.
It’s experience and capability that matters, not being tied to distinguished companies or overpriced fancy degrees.
At the end of the day, a good hire is always going to be a role of the dice. The interview process should stack the odds in your favour but there’s never any guarantees. Which is why good management is so important. Good management can bring excellence out of average engineers and identify the dead weight before they become toxic to the team.
I don't want to say they're bad at what they're doing, as I don't know how hard it is, but at least a better pinning implementation seems easy to do and relatively useful. Currently, to pin something, the command will block for hours/days or even for ever if the content is large and rare. If my torrent client worked that way, it just wouldn't get used.
From a largely outsiders perspective, it has felt like FileCoin was the thing they wanted to do. Either that, or they just don't have enough bandwidth to take on all of these projects.
Regardless, it doesn't feel to me as if IPFS has had the attention it deserves from ProtocolLabs. I love the idea of FileCoin, but I'd at least like ProtocolLabs to get one thing finished before they chase the next shiny project. Hard to imagine the "next internet" being created with such floaty attention spans.
The "canary in the coal mine" for me is that pinning API ticket, which has been open for years and amounts to "please put an async UI over pinning". If something that simple and useful doesn't get fixed quickly, I'm not optimistic about the future of the project.
It's too bad, because I think it's an extremely useful idea.
Sure, we could hack this together quickly, but we're trying hard to avoid adding technical debt at this point. Adding every feature requested would put us in a bad spot.
Prioritizing this over the countless other things people continually ask for is hard, but we hear you. We welcome pull requests, and since you needed this for the service you were building, helping us out here would be fantastic.
Maybe I'm in a small minority of people who are interested in this feature, but how do people pin things without it? Do they just `screen ipfs pin <hash>` and leave that terminal there for ever? Wouldn't it be in the project's best interest to make pinning work well?
Unfortunately, I don't know Go at all, or I would at least take a stab at this...
> in some cases, if I've pinned it at all.
What cases would those be? If the file is pinned it should always show up in an `ipfs pin ls $CID`.
In the short term, I assume your service is a server side app that is making requests to the ipfs node, right? You could set some background task in that application that just waits on the `ipfs pin add --progress` call and keeps track of the progress for that pin, that way, when a customer queries it, and its in progress, you can return that information. Agree this should be built into ipfs at this point, but that seems like a reasonable workaround for now.
That would be nice, but not really required.
> I assume your service is a server side app that is making requests to the ipfs node, right?
Yes, exactly.
> You could set some background task in that application that just waits on the `ipfs pin add --progress` call
I did that, there were problems with the fact that you can't have too many pinning calls waiting due to resource constraints. I can't just have 100k requests open and waiting, they time out, I need to restart the server, things happen. At that point, every single task needs to fire up again and make an API call to the server.
I did the same thing with batching, but then long-running pins (files not in the network) would block and new files that were on the network wouldn't work. This happens with IPFS Cluster too, I just don't have to write my own code to worry about it.
Believe me, I've spent lots of time on working around this, and the workaround can only be so good with the current system.
I mean for success you need to achieve ample storage capacity, strong financial incentives that strengthen the network, and scaling. Major issues here being that following the ICO they need to release something that matches what investors wanted to some degree (more focus on Filecoin) and that a lot of scaling/tooling needs to come from outside Protocol Labs. They are heavily relying on Ethereum to successfully scale in a way that is friendly to the Filecoin protocol and that is an issue that has been dragging out for at least the last two years on how to do it properly.
I’m a hiring manager for a pretty significant albeit non-FAANG IT firm and I’ve good and bad applicants from Google et al just like anywhere else. I’ve since come to the conclusion that many skilled engineers simply can’t be bothered to go through the hiring process that FAANG companies put you through just for a little prestige.
Plus in my experience, the different between skilled engineers is less than the different it makes hiring based on team fit. You can stick a dozen of the worlds most prestigious engineers in a room and still not produce anything better than if you put a dozen good engineers who are united in any a common vision and good team fit.
But don’t just take my word for it. Do a search on HN for comments from other managers - even ex-FAANG managers have, on occasions, echoed the same sentiment.
Your criteria are off.
Rather than wade into the self-taught vs pedigree debate, I want to point out that based on my experience working on the project, I think technical challenges are actually not the biggest challenges.
Much of the challenge has to do with process -- managing a huge issue tracker, tracking a giant project that lives in multiple repos with many interlocking components, benchmarking a system that is quite hard to benchmark, developing an effective release process, developing systematized ways to collect community feedback, etc.
IPFS needs solid product focus and team direction. From what I’ve seen, the folks at Protocol get this. That’s why they have started focusing on achievable priorities like hosting package managers. The technical problems are complicated but not insurmountable-- not beyond the scope of what many talented senior programmers can solve. I think IPFS would be improved by strong team leadership and process, not more whiz kids, and I see Protocol taking steps to insure this happens.
https://news.ycombinator.com/item?id=20162199
Direct link:
The author could have gone an step further and used ENS. IPFS+ENS is becoming a common duo. Web browsers can't do ipfs:// but they can do https://insertipfsgateway/ipfs/Qm... With ENS (.eth) domains can resolve to ipfs:// and Metamask picks those up and turns them into gateway links.
There's a list of .eth domains & other ENS+IPFS info here:
https://gateway.ipfs.io /ipfs/QmUJKsA1FQRo4rGKJ9WAXNnv6o6HXy4NoJiWH97gPZUddV/ens+ipfs/list-of_ENSIPFS-websites.html