How we put IPFS in Brave (2021)
blog.ipfs.io
blog.ipfs.io
My impression of the IPFS project is that the goals are excellent, the core protocol is quite good however they like rewriting the higher level layers far too frequently (for example they have deprecated UnixFS which seems to be the most used format and they keep switching between JSON, Protocol Buffers and CBOR) and go-ipfs seems to be a pretty garbage codebase.
I have run a public gateway on 2GB of RAM. Later 4GB because it was subject to very heavy abuse, but it was perfectly possible. Perhaps it is a matter of knowing how to configure things and how to not inflict self-pain with wrong settings.
Maybe there is a Goldilocks value I could find, but I didn't really need IPFS running that much so I just removed it.
I love the idea of decentralized content-addressed storage and wish there were a more lightweight way to get there.
if you plan to host IPFS at home and meanwhile do things on the internet then IPFS isn’t for you
although it’d be a good excuse to upgrade your home network
Is it possible to limit the bandwidth and queue depth for IPFS?
I bet you could also lower its limit dynamically whenever web traffic is seem.
without p2p IPFS is nearly useless
Presumably one could also run IPFS (or Brave) in its own VM, container, or hardware server and to rate limit traffic in and out of it.
rate-limiting will only make the matters worse and again turn IPFS nearly useless
they haven't managed to solve these issues for 6 years now
1. IPFS's bandwidth is so low that it is unusable.
2. IPFS's bandwidth usage is so high that it makes the network unusable.
Going on six years now. You can use external tools (like "trickle") or your OS knobs.
https://eng.uber.com/how-we-saved-70k-cores-across-30-missio...
I suppose that's your answer! Simple misunderstanding.
1. if new packages are produced for a release of open source, could I see if there is a copy available via IPFS? No, because one can't predict how it would be chunked. So, one would have to download and then derive a content ID and one can only tell if it is available if the same chunking algorithm is available.
2. if I want to push a package or other binary, can I figure out if it is already available via IPFS? No, one can't.
Doing it like that has many advantages, like being able to verify hashes as small blocks are downloaded and not after downloading a huge file. Being able to de-duplicate data, being able to represent files, folders and any type of linked content-addressed data structure.
As long as your content is under 4MiB you can opt out of all this and have a content ID that is exactly the hash of the content.
How do you handle conflicts where two concurrent events occur at the same time? Who wins? I know timestamps are not reliable but I want last write wins behaviour and seamless merge. The paper leaves data layer conflict resolution to the reader. It does suggest sorting by CID. I added a timestamp field for conflict resolution.
After reading Merkle-DAGs meet CRDTs whitepaper I took a go to implement a MerkleClock. It's incomplete. I need to maintain the partial order of "occurs before".
https://github.com/samsquire/merkle-crdt
I also implemented part of the YATA algorithm yesterday. So I think I could merge the plain text merging functionality of that with the Merkle CRDT.
Thus, this priority represents the current depth (or height) of the DAG at each node. It is sort of a timestamp and you could use a timestamp, or whatever helps you sort. In the case of concurrent writes, the write with highest priority wins. If we have concurrent writes of same priority, then things are sorted by CID.
The idea here is that in general, a node that is lagging behind or not syncing would have a dag with less depth, therefore its writes would have less priority when they conflict with writes from others that have built deeper DAGs. But this is after all an implementation choice, and the fact that a DAG is deeper does not mean that the last write on a key happened "later".
I thought of using user indexes by order of connection or user last active time but if you're not worried of the security implications of wall clock and time skew.
Hyperhyperspace project has a previous field on the CRDT operations and can issue undo events to reverse operations.
I suspect you could have a time validator service that is a node that issues revocations if times are in the future. It wouldn't be on the read or write path and it's more to validate that times aren't in the future.
It doesn't handle conflicts at this time. If the text can be automerged it shall be.
IPFS is probably the best contender for Web3 right now and I hope it'll see more use before the crypto bros take over the term completely
Brave just took the step of forcing you to always have the extension installed instead of making it optional, basically
I've tried IPFS a time or two and always found it to be INCREDIBLY slow (even worse than tor) with ZERO content discoverability.
I'd argue this is worse than doing nothing. This gave Firefox the ability to say they care, and yet not deliver something meaningful.
What do you think firefox could have done to improve things?
And, as someone not well versed, is there any "killer demo" that uses IPFS currently?
Your collaborative photogrammetry can be combined with the open and free species identification API and my custom OpenStreetMap data extensions and KartaView/OpenStreetCam/OpenStreetView to get more photogrammetry location integration and more free crowdsourced open data to add to photogrammetry. A demo of Seadragon/Photosynth [1] inspired me to work on this.
[1] https://www.ted.com/talks/blaise_aguera_y_arcas_how_photosyn...
Reader view, tools for the visually impaired, and browser automation are actually useful and commonly used, so that definition win the title for me.
There are certainly useful distributed web tools (e.g. email, TOR, IRC, Matrix, self-hosting, bittorrent), but they're the opposite of recent trends towards monopoly.
The distributed meaning is absolutely poisoned by blockchain at this point.
Like, if we compare this to RESTful servers, there's no set definition but nearly everyone agrees it's verbs and paths over a hierarchical API sending JSON back and forth over HTTP[S].
It seems like most people can't agree on anything except using etherium as a backbone.
So calling something web3 doesn't seem to do a good job describing things like REST or like something like you wrote above.
It's not a technical spec.
Take it a step further: Firefox could allow websites to open sockets and toss arbitrary packets around, and choose not to. If that capacity were available then Javascript could be harnessed to support all sorts of protocols and services. They could even provide Javascript access to monitoring network access point availability and connectivity management.
Imagine then a single page app you could share as an attachment through $messageService and it has all the stuff built in to create ad-hoc real networks in large gatherings that provide data resiliency against the dropping of nodes. You could have the cellular network shut down, protestors arrested, their phones taken, and the data they gathered still retained so long as any node managed to exit the area or the network itself expanded beyond the area of contention.
- SMTP/IRC spamming using Web requests (Cross-protocol scripting, 2002) - https://www.eyeonsecurity.org/papers/Extended%20HTML%20Form%...
- Webpages that detect your router and leak your SSID (or worse) - Samy Kamkar "How I met your girlfriend" (2010), excerpt: https://www.youtube.com/watch?v=tRJMIMBVqFI
Web extensions should allow you to do normal sockets, many years ago I had a Chrome app (I still miss them) as my IRC client.
Unless everyone is ok going back to running random .exe files from emails, I guess.
Not since 2017 or whenever it was that Firefox dropped XUL extensions and replaced them with WebExtensions. The legacy XUL extensions could do much, much more and there was correspondingly much, much more malware in browser extensions.
A bit of a tangent, but I really cannot stress enough that if you're using Tor to be private/anonymous that you should never use anything other than the official Tor browser, you will stand out like a sore thumb.
There are very good security and privacy reasons that all browsers (not just Firefox) work extremely hard to prevent this from being possible.
Seems no more or less confusing or understandable than allowing access to microphone or camera.
A sizable portion of internet users think that the internet is what you get when you click the Facebook icon on your phone.
But they do at least understand what a microphone and webcam are for.
As Firefox does with self signed certificates and similar.
As for IPFS, the crypto-bros seem to already be winning for taking over that term and melding it as part of a layer in web3, just like they did for 'crypto' which that has become too late and that ship has long sailed.
Perhaps the reason why they are winning is because they keep building stuff like this [0] and existing companies are jumping on board with the term already? [1][2]
[1] https://developers.cloudflare.com/web3
[2] https://stripe.com/blog/expanding-global-payouts-with-crypto
It will start a local ipfs node in the background. Which is basically a local webserver.
This means, there are no needless duplicates and if an IPFS node in your local network holds the data, you don't have to go to the source.
I already don't store needless duplicates, and when I decide to store something, I don't have to go back to the source after I have stored it. That's the whole point.
How is IPFS helpful?
Has IPFS made anything happen that couldn't have happened without it?
What are its biggest accomplishments, outside of its own spread?
Someone just has to embed an illegal image or video in their webpage, and suddenly you're downloading and seeding it.