Hacker News Reader

In my experience IPFS is only fast when you're using it through something like Cloudflare's IPFS proxy (which is basically a caching proxy). I haven't found IPFS to be actually usable any time I've tried running a node myself.

Is that unusual?

Interesting, didn't notice that before. I believed my downloads are too obscure to be cached by Cloudflare, but I may be wrong. I'll do a comparison once I fixed my server at home...

No.

The last time I tried it (maybe 9-12 months ago) it was a real resource hog (including choking the network) and really slow.

This has been the state of IPFS for years now. The protocol and client haven't been developed to where they can work reliably in a real P2P scenario. IPFS only appears to work because most requests go strait to one of the big gateways which already has the content cached.

Development of IPFS slowed to a crawl when most of Protocol Labs switched to working on Filecoin. It remains to be seen if either Filecoin or IPFS will ever be viable for their intended purpose.

I'm not sure you are right about this. The following link will stream big buck bunny directly from the network with full P2P support. It is not using a gateway node for the video content.

https://bbb.fu.io/

You can run the following in the console to see all your peers

    for await (const peer of await node.swarm.peers()) { console.log(peer.addr.toString()) }

The console currently get's spammed pretty hard because they haven't implemented filtering out plain ws (non-wss) peers if you are connecting https. But it will use webrtc to find peers, and find all wss nodes in the network.

For me at least the vast majority of the data is being downloaded from various servers under bootstrap.libp2p.io or preload.ipfs.io. These are clearly dedicated servers not true peers.

The contact info for these servers appears to be retrieved via the DNS rather than the DHT. This isn't surprising because browser clients can't participate in the DHT, but the DHT is currently one of the biggest weak points of IPFS. IPFS's DHT is so chatty that it consumes over a megabit of throughput just to maintain an idle node. Lookups are excruciatingly slow and often fail completely.

Sorta, a peer can advertise dns addresses, and the browser can participate in the dht as a client. The example above uses webrtc-star for browser to browser discovery.

    $ ipfs dht findpeer 12D3KooWJxNHY6zE1KnzFdBJrTMCbhCVNtSLvjp7qR5Wsp49DnUC
    /dnsaddr/ipfs.scalable.io
    /dns4/ipfs.scalable.io/udp/4001/quic
    /dns6/ipfs.scalable.io/tcp/443/wss
    /dns6/ipfs.scalable.io/udp/4001/quic
    /dns4/ipfs.scalable.io/tcp/443/wss

Took a good 10 seconds until the video started playing. Anything over a second is too long in terms of UX expectations.

IPFS is the engine of Filecoin, though. Filecoin just incentivizes IPFS nodes to store things. So further IPFS development should still be necessary from the perspective of making Filecoin development/adoption happen.

(I say this, and yet I know it isn't true in practice. But why isn't it true?)

Pump up Filecoin price. Sell Filecoin. Move to Fiji. Don't do any coding while in Fiji

Ha. I was super surprised that my dad(almost 70) mentioned IPFS to me the other day. It turns out the people he know are running or thinking of running IPFS nodes because of filecoin incentive. Apparently, the people don't want to miss another bitcoin opportunity.

filecoin is also promoting in China. https://mp.weixin.qq.com/s/T0Qt2CEP7QsA6Zy9Vzvdkw

Yes, this is still the case. For a file which is on exactly one other computer, you have about 45 seconds lookup time before the download can start.

Are you opening the required ports (4001, 8080)? I found it was like that too until I did opened the ports.

I have to agree. I've had cases where I had to pin data that was on the node already and IPFS just hung the pin for minutes, failed, and then the GC removed the data later on. This is just one of the issues I had, other things include IPFS not being able to discover content on nodes that were already connected to it, IPFS just failing all my queued pins at once for some reason, etc.

I regret ever building a pinning platform on IPFS. The worst thing is that, when I expressed my frustration on Twitter about an IPFS release completely breaking IPNS updates, the founder called me a Karen.

IPFS gateways are similar to Tor exit relays, they just route the traffic to the public internet. Tor exit relays still exist, and haven't been DCMAed out of existence so far.

One could also imagine IPFS over Tor.

You're almost right. The IPFS gateway nodes also cache the content. IE, they are both storing, hosting, and sharing to other IPFS nodes.

how can one validate that the cached version is indeed bit equal to the one stored within ipfs?

You hash it, everything is content addressed.

The address of the file is the hash of the file. (Really the hash of the DAG of the file)

So you could validate it if you wanted to. Because the URL _is_ the hash, if you did validate it you could be very sure that it is correct.

If a webserver hosts a file, how can you be sure that file is correct? Sometimes the ship a sums file next to it. At best this can find corruption, but it would not find malicious files.

IPFS-over-Tor has been trapped in development hell for 5 years.

https://github.com/ipfs/notes/issues/37

If IPFS-over-Tor is in the development hell, IPFS-over-I2P is in the development limbo.

https://github.com/ipfs/notes/issues/124

Tor exit nodes are forward proxies, they are used by the client-side to initiate connections to the public Internet but do not accept connections from the public Internet. You cannot host an Internet-facing webserver on a Tor exit (Onion Service exists solely within the Tor network), DMCA is not an issue, they do not act as a point of content distribution, there is nothing to take down. The main issue is the abusive outgoing traffic, which is seen as unavoidable and outweighed by Tor's greater benefits, so the existence of Tor exits is justified.

IPFS gateways (and Tor-to-Web) are reverse proxies, they are configured at the server-side to accept connections from the public Internet and route traffic to IPFS (and Tor), and they distribute content to the public Web. As far as I can see, this is a problem, someone can send DMCA notices, asking the gateway operators to take files down from the Web. The only protection I see is DMCA Section 512 (a.k.a Safe Harbor Provision), the operators are not liable for things solely hosted on IPFS, which is good, but gateways must comply with the takedown notices. So I think it's entirely possible for Cloudflare to put a huge blocklist on its IPFS gateway, and for smaller gateways that don't have the necessary resources to handle the requests, be DMCAed out of existence entirely. In the 2000s, RIAA launched campaigns against eD2k servers and BT trackers, including the use of honeypot servers to collect information. RIAA was not ultimately successful, but it created a major short-term disruption (eventually new servers would always appear in a different jurisdiction). A similar campaign against IPFS gateways sound possible if the publishers decided to launch an aggressive crackdown like the RIAA. The question is only a matter of whether the decision is made.

I'm not saying that IPFS requires the use of IPFS gateways, they don't, and in a sense, gateways are counterproductive for the decentralization goal. But they do currently provide an useful service for the Web, and I'm just speculating whether a major disruption is possible.

Thx segfaultbuserr for the detailed answer. I was wrong and I agree with you that IPFS gateways and Tor exits are two different things.

Well, then we might need something like IPFS-over-Tor, using Tors anonymity features to make it more safe for operators.

Am I understanding it right that DMCA takedown requests apply to a URL?

Maybe IPFS can introduce a feature to generate one-time URLs, e.g. `oneTimeURL = f(realURL);`.

Websites then can generate new URLs for example every hour or even for each client request. Copyright bots will file a DMCA request to a URL which only they have.

In general DMCA takedown requests reference a URL to locate infringing content, but the actual requests ask that the content itself be removed and not just the URL.

I believe this is similar logic to what MegaUpload used. i.e. They would allow users to easily create links to content, and if DMCA'd would invalidate the links but leave the content alone (and the users free to generate arbitrarily many more links to keep sharing copyrighted content).

It kind of worked for MegaUpload. Their founder got rich and a version of the company still exists. However, the founder is also gradually losing a long court battle and may find himself extradited to the US where he will face significant jail time.

They'll probably lobby the governments for more laws.

It's a politico-technological arms race. Governments make laws. People don't agree with them so they create technology that subverts the government. The state must then become even more tyrannical just to maintain the same level of power. We'll end up with either an uncontrollable population or a totalitarian government.

> send a list of IP addresses to Prentice Hall or Elsevier

I don’t know, do you think nobody has noticed LibGen by now? Do you think the reason they’re still running is because of some jurisdiction issue? Even the lawyers would be wrong on that one.

> do you think nobody has noticed LibGen by now? Do you think the reason they’re still running is because of some jurisdiction issue?

I'm not sure about the implication of your words. But, clearly, Elsevier has already tried to take legal actions against LibGen previously, and once in a while, LibGen's domain names are still being blocked and routinely replaced, just like Sci-Hub. I'd still say the reason it's still running is a jurisdiction issue, or let's say it's a geopolitical issue, it's using Russia to shield itself away from US influences.

But an IPFS gateway hosted in the United States by Cloudflare does not enjoy these legal and political benefits.

And what about when browsers (like Brave) start supporting IPFS natively, and gateways are not needed any more?

Browsers still don't support (better-established, though I'm not sure older offhand) TOR or bittorent yet, so I'm not terribly optimistic.