Hacker News Reader

Comment by Prefinem | Hacker News Reader

>Why are their costs so high if they can just host magnet links and cache the site with Varnish or something similar?

Just from personal experience, but here is my view of it.

A good search engine for millions of torrents costs several hundred a month. Scrapers to keep torrent seeders / leechers up to date cost another fifty to hundred (depending on how up to date you want your torrents). Now add in a few website servers and a load balancer for another hundred or two. You are looking at three to five hundred a month. Doesn't seem like a lot, but this is just a smallish torrent site. Isn't large enough to generate ad revenue or have enough donators to keep running. Maintaining that by yourself soon get's expensive unless you have a good job with plenty of disposable income.

View on HN

Also, accepting donations and sponsorships opens you up to being sued for profitting from copyright infringement.

Bitcoin donations was how I handled it at the time. A little more anonymous especially if you pay for anonymous hosting with bitcoin.

IANAL but purposefully hiding the transactions may just open you up to other legal issues.

Good to know (think about). Luckily, the site is no longer in operation.

EDIT: clarification

> Scrapers to keep torrent seeders / leechers up to date cost another fifty to hundred (depending on how up to date you want your torrents).

This part could be eased if there was a protocol for search engines to subscribe to a push-based firehose of presence events on trackers, no?

Trackers already do a lot of work in keeping peers up to date. Adding a subscription system would probably overload many trackers and make running one of those extremely expensive. Granted, I would have loved a system like that to subscribe to new torrents that get announced to a tracker with peer updates every so often, but that sort of addon would be incredibly high resource intensive.

A better way IMO is to allow the DHT network to query a little bit more information and have arbitrary data attached to a torrent. BEP 44 [1] should really help with that and actually might be a start with truly decentralizing the whole system. With a large enough DHT node base, you don't need to run a tracker to get peer information. On top of that, if you created an immutable record for each torrent with the torrent information (or ran a lightweight torrent system to pull torrent information from a single peer) you can easily get the torrent information to store in the search engine, and then query the DHT for unique peers to get peer information. You can then create comments or ratings with mutable data signed by your server to keep track of good / bad torrents.

I built the first half of that (DHT Crawler essentially) but it was always the peer information that was a pain. It was easier to grab the information from a tracker

1: http://www.bittorrent.org/beps/bep_0044.html

Reply on news.ycombinator.com