Researchers Showcase Decentralized AI-Powered Torrent Search Engine | Hacker News Reader

Researchers Showcase Decentralized AI-Powered Torrent Search Engine

torrentfreak.com

Tribler founding professor here, AMA. repo stuff: https://github.com/Tribler/tribler/wiki#current-items-under-...

I operate several instances of magentico with 150m+ torrents (several years worth). Is there a way I could run Tribler on a server w/o UI?

You could try https://www.coveapp.info/. It has a superior DHT indexer built in. It is intended as a end user product tho.

Looks interesting. Why it is not popular after these decades?

More a general question, what is your strategy for working on topics/technologies that seem to be quite "counter-culture", internet, cyberspace from within an 'established/establishment' academic institution?

Our university is cycling distance from Amsterdam, everything is legal there.

But more seriously, it is indeed difficult to work on CyberPunk 2.0 topics within a university. Solid question! As an academic you critically rely on incoming grant money, so my strategy for past decades have been to discover what the real problems are before they become hype.

That means you need to need to run your research team as a startup. Very few academic thinkers. We discover what is hard using our deployment-first methodology [1]. We have 2.3 million downloads of our distributed system. Our "Bandwidth as a currency" used a ledger in aug 2007 [2]. That became a big hit in jan 2009 with Bitcoin. Gave us a few years headstart for funding on others. In 2012 we had Internet deployment of fully decentralised machine learning (SGD) [3]. This showed the need for a global web-of-trust.

[1] https://arxiv.org/pdf/2301.04508 [2] http://news.bbc.co.uk/2/hi/technology/6971904.stm [3] http://www.kl.csko.hu/projektek/msc_thesis.pdf

> In 2012 we had Internet deployment of fully decentralised machine learning (SGD)

Peer-to-peer training of the next big LLM when? :)

If you just want the decentralized torrent search there's also Magnetico [1] and Bitmagnet [2].

Tribler tries to be a whole lot more, including its own tor-like network for anonymity and this latest AI powered search to make it easier to find content you might not know by name

1: https://github.com/boramalper/magnetico

2: https://bitmagnet.io/

Magnetico is pioneering work. Very simple, very effective. However, with a single 1 Gbps VPS you can take it down. Simply spam at full speed and switch VPN reguarly. Same for Bitmagnet.

Usenet, email and now Mastodon show that you need a trust framework. Otherwise you open to abuse and spam. With Tribler we're hoping that after 3 generations of failures/learnings we now have operational decentralised trust. See our public thinking for 11 years on Github [1].

[1] https://github.com/Tribler/tribler/issues/31

Try https://www.coveapp.info/

Interesting stuff. The problem with decentralized search is that it is essentially an adversarial problem. A "friends" list or social graph would go a long way towards providing a trust framework to build decentralized search on, that way trust scores for friends can be propagated out and used to rank results.

yes! Spam is real and every stranger could be a spammer. Decentralised trust is the cornerstone of our work. Big topic 25 years ago around the Pagerank patent time. Now we are one of the last remaining academic teams on this topic.

We deployed a decentralised trust algorithm based on the interaction graph to 94k people, see [1]. Theoretical foundations are based on proving that the Harvard impossibility result against Sybil attacks made too strict assumptions, [2]. Leading to trust scores with resilience against fake identities: MeritRank [3]. This is being released in first version in this Tribler version. So hopefully this trust framework will help stop spammers a bit.

[1] https://research.tudelft.nl/files/89353583/1_s2.0_S138912862... [2] https://pure.tudelft.nl/ws/files/96914542/p1263.pdf [3] https://arxiv.org/pdf/2207.09950

Just read through the article, not sure I get what's the novel thing here. Any insights?

Torrent tracker websites that provide us with magnet links may be taken down. If there is a search function which can run locally on the computer, there is no vulnerable node in the decentralized network which could be targeted.

You still need to bootstrap and maintain DHT/torrent corpus state/data in a potentially adversarial environment, correct? Solving for durable storage seems to be a challenge to overcome (unless that itself is served as a mutable torrent? or append only immutable torrents?).

The adversarial environment may become friendly if the web of trust idea materializes. Otherwise yeah, the proposed search method would encounter issues.

Could you perhaps (at a cost) burn magnet links for each torrent into the Ethereum blockchain? This would enable any node to mount and scan the ledger in a way that would be challenging to prevent, considering the financial incentives for the network to persist. Someone else is operating the decentralized, durable storage system for you.

It's a hard problem to solve, see SciHub, Libgen, Anna's Archive, etc.

For sure that could be done. I guess the benefit of having an ML search engine is that it may be able to infer what you're looking for even though you can't exactly name it.

But yeah, you're right. Burning magnet links into Ethereum seems like it might work decently, never thought of that.

Reply on news.ycombinator.com