A site to share LLMs APIs via Tor | Hacker News Reader

A site to share LLMs APIs via Tor

neuroengine.ai

43 pointsaortega21 comments

In light of upcoming regulations on artificial intelligence (AI), I created neuroengine.ai, a website that shares large language models (LLMs) in the form of chatbots and APIs. The site is accessible through Tor hidden services, allowing users to access state-of-the-art uncensored AIs anonymously. Currently, there are two 65 billion parameter class LLMs and a couple smaller fine-tuned AI models available for free with no API limits, and no registration required.

Thank you for making this :)

I wish more services were made with Tor access in mind, and that such services would feature on HN more often!

Generative AIs are perfect for TOR, as the are low-bandwidth, high-latency services.

I should say that I cannot accept all the credits for doing the site, as it was made 90% by the AI itself (under my direction, I hope).

I disagree. There is no way to do rate limiting and generative AI is slow. This means the API is trivially denial of service attacked.

> it was made 90% by the AI itself

It shows.

Unrelated to the actual site — we’ve been testing a tuned version of airoboros-64b on a H100 cluster as a drop in replacement for Claude-100k and GPT4-32k. It’s performing rather well in the generation of text but the 2k context definitely shows, also the reasoning capabilities just as in LLaMA are suboptimal.

For instance, obtaining JSON structured data from freetext is a rather impossible task.

That said, for summarization of leq 2k token texts the model performs extremely well.

This is mostly due to its unfiltered nature, where there would be clear biases visible in models like falcon-40b-instruct or other LLaMA derivatives.

Interesting. Since I'm behind Tor I will probably pick my own https client for sending json requests, instead of trusting this python client.

How will you ban bad actors if there's no keys and no registration?

Sure, the API is a very simple json request.

Currently, registration is needed for the sharing of AIs, as you rightly mentioned, to prevent malicious individuals from distributing malware links. However, accessing the AIs does not necessitate registration. This is because, depending on your definition of "bad actors," users have limitations in terms of what they can presently accomplish with an AI, even in an uncensored state.

But they can stress your servers with tons of requests. The point of requiring registration is usually to prevent DoS, not to prevent the models from being used for certain purposes.

I can handle that with automatic throttling, unless you want to spend a lot of resources doing a malicious attack.

> unless you want to spend a lot of resources doing a malicious attack

I don't want to perform an attack at all. I just think you probably underestimate the resources that any random darknet hacker possesses.

You can easily limit the damage with a Captcha or similar. Anyway, it survived being in the from page of Hacker News for a couple hours, so I think it's fine.

Could you put a license on https://github.com/ortegaalfredo/neuroengine/blob/main/neuro... ? I'd love to include it in some code I'm writing but I can't if it doesn't have a clear license.

Added the 2-clause BSD license. Thank you for bringing it to my attention.

Is API documentation planned? i.e. I should be able to write a client library in Rust without ever looking at Python code.

Yes, it's coming tomorrow.

So where is it?

No only are there ethical and privacy concerns about how we use your data, we're also now completely untraceable!

There will always be a delicate equilibrium between freedom and security. This site provides a greater degree of freedom compared to other services; however, it still necessitates registration for the purpose of sharing AIs. If you employ it to disseminate malware links, you will promptly face a ban. Nevertheless, apart from potential throttling in the event of excessive usage, there are no other restrictions imposed.

Yeah but what do you do with the data people send you

Currently I do not store any logs, nor I have any use for them. But you should not have to trust my word.

I can guarantee that if you use TOR, I do not have your IP or any other info that can be used to identify any client.

Reply on news.ycombinator.com