Self-Hosting on the Dark Web | Hacker News Reader

Self-Hosting on the Dark Web

david.alvarezrosa.com

356 pointsmooreds117 comments

What I really love about Onion sites is that if they are big enough, performance engineering really becomes Tor-specific. A few examples:

- Making assets embedded as base64 (img src the header logo as base64, all CSS should be inline, etc.).

- Leveraging CSS as much as possible (if you use animations and transitions, use CSS as much as possible for these, avoid JS for them).

- Make sure your website is mostly rendered on the backend. If you're to have JS, your website should work without it.

- Security becomes REALLY fun, as in, avoid XSS, CSRF, SQL Injection attacks and any other injections as much as possible.

As someone summarizes in another comment[0], keep the chattiness as minimal as possible. By chattiness I understand they mean, pack as much data as you can in the same Keep-Alive connection. Avoid making new HTTP requests as much as possible, as each one might get assigned to a new Onion route making things slow.

If you can ship your website to the browser in a single connection, you've won.

I've always been impressed by performance of these big Onion sites, they really push the limits of software engineering creativity, given these constraints and nature of Tor.

[0]: https://news.ycombinator.com/item?id=49872320

EDIT: Formatting of bullet points.

Are these simply good ideas regardless of tor?

With CDNs of today, they are not so much relevant for the clearnet.

Depends on which ones. Embedding assets as base64 makes little sense nowadays with http pipelining.

Relyinging the least possible on js, and using CSS for animations sounds like good engineering to me

Not really. The latency between a client and clearnet sites is tiny, and splitting assets in to separate resources makes caching work better.

Some people live in Australia though

Minimizing latency and connection count is only half of the problem, though. To the greatest extent possible not relying on JavaScript, and to the greatest extent possible not relying on third party libraries when you do seems like a good general takeaway.

Yes, but

Also DDoS becomes a problem, and the ways it's done are pretty specific to Tor. Double captchas are necessary when you're getting DDoSed, due to performance reasons. Oh, and captchas are also pretty specific to Tor as well.

There's also a problem of site fronting. Anyone could run a proxy pretending to be you, for arbitrary reasons (not even necessarily the obvious forging and credential stealing). Every site, even a personal blog, has dozens of parasitic fronts, either actively malicious or dormant. You need off-site ways to tell users what is the real address, and provide a smoke test for them (often a part of the address as a picture, for example in a captcha).

>avoid JS for them

Using any JS defies the point and makes your site instantly suspicious.

How are captchas done?

A combination of different ones, sometimes close to what you know from the clearnet, i.e. click pictures that match a description. For the proxy defeating one, the server can send you a picture of the real .onion address where some letters are blanked out, and with noise added to it to prevent it from being solved by the proxy. You then have to enter the blanked out letters from the browser URL.

This can of course still be defeated if the proxy URL receives very few requests and just have a human in the middle to solve the CAPTCHAs. But it does make the attack non-automated.

Thanks!

> There's also a problem of site fronting. Anyone could run a proxy pretending to be you, for arbitrary reasons (not even necessarily the obvious forging and credential stealing). Every site, even a personal blog, has dozens of parasitic fronts, either actively malicious or dormant. You need off-site ways to tell users what is the real address, and provide a smoke test for them (often a part of the address as a picture, for example in a captcha).

How can you do this without relying on the normal web? Let’s say you use a normal website to show the onion link, if the website gets taken down, you lost your user-trusted mean to do that.

Your users should have bookmarked it

What about new users?

How did they learn about the site? There's nothing you can do to stop your competitor advertising their own identical site in the same way you did - a malicious proxy is just a special case of this principle.

I’m thinking about blockchain provided onion URLs, but maybe I’m fantasizing a bit too much.

It's not an easy problem due to mutually exclusive constraints, look at Yggdrasil and adjacent decentralized DNS tech as an example of thinking in that direction.

Yggdrasil doesn't have a name system. It often uses internet DNS.

"Adjacent" projects, for example Wyrd and its successor Alfis by the same dev:

https://github.com/Revertron/wyrd

https://github.com/Revertron/Alfis

Zooko's Triangle

>How can you do this without relying on the normal web?

How can you trust anything you haven't experienced personally? By using chains of trust, of course. There are directories that list onion sites, and also sites that link to their peers. That way you can be sure you're still in the same bubble at least, and convert the problem into trusting the entire bubble. It's not automated and pretty ad hoc, if that's what you're wondering. Automation in Tor has a history of being circumvented or exploited with novel scams, this is an adversarial environment.

If the website is available on both clearnet and Tor, add a `Onion-Location` header.

https://community.torproject.org/onion-services/advanced/oni...

This way you advertise the onion domain through an established chain of trust and visitors can decide to use that the next time.

I've done some searching and can't find a description of "site fronting" that fits with my read of your comment.

I thought the whole point of Tor is that I (and only I) am able to serve traffic at a .onion URL that I have generated. How could someone else get in front of that?

The attacker simply proxies your site on another .onion address and advertises that fake address in a popular directory or an ad. Any visitors coming through that fake URL interact with your site through the attacker's front. Since .onion URLs aren't easy to tell apart, this kind of phishing works well. If you run a discovery bot you can observe that the .onion zone is full of these fake fronts for all kinds of sites, because even if your site aren't of any interest to an attacker but you link to any other site, then the attacker of that site needs to front yours with that link replaced with a fake, to create a separate circle of trust for the victims.

That's why you need to very carefully choose a trusted entry point into the Torosphere and revise your choice from time to time; always remember your initial entry point because if there's any suspicious drama around it or if you notice a mismatching link on different sites, you might have been duped to enter an impersonator-controlled bubble.

Many things can be done about it: claiming your spot in the directories, smoke test captchas, chains of trust, or you can brute force your .onion to find a valid one that starts with a memorable string (the longer the better, but also the harder). Vanity addresses like this deter non-targeted adversaries by requiring proof of work to forge the lookalike URL. None of that is bulletproof, of course.

Apart from the 1st one these just seem like good practices in general

Until you got to the network traffic tricks, those were good practices for any landing page you want to open quickly!

My personal website has been hosted on Tor for years. It's easy to do from your home even behind a NAT because it's an outgoing connection from your point of view (which also makes it a great way to expose local services even when you are behind a NAT and don't not have a static IP), and by design your personal IP is hidden from your visitors.

I wrote about it in 2600 almost ten years ago (already?!). A copy of my article can be found here: https://pablorauzy.fr/outreach/2600/how-to-run-a-tor-hidden-...

If you have an Onion copy of your website, don't forget the Onion-Location http header which will automatically redirect Tor Browser users to the onion version of the website even if they visit it at the clear web address.

If it interests people, I also have a follow up article about I2P: https://pablorauzy.fr/outreach/2600/how-to-run-an-i2p-hidden...

I had never heard of the Onion-Location header, thanks!

onion-location:

http://pablo2httpff4vogufavlmbxw4jkgb3amnywex2xdnchpztkdu2lu...

And for the OP's site

onion-location:

http://dhevt6e4rtgbtr3jh53xrpwmgtilkah6nyjujocsspssrsexc7omx...

Question: Can the .onion sites withstand HN front page traffic

A link in a comment is not the same as a front page link it terms of traffic, and when this link is a .onion, you probably divide at least by a few hundreds if not thousands the number of people who will click on it (even the few percents who have Tor Browser installed probably visit HN using their regular browser so visiting the link requires to open a new browser — or to change your proxy settings of you use Tor the old school way).

I really believe the single threaded BusyBox httpd running on a low end mini PC in my bedroom will stand hosting my static web site without any trouble.

To rephrase the question, if a story with an .onion link appeared on the HN front page, could it withstand the traffic from HN users who understand how to access .onion sites

NB. The question poses a hypothetical. No one is suggesting that HN allows stories (submissions) with .onion URLs or that .onion URLs would receive the same amount of traffic as "clearnet" ones

For example, a NYT article using an .onion URL rather than a "clearnet" URL

NYT is probably a bad example. The only .onion site they have now is a secure drop

You probably want to add the Onion-Location header to the clearnet site so Tor Browser can automatically inform the visitor about it: https://community.torproject.org/onion-services/advanced/oni...

Good call, I'd missed that. Adding it now, thanks.

Done now https://github.com/david-alvarez-rosa/homelab/commit/b495cbe...

Good thought

Besides using a separate port, I would also suggest running the hidden service on a non-127.0.0.1 bind address, just in case you ever host something else on that port and forget to disable the hidden service:

> HiddenServicePort 80 127.13.37.1:8080

> listen 127.13.37.1:8080;

This way, strangers won't be able to connect to a service bound to 127.0.0.1, should you ever decide to re-use the port and forget to disable the hidden service.

You'll also need to use separate ports and/or bind addresses if you host multiple hidden services and don't want people to correlate them - if nginx doesn't match the Host header, it will serve whichever site comes first alphabetically.

It's also possible to use a Unix socket, which can have a descriptive pathname like /var/run/my-service.sock: https://stackoverflow.com/questions/69313114/using-nginx-to-...

using unix socket rather than tcp has advantages

Can you elaborate?

For starters, it’s twice as fast

AFAIK TCP outperforms Unix sockets for some odd reason, but it's irrelevant anyway because you aren't getting that much bandwidth through Tor.

Every time I've tried using Unix sockets, I've run into the problem as described in your stackoverflow link. The suggested solution ("just run tor as root") is not exactly best practice.

You can monkey-patch scripts around Tor service activation, but I haven't been able to get my Tor+nginx setup to work reliably after updates/service restarts when using unix sockets.

A few more tips.

1. If you want to improve page load speed you need to buy a HTTPS certificate so you are not limited to HTTP/1.1. Multiplexing in HTTP/2 is important for getting sites to load fast.

2. You can set the HiddenServiceExportCircuitID configuration to pass the circuit id to your web server for telemetry or anti abuse purposes. Otherwise your logs will say that all users are coming from the same IP.

https://blog.cloudflare.com/cloudflare-onion-service

Fascinating, onion services are always encrypted by the tor network but still tunnel "cleartext" http inside that, and there are no CAs that issue free of charge certificates for .onion domains, and therefore there's no free of charge way to get http/2 on onion services without self signing.

Which raises the question: why not just trust self-signed certificates on onion services? From my brief look it seems to be because the Tor project views the primary purposes of HTTPS on onion services to be other things rather than just http/2 support: http/2 isn't even mentioned on their page about https for onion services (https://community.torproject.org/onion-services/advanced/htt...). Unfortunate.

I personally would support automatically trusting self signed https certs since their key is typically secured under the same safety as the hidden service's key. And even when they are not the browser has no warning when you get downgraded to HTTP on an onion compared to a regular site.

Trying to push hidden services to stay on HTTP is going against what the rest of the web is doing and as a minority of web traffic it really should be aligned to the rest of the web and also require HTTPS. Yes, it's technically wasteful, but reduces both work and security risk by keeping security models aligned with the rest of the web.

And it breaks Javascript as a lot of APIs only work when the site is served via HTTP - which .onion sites won't be usually. Tor browser treats .onion sites as secure content, but not your regular browser using TOR via proxy.

Who would issue the certificates?

The "self" in "self-signed" means "you"

There's no point teaching the browser that self signed certificates are secure instead of teaching the browser that onion addresses are secure.

I believe the http/2 implementations are often quite intertwined with TLS in particular

Negotiating HTTP 2 requires an extra round trip btw. It gets absorbed into the several round trips required for TLS.

Despite that my benchmark was faster with HTTP 2.

Can you buy one of these in XMR?

Not directly.

What is the benefit of building the same website twice with different hostnames instead of using relative links to content on the same domain?

Fair point. Very small things like RSS, canonical link or og:url or microformats use absolute URL

To make sure once in the .onion, you never leave the .onion

Website impersonation/fronting is a big problem in onionland, this at least makes fronters need to take more effort.

Not enough to be worth it.

Besides accessing your page are random people able to use your server as an exit node? Am I thinking the right thing... I met someone in Switzerland that was hosting anonymous exit nodes to some anonymous network and he said that it was a pain having to explain what was happening to the police.

That'd be an exit relay, not doing that atm, just in case

Running a Tor exit node is a manual process. Running a hidden service like a website, or chat server doesn't involve anything like that.

Exit node are an entirely optional part of the Tor network. If you run a relay or a hidden service you are not forced to participate in the exit node side of things. It's also not recommended to combine these roles because it could have security implications for your hidden service.

You can't accidentally run an exit node.

The exit nodes of Tor are a big weakness of the connective layer with the internet. For this reason I prefer to think of Tor as its own internet. Exit nodes are basically just inefficient VPNs. If traffic stays inside Tor it stays secure.

The so-called "clearnet" www has suffered from "developers" hell bent on "scraping" and crawling to support their so-called "tech" companies. The situation has gotten worse with "AI" companies scraping and crawling for LLM "training data"

These companies also attempt to create "walled gardens" to enable behavioral surveillance and ad targeting on pools of "logged in" users

Many of these websites are hidden behind "CDNs" that try to force visitors to use surveillance and advertising-friendly web browsers, allegedly to protect them from those developers hell bent on scraping/crawling (who they refer to as "bots")

The end result is that in _some cases_ it can actually be more difficult for me to do information retrieval from popular websites than from so-called "darknet" sites

I prefer textmode to graphics (e.g., X11, etc.) and I'm not a fan of the so-called "modern" browser. As such, .onion sites are more user-friendly for me

I like the design of using public keys, or portions thereof, as hostnames. No ICANN registrar "domain name business" and extortion from trademark holders

The point is that regardless of what might be offered over the "darknet", its design actually makes it more accessible to me

It supports commerce but there's no surveillance or ads so it's quick and efficient

IMHO, this is what the "clearnet" www should be like, but isn't

"The situtation has gotten worse with "AI" companies scraping and crawling for LLM "training data""

I keep encountering more and more sites complaining about it. And of course any visitor not using a popular browser running Javascript will suffer as they are likely to be mistaken for a "bot"

It's so bad I'm even seeing software mirrors complaining about it

For example, one has disabled indexing ("index of" listing)

I recommend that people give I2P and Yggdrasil a try as well, especially Yggdrasil. It makes no compromises on speed and latency, but it has no anonymity.

absolutely love how OP's site is designed. It's clean and minimal, fast and user-friendly, but also stylish.

Indeed. Also interesting to mouse hover on the avatar in the home page: https://david.alvarezrosa.com

yes, interesting enough, I also did hover and when it rolled i chuckled... like a head rolling... =LOL.

no spoilers, 'cos it made me laugh, but leave your cursor on the image - there's more to it than just the initial hover effect

;)

Agree completely. First thing that stands out about it: unique design. I immediately enjoy looking at it.

It reminded me of a simpler version of https://gwern.net/.

Thank you for the compliment <3

my pleasure. :)

The one thing I learned from hosting superkuhbitj6tul.onion (from a home computer) for ham radio and science stuff for about a decade was that EVERYTHING ON A .ONION IS EPHEMERAL. When the tor project correctly decided that for high security torv2 no longer was anonymous enough they unilaterally wiped out every torv2 .onion site that existed. Every link that was made between these sites came to an end in 2021 when they released a tor client without support for torv2 onions and tore the web to pieces.

Know this: the "dark web" is not for people who just want to own your domain name. It's for SECURITY and that use case is going to drive all their decisions. And if it wipes out every community in the entire tor dark web? So be it. And they'll do it again. Don't build your communities on the sand that is the dark web. You won't like the result.

Don't tie anything to a domain name you mean. Advice that's just as valid for clearnet.

v2 onions being deprecated was announced long in advance so many sites managed their transfer by announced the new URL well in advance. I don't think anyone was really bothered.

Oh, and all the links got re-written to the new URLs then? Between every site? No, they didn't. If every link breaking, every write-up about sites, every single reference across the entire .onion web breaking doesn't seem like a problem to you I'm not sure what to say.

Yeah, some sites told their users about their new v3 domain. But the web was entirely destroyed.

While unfortunate, I see no difference with the clearnet.

One might even see it as positive, because it forces maintenance. Anyone still linking to v2 addresses clearly is letting things rot. Nobody should ever assume that content available under a certain URL will be there forever, after all, even if the domain stays valid and existant.

It's not that some URLs change over time. It's that ALL URLS EVER suddenly stopped working all at the same exact time. This was the wholesale destruction of everything built on that darkweb. Nothing like it has ever happened to the clearnet web and nothing ever will. It is likely to happen again on Tor. And that is what I am warning about.

You're not wrong about significant brittleness and lack of sovereignty (and the same should be said about the web, too, roughly speaking), but I don't think your prescription is correct. Thus far, my only solution has been to maintain a variety of gateways to the same signed object which itself provides the evolving list of connection gateways. Gotta stay ahead of the whackamole through diversity, imho. I've found my i2p and tor identities have actually been some of the longest lasting, especially compared to non-bigcorpo clearnet web hosts. That's definitely not how most people want to use the net, ofc, but it's probably the right kind of practice that we should collectively be engaged in together if we want communities that actually own the means of production about as far as we reasonably can atm. And, these days, LLMs make this process much, much easier, too, as rediscovery and rebuilding custom infra per individual or community is far more doable for the average person (if they really wanted to do it, and I'm not claiming they would*).

Thanks for sharing! Happy to get feedback :)

If we could not call it the “dark” web that would be great. How about free web.

In a guide like this, it might be helpful to emphasize backing up the generated private key, loss of which would cause you to have to change your onion address.

I thought about doing this back when I was self-hosting my blog, purely because I thought there would be a neatness factor to being able to honestly say "I have a site on the dark web".

I eventually moved the blog to Cloudflare Pages primarily because I wanted to use Cloudflare Workers to handle the comments, though I have still considered dual-publishing it just to still say I've done it.

A bit late to the party but check out https://github.com/brewsterkahle/onionpress/ - it's an effort to make self-hosting on Tor hidden services as easy as possible.

I like the idea of TOR, but whenever I used it, I hit a speed penalty.

This, in turn, handicaps me searching for information. If they could fix this problem then I would be more likely to make use of TOR. We really need to think long-term about a future web that isn't ruined by Google etc... while also not being locked down such as via age-gating.

Agreed, it can be quite painful

I think it must depend on how far away you are from a gateway into the Tor network. There's a slight delay for getting the network ready when I start Tor on my devices, but after that it's not so bad.

Tor amplifies the problems caused by badly written websites that do a billion HTTP calls (unless you have HTTPS enabled on Tor) which does make it noticeably slower in a few edge cases, but I don't think it's as bad as its reputation suggests. For this particular blog, the Tor version loads maybe a couple hundred milliseconds slower, mostly because of images coming in slower (content is ready almost immediately).

Lots of hidden services seem to be under constant DDoS attack, though.

There are still plenty of issues with TOR (my main gripe is the "domain names" being practically impossible to recognize/trust or tell anyone about in the real world), but I don't think speed is the biggest factor anymore.

> so that no single party can link who you are to what you are doing

some single parties called government agencies pretty much can. it’s just much harder to do, so you’re safe from random people

we have been hearing this all the time whenever Dark Web is being discussed, but fact of the matter is that all the big darknet market busts have been done in past because of sloppy opsec by their operators and not some magical government wand which can pinpoint location of a hidden service

It's also a matter of whether it would be worthwhile to reveal the existence of such a capability in a given case.

If I were a government and had just spent a pretty penny on implementing the necessary infrastructure to defeat an encrypted routing scheme, I would probably not use that capability to bust internet drug dealers and other petty criminals, resulting in everyone scattering to a new scheme or to seek security through obscurity in some hand-rolled solution. I'd save it for national security level threats. High value targets.

Though thinking about it further, it would still be workable to gather compromising information on low-value targets without necessarily acting on it.

> It's also a matter of whether it would be worthwhile to reveal the existence of such a capability in a given case.

That would require many individuals to keep a juicy secret for a hypothetical future collective reward.

Adds a whole new layer of paranoia, but for some things, it's probably the only way to genuinely stay off the grid.

No DNS, no CA, no exposed IP. Just a ridiculously long string of characters and a bit of determination.

Imagine if normal people could install a single normal application and just run a website from a folder. CLI makes it more difficult than hosting a normal website. Typing commands you don't understand doesn't seem all that of a great idea.

If you don't understand what a command does, go learn it.

For us that's fine. It's quite a hill to climb for non technical folks. Even AIs holding their hand will have to do a lot of explaining.

That would severely limit the amount and the kinds of content on tor. People know/understand different things, they spend their time learning more about those things. I think we would want them to make pages about those things rather than stop what they are doing to do something entirely different. People aren't stupid, it's an unattractive proposition. Publishing online is usually as easy as to register once then post into a text area.

Kinda funny, I wondered what it would be like from that perspective but the first website about editing the config file strongly discourages doing that (specifically by instruction of random websites) then they throw you into an abyss of config file madness.

Just the first task is already asking a lot from our proverbial botanist.

https://onionshare.org

https://docs.onionshare.org/2.6.5/en/features.html#host-a-we...

Onionshare runs on basically every device out there (for obscure devices like Linux ARM phones you may need to compile it manually). Install the app, point it at a directory with an index.html file inside, and you're done.

Used to work like this when you had only one computer on an internet connection. With ipv6 it could be a reality once again.

Nice thing is you skip port forwarding entirely, which matters a lot if your ISP has you behind CGNAT. Curious how people handle uptime though, since a hidden service going down isn't something you notice until someone tells you.

Agreed yeah. Mine has been up with no issues so far for ~half a year.

(There are solutions for CGNAT - https://david.alvarezrosa.com/posts/self-hosting-behind-cgna...)

Can't you just forward ports through ssh?

That approach forwards everyrhing so can be used for multiple things (ftp, web, email, etc)

Same as you handle uptime on anything else: you use monitoring software.

If people need to tell you your services are down, you end up eating a huge amount of downtime.

My open source project (https://github.com/du82/nonograph) spawns a Tor hidden service with Onion-Location advertising by default, and on the Docker container its always on and self-healing

Thanks for sharing

Ran a small onion site for a couple years and the nice part is you never touch a public IP or a cert. Downside is onion v3 addresses are impossible to share verbally and the latency makes anything chatty feel broken. Static pages only, honestly.

You have to keep chattiness low, but a lot of SSR stuff works fine. Dread uses SSR, and even nags you if you have JavaScript enabled.

the thing that surprised me running an onion service was reachability, not setup. a fresh or restarted service can take a while before clients find it, so anything that has to work on first try needs a fallback. and if some of your users are where tor is blocked, plain tor isn't enough, you need bridges like obfs4 or snowflake

The "thousands of volunteer-run servers" on Tor is mainly the CIA and FBI.

Reply on news.ycombinator.com