Roll your own Ngrok with Nginx, Letsencrypt, and SSH reverse tunnelling | Hacker News Reader

Roll your own Ngrok with Nginx, Letsencrypt, and SSH reverse tunnelling

jerrington.me

276 pointslioeters54 comments

I made sish [0] so this could be done for myself and friends automatically. It’s a binary written in Go that implements the SSH protocol and allows TCP forwards, HTTP(S) w/ built in requesting of certs from LE (including for custom domains) forwards, aliases (tunnels kept local to the daemon that does not bind a port) forwards, and TLS forwards using SNI for routing.

sish implements a web front end for inspecting web traffic on tunnels as well, albeit not as nice as Ngrok :)

[0] https://github.com/antoniomika/sish

The coolest thing about ngrok is that it has a request log in its web UI. And, it allows replaying requests! This is extremely handy for debugging webhooks without having to trigger them at the source every time you change something.

Might I recommend Hookdeck as well?

https://hookdeck.com/

Very very useful for handling webhooks and making working with them including locally a pleasure.

Also, if you’re using Stripe, the Stripe CLI is pretty great. It has webhook proxying function built-in.

We created Svix Play that gives the Stripe CLI experience to everyone for free: https://www.svix.com/play/

You can also do this with ssh and Unix sockets if you’d like to not leave ports open and accessible from the proxy server. It’s a little more work, but both openssh and nginx support this. I’m doing that on a remote server to run code server (vscode w/o the MS specific code) on a multi user system. I’d rather not have my code editor available for anyone else on the system. So, node is getting served on a socket, and that is the forwarded to a web reverse proxy.

There is extra authentication/authorization as part of the web server itself, but it is nice that I don’t have open IP ports on the shared server.

That said, for the use case from the article, if you have a more permanent setup, using something like wireguard would be more robust than an ssh reverse tunnel. But the ssh tunnel is great for more ephemeral connections.

This doesn’t duplicate one of the best features of Ngrok: the local web server that lets you inspect and replay recent requests. This feature makes writing webhook code so much easier, because if your code doesn’t work right, you can inspect the request, fix your code, and have Ngrok replay the request until it’s all working properly. That’s even in the free version! And the paid version is also pretty cheap, and you don’t have to maintain, nor pay for, a cloud instance

Sometimes I feel like most of my headache is in just getting that first request to hit ngrok. Then I’ll replay it dozens of times as I figure out the functionally.

TBH, that is not terribly hard to replicate, and certainly not worth it risking accidentally sending auth tokens to some third party.

Ngrok is indeed nice for this. I’ve found that Firefox dev tools can replay requests out of the box which is pretty handy.

I have left ngrok for cloudflared.

It is really nice if you are already a cloudflare customer and is cheaper last I looked.

https://developers.cloudflare.com/cloudflare-one/connections...

Not sure what this is called but, does this support when

the server needs to connect to the client (server has ports open but the client doesn't), so the client connects to the server and then the connection is like... flipped? flippable?

basically when I try to connect to 127.0.0.1:4500 as the server, I want it to go to the client (which has a bound server listening on a socket, but is closed port/firewall wise). Is this what "reverse tunneling" is?

I think you want something like the -R switch for the standard openssh client (that is, the ssh command in most Linux distributions -- read man ssh) for this; it allows you to have the remote client connect to your server running on your localhost.

While you're reading the man page, also check out -L, which allows you to connect to a remote server through the encrypted tunnel. This is the most common usage.

For example, let's say that you have a webserver running remotely that is only listening on localhost. Why would you do this? Let's say that you are running some sort of server management dashboard, but you're not sure how secure it is and you want to only connect to it yourself. Even though it is only listening on localhost, you can still connect to it remotely.

From your workstation,

    ssh -L 9000:localhost:8000 remoteserver

This will allow you to open up http://localhost:9000 in your browser and access the remote web server that's listening only on its localhost on port 8000.

Not sure if it has a name, but I've seen it called "reverse dialing" (which apparently golang's official build servers use to bust nat & firewalls): https://github.com/inetaf/tcpproxy/issues/8

I've been using FRP with Caddy instead and it's quite nice. FRP config is a bit clumsy and awkward but otherwise love it.

https://github.com/fatedier/frp

https://caddyserver.com/

Or just use

https://localhost.run/

Just one SSH command, and you're done.

Impressive - at least until this business model makes sense for them!

I used Oracle free tier VPS and threw Rathole on it. I haven't noticed any latency or performance issues.

https://github.com/rapiz1/rathole

This looks interesting thanks..

Nice! I use rathole with kamatera- it's nice to see a fellow ratter :).

> Roll your own Ngrok with Nginx, Letsencrypt, and SSH reverse tunnelling

And "a server in the public cloud".

By far the most important feature that ngrok provides, I’ve never used ngrok for its ability to replay requests

Why not WireGuard? WireGuard + SNAT (port forwarding on the VPS) + DNAT (route the packets back) does the same thing, and it's faster than using SSH to do HTTP reverse proxy.

You can also have public IPv6 address on all your LAN computers, and just open the firewall when needed. :D Even simpler.

Or even assign random unique public IPv6 address to your computer per individual customer, and listen on that address. That way, other customers will not accidentally hit the endpoints you don't want them to when demoing an app.

The same can be done for any other exposure of services on your LAN.

Is there a tutorial that explains how to do that? I'd be interested in learning more about this!

First, setup a pair of tunnels. There are multiple tutorials doing this. You could check Arch Linux Wiki of how to. On my setup, I assigned a pair of /31 addresses to each side just for the internal communication.

Then, if you want to use IPv4 NAT, set up SNAT and DNAT. I'm taking Linux's iptables for an example: on the VPS:

iptables -t nat -A PREROUTING -d <VPS Public IP>/32 -p tcp --dport port -j DNAT --to-destination <IP of another side of the WireGuard tunnel> iptables -t nat -A POSTROUTING -o <VPS Internet facing NIC> -j MASQUERADE

The first statement sets SNAT on IPv4 TCP (UDP is also possible) on a specified port. All packets going into this machine (dst = VPS public IP) will go into the other side of your WireGuard tunnel. -d <VPS Public IP>/32 makes sure that the packets forwarding to the VPN side won't get NATed. If you need multiple ports or protocols, just duplicate this line.

The second statement sets DNAT on the VPS. It makes the packets from your VPN side goes to the outside (sorry I'm not an expert in networking principals so that's my understanding).

On your local side, you just set up the tunnel and set your machine's default gateway to your VPS's tunnel IP address. Make sure you add a static route to your VPS's public address (WireGuard Endpoint) if you are connecting WireGuard using an IPv4 endpoint or it will get routed to your tunnel and you will disconnect.

I also sometimes setup IPv6 tunnels without NAT. On the VPS I just setup a WireGuard interface with /64 and assign each peer a /128. No NAT is needed for this case.

Thanks

I also built something similar once when I needed something like ngrok but wanted to use open source:

https://github.com/tweedegolf/henk

It has a very similar approach, but uses about a hundred lines of Go instead of nginx. It's based on unix sockets created by SSH reverse tunneling, whose names are used to select the desired subdomain. This makes it possible to add more reverse proxies with just an ssh command, without changing anything on the server. It's also small enough that it's easy to add custom logic such as request logging.

Why not use socat? Quick reverse proxy one line.

I've used ngrok every single day for ~8 years for work and didn't have the slightest clue how it worked. I'll still be paying for it but I learned a lot reading this.

Or use https://theboroer.github.io/localtunnel-www/

Doesn't this only work with TCP and not UDP?

Or not. Just use all-in-one managed services like serverless or PaaS.

In the past, I used to spend so much time on infrastructure, like setting up databases, HTTPS/SSL, network ports and security rules. Didn't understand back then that solutions I can buy with money are the cheapest solutions, because I should be focusing on problems I cannot solve with money (but rather with my time and skills).

To qualify, I know ngrok and lightweight set up can be super useful. When I was in college, ngrok was so awesome for doing small ad hoc fun things.

But as soon as someone needs to set up even the simplest SSL set up (which may take an inexperienced dev anywhere from 1 hour to multiple hours), I'd say that's not the problem to focus on.

Deploying a production app on serverless or a PaaS isn’t a substitute for making a local development app available on the wider internet.

There are a lot of apps out there running on managed backends.

ngrok is perfect for quickly making local development available. It's even free if you don't care about subdomains.

I don't get this argument. You can solve all problems with money.

You can't buy my love with money.

I don't need your love to run a server.

Okay fine. If you buy people's time and skills with money, I guess you can solve all software development problems with money.

I meant, say you want to ship [any new software product]. There are parts that are common to everyone, and parts that are unique to your product. Whatever is common to others, see what you can leverage (i.e. buy with money), so that you can make time to focus on what is unique to your problem.

I can buy programming time from others who will solve the unique part, too. I don't imagine I'm somehow the only person in the world who can solve some particular problem.

I can even buy time from people who will develop the description of the problem well enough from my wague idea to make it comprehesible/implementable by programmers, and who will lead those programmers for me.

My point is that it's not important whether something is common or unique to my product. For some "rolling their own" even if it's a common solution may be a good and comparatively cheeper proposition, if that will end up costing less as a result and will leave more resources to devote to their product overall.

How to end up with no one able to put these systems together in the first place

As humankind progresses, we'll abstract more and more.

Already, most are not able to put together these systems from semiconductor design. Or writing the OS. Or programming languages. And that is fine.

I envision that in a decade, we (the humankind) will have way more devs, creating way more apps, and not knowing a thing about NodeJS or PostgreSQL or such things that you and I take as "minimum requirements".

The real problem is that people don't connect the immediacy of adding abstractions to the fragility and complexity that it leads to. Because people today only learn the abstractions, they have a weak understanding of the system as a whole, and as such, the things they can do with it are severely limited. That's why modern apps seem to be getting worse rather than better. Young people writing code today literally don't know any better, and that's bad for our future. We're teaching people to hang big heavy lights from the ceiling without them learning rigging.

I disagree, but still respect your differing perspective.

Does anybody know a tool for tunneling/reverse tunneling that isn't SSH since SSH is doing TCP over TCP and is slow?

Wireguard

As far as I know the simple SSH "port forwarding" (-L and -R) used here is just layer 4 reading/writing from one TCP socket to another (plus some multiplexing/encryption in between) and doesn't suffer from any TCP over TCP problems, only the ssh tun forwarding option (-w) is level 2/3 and has that issue but that seems to be much more rarely used.

for tunnelling better use autossh

tailscale!

Or a VPN such as wireshark or OpenVPN, though that is a little extra setup usually.

(or one of the many public VPNs - though I'm wary of those, if I want a VPN I want to set it up myself and not have my traffic pass through someone else's control)

i believe you mean wireguard and not wireshark

I must certainly did. My brain slipped a beat and pickeda similar word from a thread I was also talking in elsewhere...

Reply on news.ycombinator.com