Mitmproxy 9: WireGuard Mode | Hacker News Reader

Mitmproxy 9: WireGuard Mode

mitmproxy.org

158 pointsmhils46 comments

mitmproxy dev here, happy to answer questions once I'm back home later! :)

First of all thanks! Not sure if you know any of the Linux maintainers, but if you do I'd love to know how to get in touch with some of them.

Are you looking for mitmproxy-on-linux maintainers, or Linux kernel maintainers? You have found the former I suppose, but I'm the wrong person to ask for the latter. :)

I was looking for a quick way to update my version from the command line (ubuntu), but --options didn't show anything off hand like an --update/--upgrade. I just briefly skimmed the docs/google for something that would avoid having to reinstall the binary.

EDIT: also is there a way to configure the IDE that pops up when I go to edit path? It's vim which is fine, I'm just curious if I can either integrate a custom vim installation or route it to another IDE.

The recommended way to install mitmproxy on Linux is to download new standalone binaries. They are self-contained and can just be dropped into /usr/local/bin. If you want some way to update automatically, use our Docker images, install via pipx (https://docs.mitmproxy.org/stable/overview-installation/#ins...), or (for Windows users) use the Microsoft Store.

> also is there a way to configure the IDE that pops up when I go to edit path?

You can set $MITMPROXY_EDITOR or more generally $EDITOR. :)

Thanks for creating awesome software. Being able to script it with python is amazing.

One problem I have been having on MacOS with Android emulator: proxying traffic from the emulator (my own app) will work for about 15 minutes or so... then all traffic ceases to flow, and Emulator / Setting / "AndroidWifi" says limited connectivity. If I forget AndroidWifi access point, re-add it, and restart, it all works again for about 15 minutes. Any ideas how I would debug this?

Hard to say without more details. Once you reach that limited connectivity state, do you see anything that stands out in the mitmproxy event log? Feel free to open a thread at https://github.com/mitmproxy/mitmproxy/discussions and I'm happy to help a bit more! Also it would be interesting to know if you observe the same thing with our new WireGuard mode. :)

First of all, thank you for all your work on mitmproxy! :)

Could you elaborate on how exactly the WireGuard mode works compared to e.g. TLS interception? After all, WG doesn't come with certificate authorities or anything like that.

WireGuard mode does not help with interception, it helps with getting traffic into mitmproxy. Put differently, it's a user-friendly alternative to

  1. configuring an explicit HTTP proxy in your system settings, or
  2. fiddling with iptables on your router and/or your mitmproxy device.

The downside of 1) is that it does not work for UDP-based traffic such as DNS and can often be easily bypassed by applications. The downside of 2) is that it's generally fiddly and does not work very well for users with consumer routers (no iptables). In all cases you still need to install a trusted mitmproxy CA for TLS interception, that part does not change with WireGuard mode. :)

Sorry, I'm not following. With e.g. HTTPS traffic all I have to do is make sure that the browser sends all its traffic through 127.0.0.1:8080 and trusts mitmproxy's certificate. In that case, mitmproxy will show me the plaintext traffic.

Now, let's say I've set up WireGuard with three peers, i.e. each of these peer's public key appears in my WireGuard config along with an IP address. Now I start sending traffic to one of these IP addresses which WG would normally encrypt and then route directly to the peer in question. How can mitmproxy's WG mode help me here with "getting [the encrypted] traffic into mitmproxy", as you say, and reading the plaintext?

Suppose you have an Android device for which you want to see all traffic. You could configure an HTTP proxy in your system settings, but this does not capture any UDP-based protocols. Additionally, apps may choose to ignore the proxy settings and it's hard to tell if they do. To overcome these shortcomings, we now have WireGuard mode: Mitmproxy spawns a WireGuard server on startup (instead of an HTTP proxy listener). Now you don't set a proxy on your device, but you configure your device to use WireGuard with a config that sends all traffic to mitmproxy. mitmproxy then transparently intercepts all requests that are coming through that WireGuard tunnel (the device still needs to trust the mitmproxy CA). Put differently, instead of using an explicit proxy configuration or something like iptables to route packets to mitmproxy, you use a VPN (WireGuard). The benefits are:

  - You can intercept/modify UDP, in particular DNS.
  - You avoid the "apps ignore proxy settings" problem.
  - On Android specifically, the WireGuard app allows you to only proxy specific apps (not possible with a global proxy config)

Does that make more sense now? We also have a bit more documentation at https://docs.mitmproxy.org/stable/concepts-modes/#wireguard-....

Thank you, it now makes a lot of sense. My apologies, I thought this was about reading WireGuard traffic, not about using WireGuard to replace the usual HTTP proxy.

I've inadvertently tried this mode yesterday, even before the announcement was made, but found that something weird was going on:

- it eats tons and tons of CPU, like this is a common picture

6568 root 20 0 7357216 3.2g 20072 R 109.6 1.3 6:10.60 mitmweb

- lots of packets dropped, sometimes at a considerable rate

[14:38:33.501] Dropping incoming packet, TCP channel is full.

This isn't me running some http-hungry application, just immediately after the phone restart. I'm running mitmproxy 9.0.0 installed through pip on Linux with python 3.9.2. Unfortunately it's not easy for me to try any other mode - i've first tried setting up transparent proxy on MacOS, but ssl bindings crash on M1.

This really shouldn't be the case. What kind of client are you using? Could you open an issue over at https://github.com/mitmproxy/mitmproxy/issues please?

Done. Mobile client is the official WireGuard mobile client.

Is the UDP work a precursor to HTTP/3 support? I don't see HTTP/3 mentioned in the announcement, so I assume that something else is needed to make it work still?

Edit: I take that back, I see "Add HTTP/3 binary frame content view" in the commits. So does that mean it works? I would have thought that would be a headline-level announcement, though?

Either way, amazing stuff, and thanks for the brilliant work!

Yes, the UDP parts are preparation for QUIC/HTTP/3! We will be there soon. :-)

This looks great! My only frustration is not with mitmproxy, but macOS -- setting a proxy in System Preferences->Network isn't a guarantee that it will be used by apps, and there doesn't appear to be a clean way to force all traffic through a proxy.

Isn't that closer to what a VPN would try to do than a proxy? I guess it depends on whether you mean all traffic on the machine or only the kind of traffic that travels over proxies (I think socks4/5 are special cases and presumably not what you meant on a post about mitmproxy)

I was just thinking TCP, although since mitmproxy now supports UDP perhaps it can be all traffic?

If you can run mitmproxy on another device, then our new WireGuard mode does exactly fix that problem. At least as long as WireGuard is able to capture all traffic. :)

Transparent same-device interception is something where we still need to do some work on. I've started some work for Windows (https://github.com/mitmproxy/mitmproxy/pull/5543), but the top priority is HTTP/3 first. :)

A friend of mine claims to have found mitmproxy on her phone, and is worried that her ex husband is using it to track her movements and interactions. I am unable to find anything concrete to say whether or not this is a possibility. Anyone heard of something similar being done, and know how this could have been accomplished. She says he didn't have access to her phone, but I know he is technically capable.

Is it iOS or Android? I’d suspect it’s way easier to do this sort of thing on Android.

But iOS also has certain network extensions you can use (like for VPN) that can redirect traffic.

Not staying that’s what happened, just saying that alters the likeliness.

mitmproxy would not really be helpful for what you describe, this sounds like she is confusing it with some other software/stalkerware.

Nice work! Does this mean that intercepting WebRTC media traffic is now possible? It's DTLS, so I guess I'll have to check it out!

We can definitely intercept DTLS now, there are no specific contentviews (pretty-printers) for WebRTC yet. I don't know how much of a binary protocol WebRTC is that would make pretty-printing necessary. :)

This is crying to be rewritten in a faster language. Python is too inefficient for a lot of production mitm use cases.

Which use-cases are you thinking of? I'm sure they exist but they've never come up for me; I've used mitmproxy many times and it's always worked gloriously, but I never use it to capture / re-write more than a few dozen requests at a time.

https://github.com/google/martian

I don't know anything about this project, can somebody explain what the (nonmalicious) use case would be?

Debugging first or third party applications that are acting up.

I use mitmproxy (mainly mitmweb as a reverse proxy) on a daily basis to test and debug front-end applications. Mitmproxy allows me to reach application states that would otherwise be unreachable without modifying the application code. There are or course alternatives but mitmproxy is very capable, the mitmweb ui is really convenient and the scripting interface is a pleasure to use.

What I've used it for on the past:

1. Debugging client/server API calls to isolate problems in complex JSON interactions. You can even set breakpoints to modify server responses on the fly (within the client timeout period)

2. Capturing live responses from API for replaying in client regression tests.

3. Faking server responses for a subset of routes that haven't been implemented on the server-side yet, while keeping all other routes intact (e.g. auth). This enables parallel client & server development. Scripting Mitmproxy in Python is easy.

4. Non-software engineering use case: replacing resources on third party websites. e.g. injecting a dark mode css file into a website rhat doesn't officially support one.

Do you have links for the Magisk module? (and ideally a quick android tutorial)

The source code seems to be here: https://github.com/jorants/mitmproxy/blob/d3ceb1b0f8a5a09e63...

Installation steps seems to be:

- Navigate to /cert/magisk and download the ZIP file

- Load the ZIP file in your Magisk app

- Might be necessary: reboot

The link to the Magisk module is also on the normal Mitmproxy cert page. Because each install generates (or should generate) its own certificate, the module needs to be generated on the fly.

I haven't played around with this myself yet, but basically:

  1. Configure your device to use mitmproxy.
  2. Visit our magic mitm.it domain on the device.
  3. Click on "Show Instructions" for Android.

Is there something like this that can introduce network latency, but on the TLS and/or HTTP level? E.g. delay TLS handshake (to simulate CRL checks), send headers but delay streaming, etc.

I think that's what comcast (https://github.com/tylertreat/comcast#readme) is trying to do more than mitmproxy. I think there's a competing project but that one is the easiest to remember :-)

Is there a good mitmproxy configuration that blocks ads network wide particularly YouTube? I tried using mitm-adblock but it was so slow it was unusable.

I don't think I'm aware of any. If possible I would recommend using a browser extension like uBlock Origin instead, being in the browser context makes adblocking more efficient. (I realize not every client is a browser, I simply don't have good recommendations for this then)

Is the wireguard option also supported on the Basis? Especially FreeBSD which now has wireguard kernel support.

What do you mean by "on the basis"? :)

I mean on the BSDs. Sorry. I was typing from my phone and it autocorrected it, normally I notice but here I didn't.

Our WireGuard mode is entirely user space and works on all operating systems mitmproxy runs on. :)

Oh great, I will try it thanks so much for writing it!

I really love terminal UI applications for their speed and low footprint.

Thanks for this! I haven't really played around with packet captures since back before HTTPS was widespread but I've heard a lot of people gripe about it over the years.

Big shoutout to one of the poor bastards who had me as their student back when Pitt segmented the school of information science away from the rest of the network.

I have no idea if that's still the case, because those moron librarians added a card reader, and I don't know how to parkour onto the roof of that building.

I got hung up on the fact an extra credit question go marked wrong because I said based on the SSID, the packets were from a BSD operating system. The TA marked it wrong and said "no, it's from an OSX system"... I ended up in the actual professor's office, with a listout of the various codes for manufacturers and pointing out there's no "OSX" specific one.

(In retrospect, it was a situation where I was rules lawyering because I was pissed because I kept bumping into a string of people who'd tell me "It's not my job to teach you" only... they weren't some cute woman from Iowa in town for a book signing meeting me for espresso, they were literally a professor teaching, in one case, a "statistics for psychology students" course I selected EXPLICITLY because their students also haven't had Calculus... and they were tanking my GPA.

Anyways, I'm re-hash-ing very old gripes, but it felt like something fundemental shifted when https went widespread, while universities seem to focus on weird CTF bullshit.

(You can just message them on a dating application, though then they'll make a show of refusing to hire you and reporting you to the FBI as a possible cyberterrorist... like uh, I get that it's spooky season or whatever but if someone made an offer I wouldn't be a cyberterrorist anymore!!)

Anyways, I am looking forward to playing with this after I clear my todos. Thanks for the hard work.

Reply on news.ycombinator.com