Mitmproxy 5.3 | Hacker News Reader

Mitmproxy 5.3

github.com

168 pointscwaffles42 comments

Great to see us on HN, even if we don't have our 5.3 blogpost up yet :)

This release continues our attempts to become a bit more friendly to new users. Our docs now include a beginners guide created by our GSoC student this summer, which hopefully lowers the entry barrier a bit. We still want to improve this substantially, but hey here's a first iteration. :)

Happy to answer any questions!

TLS proxies are easy to fingerprint. The remote end (or intermediate routers) may not be able to deduce that it is in fact Mitmproxy that is running, but at least it can see a non standard ja3 fingerprint. Depending on that it might take different actions on what to show/do to the client. Some way to customize the client hello (look like the one from chrome, firefox ...) would be awesome :) afaik it would be the first proxy solution which does that.

Came across this recently as a way to proxy a local HTTP server to HTTPS, which you can do with e.g. mitmproxy --mode reverse:http://localhost:8081/ -p 8082 -q (to make local port 8081 accessible over HTTPS on local port 8082)

Looks like a really powerful tool, must make time to learn more about it!

If you’re doing this for a real service, it would make much more sense to use nginx or Apache httpd. Mitmproxy is very cool, but it’s really meant more to be used for investigation, pen testing, etc.

Yeah sorry, should have said this was for testing out something quickly on my local machine!

Squid also works great for this. I am posting this message through a squid ssl-bump proxy.

IME mitmproxy (along w mitmdump) provides a remarkably powerful and useful toolset that every web(/app) developer should learn.

I'm a developer and always on the lookout for tools that could help my workflow. Genuinely curious, what kind of stuff do people usually use mitmproxy for?

Reverse engineering / sniffing HTTPS traffic to figure out what API calls an app makes.

Using it every day at work as a simple reverse proxy with debugging capabilities. Any call my (local) frontend does to the backend goes through mitmproxy so I can quickly figure out what's wrong, edit and replay calls. Production env uses more scalable tools but mitmproxy does a formidable job for development environments with almost zero setup costs :)

What do you use in production? I don't have anything particularly high traffic, but knowing how the third party API barfed would be very useful, rather than just being alerted something went wrong.

I use it when pentesting/bug hunting to examine the way the application works and to modify traffic in order to break stuff (either the app or the server-side API it uses).

Basically the same things you would use Burp for, except mitmproxy has two important advantages:

1. Supports a SOCKS5 mode and has much better support for intercepting non-HTTP TCP traffic.

2. Easier to extend quickly via Python scripts.

3. Doesn't require an account/EULA.

Not really sure if an account/EULA is actually required for Burp, it's been a while since I looked at it. I just remember a window popping up on first run in my Kali VM, me looking at it and deciding to look for alternatives.

Some frameworks make it surprisingly difficult to see raw, unaltered traffic. I have used tools similar to mitmproxy to debug HTTPS traffic for Apache CXF because the framework's built in logging is kind of useless for debugging errors.

We used this in an initial prototype of a black box testing framework. The idea being that you can record responses from AWS and other external systems and play them back at a mitm layer so you can write end to end tests without modifying application code and without setting up all real third party systems.

But we ended up switching to our mitm server since it's pretty easy to serve http and you just set http_proxy in all relevant application environments and you're done!

Congrats on the release to the maintainers. I like the way this link to mitmproxy credits contributors. What are the maintainers using for release notes?

I would also recommend people look at Hetty which I just learned about this week. I have no connection to it. https://github.com/dstotijn/hetty/releases

Off topic:

1. This uses python to allow easy custom scripts. If they instead had wanted to offer the same functionality in a typed language, what would have been the easiest way to do that? Golang?

2. The docs are very cool - they have demo “videos” that actually have selectable text, I haven’t seen that before (via https://asciinema.org)

The "native app is always best!" crowd needs to spend more time using mitmproxy for apps on their devices to realize the trade-off they're making. At least the web has ublock and a developer toolbar.

I actually agree with you on this, but it is at least worth seeing what we do on jailbroken phones using Substrate (my runtime code modification framework, which shines with Objective-C); I would go so far as to say that making complex changes to an app is easier for a number of esoteric reasons I am too tired to describe (but are probably obvious to anyone who has worked with both).

I’m unsure if this is because I’m just more familiar with introspecting native code or because the tooling, ecosystem, language runtime, platform are more amenable to it.

mitmproxy is great! I wish there was a way to conveniently look at non-HTTP TLS traffic, though. I saw a few documented "TCP mode" options but they don't seem to get me access to plaintext. Any ideas?

Can you even do this if the client hasn't installed your cert?

It depends on how the client will treat an untrusted cert.

No, only works if there's a way to install custom certificates on the device you are monitoring. Otherwise that would defeat the whole purpose of https (If there are no bugs in the implementation or other security issues of course).

For mitmproxy that would be achieved like this: https://docs.mitmproxy.org/stable/concepts-certificates/

Thought so i played around with this and my phone, self hacking ;)

It's also very nice to intercept app traffic. Some use cert pinning, so the custom certificate won't work.

Think there are a few tools for getting around the certificate pinning, projects like sensepost/objection

In most cases you can get away with simply connecting to the application with `objection` (mentioned in a sibling post) and running `android sslpinning disable`.

This will fail in some cases when the application is obfuscated or uses a non-standard pinning mechanism. In this case you can decompile the application to determine the methods used to accomplish the pinning and use frida (https://frida.re) directly in order to manually override them.

Wireshark might be a better tool for this

Wireshark doesn't natively decrypt TLS traffic, though. And SSLKEYLOGFILE is not supported by many network apps.

mitmproxy will generate a certificate for you to install on the client.

You can view TCP contents in the mitmproxy UI since a few months (thanks to some excellent community contributions). Minimal example:

  mitmproxy --mode reverse:http://example.com --tcp-hosts example.com
  curl http://localhost:8080/

There's still quite a bit we want to improve on (e.g., our protocol detection currently fails for protocols that start with a server-side greeting), but the basic TCP viewing functionality is there. :)

If you just need to look at the traffic, maybe wireshark in combination with SSLKEYLOGFILE can help, if your client supports that option (eg: libssl):

https://gitlab.com/wireshark/wireshark/-/wikis/TLS#using-the...

Yeah, sadly the clients I care about never seem to support that.

Which ssl libraries do they use?

Why would I consider using this over Charles Proxy?

But it does look like a cool project.

mitmproxy is scriptable with Python which is very useful. For example, you can use this to record statistics about the requests or even alter responses/requests in arbtirary and complex ways

Performance and open source! I used Charles for years and always had issues to run it on lower spec devices. And after you get used to the navigation and shortcuts it can also save you time.

used this before for web crawling, very useful tool!

Big Big fan of mitmproxy. This is a wonderful tool, and with scripting functionality it helps achieve so much more. I have been introducing this tool to so many developers and quality assessment teams, to not just improve day to day things but also perform privacy assessments w.r.t GDPR etc.

Thank you so much for your work. What is the best way to donate to your project?

Interesting information! thaks for the answers and links

“Let’s contract out the development of our infrastructure,” they said.

“Use the cloud,” they said.

My point was that giving others the keys to the kingdom with request handling puts you at risk of using MIM attacks, which people think are overblown, but they aren’t. I read an HN poster recently that was effectively admitting to doing this, by redirecting traffic to their domain from their company then back out again. One environment variable changes on a “secure and tested” Docker image, and you just started giving your data to someone else. Or, change the hosts file to point at some external IP, then anything logging the host/FQDN would seem to show nothing changed. Don’t let this happen. Better yet, just don’t have any data anyone would want. Stream hamster yodeling, because you and your non-differentiatable AWS and Azure IPs are f’d anyway.

Reply on news.ycombinator.com