Reverse Engineering Native Apps by Intercepting Network Traffic | Hacker News Reader

Reverse Engineering Native Apps by Intercepting Network Traffic

nickfishman.com

188 pointsandrewrice69 comments

Let us also mention the great mitmproxy, an open source equivalent to the Charles proxy: https://github.com/mitmproxy/mitmproxy / https://mitmproxy.org/

My go-to has always been Fiddler[1] (for windows). I wish it were open-source though.

I'll give mitmproxy a try. Thanks!

[1] http://www.telerik.com/fiddler

Fiddler is amazing!

Another option is Burp Suite (https://portswigger.net/burp/ ) -- ubiquitous in the security world.

Wow, this guy has the completely opposite attitude of me. He seems to think it's a bad thing, an attack!, for users to see just what the hell data you're pulling off someone's phone. And, bizarrely, uses an example of an app that essentially stole data from its users.

I should be able to see what data an app is sending, and certificate pinning (and ATS according to another comment) kills that. That's not a good thing.

It's not an uncommon attitude, and it's also one I detest --- many articles about "security" have some authoritarian/paternalistic aspects to them. Then again, encouraging ignorant users is the easiest way to monetise, analyse, and otherwise manipulate them... and if this ignorance is a convenient byproduct of arranging things so "it's for your security" ostensibly, all the better.

The obligatory related link: https://www.gnu.org/philosophy/right-to-read.en.html

Certificate pinning does prevent an important class of MitM attacks, though.

I think some applications use certificate pinning when validating a certificate provided by a default certificate authority, but, if you manually install a root certificate onto your device, the app will accept the override. That's one possible middle ground.

Android Nougat disabled that — now it will not accept user-added CAs anymore, anywhere, ever.

While the privacy concerns are more than valid, reverse engineering is common practice in trying to copy your product.

Reverse engineering isn't inherently good or bad, it's just a tool. That tool can be used for both good and bad.

I always recommend certificate pinning in order to prevent MITM attacks. I also recommend it if you're backend API gives out a lot of information about your product's "secret ingredient".

That said - certificate pinning can often be bypassed: http://blog.dewhurstsecurity.com/2015/11/10/mobile-security-...

Reverse Engineering to copy is even considered a legal tool in the EU.

The argument of the European Court of Justice was that car manufacturers also buy cars from competitors, take them apart, and use the gained knowledge on their own products. The same happens in every industry — but in Software Engineering, it should now be forbidden? That's not possible.

If you want to protect yourself from that, publish your secret sauce and patent it.

I agree. Cert pinning is fine but there should be an option to disable it (maybe system-wide) for people who want to analyze traffic.

disabling system-wide is pretty much impossible. If OSes added such a toggle, people would start using their own SSL stack and overall security would suffer (because people won't be keeping their SSL stacks up to date)

This is already a reality. There are at least 5 banking apps I can think of that statically link OpenSSL and use that instead of OS crypto.

That would pretty much beat the purpose, the author of the app doesn't anyone to snoop.

Data generated on my phone belongs to me. By definition I cannot snoop on myself.

That's what you'd like to think, but the numerous terms & conditions that you've agreed to for your apps means that you're wrong, some data isn't yours.

No, it really doesnt. Terms of service are only legally binding within the law and the EU has already ruled that reverse Engineering software on your own device is just fine.

So I'll happily keep on ignoring those tldrs.

OK, then how do we separate the data that is his versus data that belongs to somene else? Maybe we would have to look at what is being sent from the app?

Who knows? The two things are unrelated. My comment wasn't anti-reverse engineering, just stating a fact: just because something is in your phone doesn't mean it's yours.

The parent stated "Data generated on my phone belongs to me."

I interpreted this as "Data generated by me on my phone belongs to me."

The user could agree to license her rights to the data, e.g. via terms and conditions. But it's still her data. That's why the agreement is necessary.

None of this has anything to do with "reverse engineering".

The scenario I am thinking of is a user looking at her data being sent from her hardware over an internet connection that she is paying for.

Except you can with a jailbroken phone. Never trust the client, ever.

"I should be able to see what data an app is sending..."

And you can. There are many ways. Something as simple as 2 socat instances and netsed works great as a quick and dirty but very robust solution. See also sslsplit which will generate certificates on the fly.

Anyone who is telling you that you can place complete trust in the use of x509 certificates on the open internet is either naive or dishonest.

I think you have the right attitude.

You should read up on how certificate pinning works. Obviously you can still decompile and recompile the app with different certificates baked in but it's a bit more difficult than what you're implying.

Any device that prevents a user from installing their own CA root cert is not one I would use. I'm not sure if they have started to do this or not.

In terms of protecting users, there's no valid reason the owner of the hardware should not be able to control the list of endpoints that she has already authenticated and is willing to trust.

It would be like if SSH did not allow the user any control over known_hosts or to decide that she will accept or not accept a connection.

The entire point of certificate pinning is to ignore the certificates the user installs... Nothing to do with the operating system at all. If the software doesn't want to connect because the wrong certificate is presented to it that's where it ends. You can install as many local roots as you want, it won't change a thing.

"... it won't change a thing."

So if the user does not want to trust a certificate installed by someone else on the device, she can "revoke" it?

And by the same token if she wants to explicitly trust a certificate, regardless of who installed it, she can do so?

Does the user have control of the process of "trust" or not? The entire point of the device, OS and apps is to benefit the user, not some third party trying to hide data being sent from the device... from the user.

Do you believe a user should be able to "MITM" her own traffic or not?

> Do you believe a user should be able to "MITM" her own traffic or not?

I do, but that is utterly irrelevant to this discussion. We are discussing what certificate pinning is and how it works.

You can currently perform certificate pinning on every single operating system you can imagine. You can do this in a way that completely ignores the trust store of that operating system, and anything the user does to this is ignored by the application.

This has been possible for years on Android. This has been possible for years on Windows. This has been possible for years on Linux.

All the developer has to do is include the certificate of their own CA with the application, restrict the SSL's trust store to this one certificate, and then also check the fingerprint of the resulting certificate offered by the server. Then if the application notices this fingerprint is incorrect, it bails.

This is reality. This is how it works. Nothing I believe or want will change this. No amount of certificates I install in my operating system's trust store will change this either.

What android is doing is making MITMing yourself harder. But it's always been 100% possible for developers to make MITMing impossible without first reverse engineering the app and replacing the baked in certificate.

That's just the way it works.

"What android is doing is making MITMing yourself harder."

That's not good for users.

I do not rely on Android, Windows or Linux. Not really much an app user either.

But if I were a user of these systems I would avoid apps where the user is not allowed to see what is being sent. Irrespective of any justifications put forth. By a company that relies on collecting personal information and selling advertising to make money.

> That's not good for users.

Yeah, no shit. But that's not my, or your, point. Do you even have a concrete point you're getting at, or is it just "I don't like Google"?

Android is making it MITMing apps harder if the application itself did not attempt to make it hard. It has ALWAYS been possible for applications to pin their certificates and to make MITMnig them a pain in the butt. On every OS. On Windows. On Linux. On Mac. On iOS. And yes, on Android too. It has always been that way. I already went over this. Here, go read this: http://security.stackexchange.com/questions/29988/what-is-ce...

But, there is a solution! One that they (Google) also clearly indicated in the blog post when they announced this whole thing. And that solution is: Install a custom rom which does not have these security features. Bam. Done! That's all it takes. If you care about your privacy you're already running a custom rom. And if you care about MTIMing apps, then installing a custom ROM is not that much of a hurdle either.

There's another solution: Avoid closed source applications and closed source OS.

There's other OS besides the ones you mentioned. And there's other small form factor computers besides mobile "phones" sold by telecom companies.

Ok so you don't have a point or anything you wish to discuss. You're just angry at the way the world works?

A million times as many people are being spied on without their knowledge than want to check their own data though. Not like these things were invented because of users, they were added to stop MITM by governments etc.

If you don't want to use OSes, devices and software that's designed to protect the average user from being hacked, then pick a system which gives you root.

There is also a way, on rooted Androids, to sniff SSL pinned Apps.

SSL pinned is not an protection for reverse engineering anymore, you may want to add this info on your post.

More info at

https://github.com/ac-pm/SSLUnpinning_Xposed

+1. Thanks for the link, I didn't know bypassing certificate pinning became so easy.

Here's a few other ways as well: http://blog.dewhurstsecurity.com/2015/11/10/mobile-security-...

I had to do this recently and found a great tool for Android for sniffing traffic on the device called Packet Capture. It can even sniff SSL without root permissions by installing a self-signed certificate and running an in-app local VPN proxy. It also had a bunch of other nice features like parsing common protocols, showing the good bits of HTTP, etc. Much nicer than the approach described here (this article is from 2013), although it's certainly only for Android folk.

I don't think it's FOSS, but hopefully a FOSS alternative will come along and use this approach.

I've been using Packet Capture to get some data I wanted out of an app recently and in the process I started generally snooping around. What I saw disgusts me. First, just how much waste there is in communication - so many requests with so much JSON traffic for no real reason (except maybe laziness). Second - just how much various apps report on you. I've seen everything I could possibly imagine the phone could know about me and my operator sent to the various "motherships", sans actually transcribing my contact lists and messages. I'm not even going to ask what they do with this data.

I recommend the experience highly. Just be warned, you may not like what you see.

Usually it's just usage analytics, but the potential for abuse is great because individual data is still sent.

For those looking for a fully native experience, give https://interceptapp.xyz a try. Currently in alpha. Feedback is welcome and appreciated.

Disclosure: I'm the developer.

Nice, I reckon you're targeting the primary 'benefit' sought by MITM'ers. Most of my Android MiTM'ing is because I want to get at the data behind the app. Trying to comprehend some of the crazy, ultra-nested JSON schemas that some app devs use is a PITA.

Just a suggestion: I know you mention it in one of the marketing lines, but it might be worth stating that it's for Mac in the headline (or in the download button). Otherwise people like me, who just breeze past the marketing material to the download link, will waste 6.8MB of your bandwidth (sorry about that...)

This article is from 2013. The ssl vulnerability mentioned is no longer present: https://www.charlesproxy.com/documentation/using-charles/ssl...

What are some other ways to prevent people from discovering your API endpoints?

One terrible idea I just had was creating only one publicly accessible API and then encrypting the actual endpoint in the payload which the server would decrypt and then redirect.

Don't rely on obscurity and obfuscation. Instead build your API in such a way that handing someone the complete documentation would have no negative impact. Assume the client is always compromised. My $0.02

One terrible idea I just had was creating only one publicly accessible API and then encrypting the actual endpoint in the payload which the server would decrypt and then redirect.

You just reinvented the TLS socket connection.

- encrypt everything using AES and pray that nobody will find the key in the apk - use your own crooked Http implementation that violates the official specs (e.g. missing \r\n's or introducing random whitespace characters...). Simple http parsing tools won't work anymore and the attacker will get grey hairs when he tries to test/use your endpoints

Just a heads up that changing the HTTP protocol can break a load balancer. Experienced this one time while using Amazon's ELB: the response had invalid HTTP and the ELB instances would silently lock up and stop responding, eventually draining the pool. Was not fun to debug.

Don't use a static key. At least use DHE or ECDH to negotiate the key.

I wonder how many techies could be silently MITM'd using the Charles root.

I worried about the same thing. Modern versions of Charles use a per-user root CA that is generated in the client.

Aha, much more sensible.

I can confirm Android detects it and sternly warns you (but allows you to keep doing it if you want).

Isn't it possible for apps for ignore the OS's proxy settings and make a direct TCP connection? In that case the proxy man-in-the-middle trick won't work.

Yes. Socket programming. A number of banking apps in the UK use the connect(2) etc, although I can't say if this is the reason. It's most likely to make it a touch more difficult to reverse than hooking NSURLConnection etc.

At least on Android you can generate a fake VPN-esque connection locally that passes everything through a proxy, so the proxy isn't exposed to the application

Sure but then the verification will fail since you won't be able to sign the handshake with the "pin'd" cert. (Assuming they implement TLS or other crypto in their own code.) If you aren't modifying the execution environment then it's possible for an app to be "safe".

An clever-enough emulator can just lie to an application and say, "You're running on a stock device. Everything is fine".

Clever-enough is the key word, with Safetynet involved, which dynamically executes signed classes and you don't know what checks will be done

In that case I would look into ARP poisoning.

this doesn't work anymore on iOS apps that use ATS. ATS is enabled by default and will be required by Apple by the end of 2016

Sounds like Charles can work with ATS, just not pinned certs.[1]

[1] https://www.charlesproxy.com/documentation/faqs/ssl-proxying...

I'm pretty heavy into home automation and I use this technique all the time to learn how to control various walled garden home automation systems. Even worked with my Alarm company's system.

However, if the phone app uses certificate pinning and SSL it doesn't work. (yes, that means my alarm company doesn't use certificate pinning).

Is there any scenario where snooping on the upstream network connection can give you valuable reverse engineering information? Because of latency/reliability issues, most mobile apps have pretty simple and stateless protocols with the server.

On android, this only works when the app is using http urlconnection provided by java. If the app uses apache http stack, the request is not routed through the proxy settings. Such traffic will not showup in charles or the great MITM.

Even on iOS it's possible to bypass certificate pinning by using a jailbroken device and tool. I wouldn't say that's a good meassure against reverse engineering.

That's what I call debug mode enabled in production. RESTful JSON APIs are truly in debug mode in production

What would you recommend in place of RESTful JSON API's?

Seems ignorant not to have a unique Charles certificate

Apparently modern Charles versions have a unique CA generated by each client.

Did someone just post this guys blog from 2013 up?

Reply on news.ycombinator.com