Hacker News Reader

Comment by peterwwillis | Hacker News Reader

Firewall packet inspection is basically just unpacking protocols layer by layer. Once you find the TCP or HTTP layer, you see if what follows looks like a WebSocket, which has a unique fingerprint and predictable elements. You can also look at streams and find patterns, like how one packet looks like HTTP, and then the following ones on the same stream don't look like HTTP. Initial versions may not be very accurate, but on "extremely paranoid" settings you can simply reject traffic that seems suspicious.

Also, having the DoD as a client will buy you some serious R&D time.

View on HN

You can't unpack protocols layer by layer when they're encrypted.

True, you would typically use machine learning, which can be used to identify and separate layers in protocols.

Machine learning on random data won't get you anything.

And is machine learning fast enough for live network filters?

It was fast enough 10 years ago. And it worked on random data. Luckily though, internet traffic is not random data, and encrypted internet traffic is much less random than you'd think. Of course a lot of it involves inferring information about different parts of flows, but high accuracy is not difficult. (What's difficult to detect is traffic you haven't got any trained models for)

If it's actually random data then no it won't work. Actual random data is 100% noise and 0% signal. If the ML thinks it's picked up some signal, then it's wrong.

You're right that TLS is not 100% random, there is some information to work with: the size of the encrypted blobs (they're rounded because of padding) and the timing of them. Are there any commercial firewalls that do analysis of this kind for the purpose of blocking the traffic? It wouldn't be able the block all the traffic, because it would have to wait a while to get more timing data that it can apply its heuristics, then end the connection.

And it wouldn't be able to unpack protocol layers.

You're right, I misspoke, "random" data is nothing, but "random" as in "arbitrary selections of network data" it will work on. And no, it doesn't unpack layers per-se, but it detects signatures and can be used to find patterns and eventually separate layers of protocols.

The two oldest and most successful methods that I know of are matching on payload length and models trained on the initialization of network application protocols, neither of which requires constantly re-sampling and re-classifying to get a hit. And blocking traffic is often more about terminating an existing connection once you detect something bad going over it (deep content inspection).

Yes, commercial firewalls do look for tunneled applications. Palo Alto Networks has a patent on it (App-ID), Websense/Forcepoint does it (Content Gateway Analysis), Cisco sort of implements it (Network Based Application Recognition/Application Visibility and Control).

A bunch of open source software implements it, too. Some commercial proprietary software is also out there, with PACE leading the pack. Wikileaks has one of their product data sheets: https://wikileaks.org/spyfiles/docs/IPOQUE-PACEProtAppl-en.p...

Honestly, a lot of customers simply force proxies on their users and inspect all their traffic and drop anything that it can't inspect, so there probably isn't a lot of commercial need for this. But it is out there.

Random data, but what about traffic patterns and timings? I am pretty sure DPI engines are not just based on the packet's data anymore.

Reply on news.ycombinator.com