Privoxy, a non-caching web proxy with filtering capabilities | Hacker News Reader

Privoxy, a non-caching web proxy with filtering capabilities

privoxy.org

78 pointsmutin-sa29 comments

I used this maybe a decade back on Windows. Currently using uBlock Origin + Privacy Badger + DNS blocking over dnscrypt-proxy (gives me a local caching resolver as well), but I miss Privoxy.

Me too. As I recall, I used it to force proxification of traffic for applications that didn't normally support proxies (although I may be mistaken...). Great bit of kit.

I am stopped using privoxy for adblocking few years ago with widespread HTTPS adoption. Their filters + small number of custom rules provided good protection long before adblockplus.

No doubt based on or inspired by http://en.wikipedia.org/wiki/Proxomitron , which does have HTTPS MITM capability --- the one thing a lot of filtering proxies lack. There's unofficial (as they all are --- the author is long deceased) patches to add support for the latest OpenSSL libraries and other minor useful enhancements.

Privoxy was based on Internet Junkbuster[1], whose last public release (1998) was a year before the first release of Proxomitron (1999).

[1] - https://en.wikipedia.org/wiki/Internet_Junkbuster

From 2015: https://news.ycombinator.com/item?id=8822974.

it actually becomes less and less useful nowadays, due to the fact that most sites are running https instead of http. privoxy can not proxy https well.

Last updated over a year ago. Is there any benefit to using this over something like Pi-hole?

Can't quite remember the setup but I used to use Privoxy on the N900 to route my traffic through a remote connection while mobile. VPN/SSH tunnel style.

I do use Pihole, but I think this is more versatile. At least it would be if mobile devices were more open... grrmble.

I am still using it but it becomes obsolete fast as more traffic becomes HTTPS.

You can pair it with something that generates https certificates on the fly, like mitmproxy.

It is generally easier to use Squid in reverse mode. It is a production quality proxy that can do reverse proxying. You can generate a CA cert, install it onto Squid chain it to privoxy and install the CA cert into your browsers.

Mitmproxy is great and we use it a lot, but it will have a pretty big hit on your performance. They have a more streamlined tool, mitmdump that is more aimed for these use cases, but it is still not written for performance first.

https://www.cyberciti.biz/tips/linux-setup-transparent-proxy... https://wiki.squid-cache.org/Features/DynamicSslCert https://wiki.squid-cache.org/Features/SslPeekAndSplice

That link is an exact recipe for what I am talking about. It isn't really trivial to setup, but it does work well. Then, you can transparently privoxy ALL of your connections. You have to mix and grind several ingredients together.

Specifying proxy is always easier/better, but transparent with upstream (from squid) proxies is possible. Squid can also do much of the ad blocking as well like privoxy, just without as nice of a config look and feel.

edit: Just keeping things simple, I use squid with dynamic ssl certs, non transparently and have most of the rules I used in Privoxy working fine on Squid as the "one proxy" to run browser through. This prevents needing to monkey with proxy rules. As a bonus idea I also run this Squid on a perma-privacy-VPNd box, and always force certain site traffic through using a browser configuration like ProxySwitch Sharp. If you are really paranoid you can keep your sensitive traffic through your home / more trusted connection (whitelist style) and then route everything else through Squid/Privoxy.

Setting up a nice little Linux VM that simply can't route traffic without the VPN connection is very nice, it ensures traffic thru your little VM can't leak if something on the VPN fails.

Is there anywhere I could read more detail on this setup? I'm not exactly clear on how it works.

Do you have:

A: client -> squid -> privoxy -> internet

B: client -> privoxy -> squid -> internet

And some more details on the exact squid & privoxy setup would be nice too.

I've long wanted to get privoxy working over HTTPS, and would love to know the details of how this is done.

Direct proxy, no transparent, client -> privoxy -> squid -> http(s) sites. I trust Squid at the tip a lot more. Forwarding Privoxy to another proxy is easy. Read the Forwarding section of the Privoxy manual. Setup Privoxy how you want. Forward to Squid. Follow: https://wiki.squid-cache.org/Features/DynamicSslCert

You should then have the perfect proxy centipede. For bonus fun use dnsmasq and blacklist domains via DNS too.

Things get much more complex if you want to do things transparently. Now you have to monkey with iptables. The idea is similar though. Start with Privoxy getting all HTTP traffic on a router. Forward to Squid. Have Squids dynamic SSL cert on your client devices.

So is privoxy happy handling HTTPS requests as long as it's fronted by Squid? I guess I'm still not clear on how/why that works. I thought privoxy could only handle HTTP.

Only usefull if Privoxy also validates the certificate and aborts the connection if not valid. Otherwise you might get MITM without you knowing.

The best scenario for using privoxy I have found were complining it '--with-compression' option and setting on my private VPN server that my iPhone had been connecting to.

There was a site with stock quotes that I have been refreshing hundreds times a day so I set privoxy to strip all unnecessary crap like ads, javascript, some gifs etc. and then compress. From 2MB of original HTML page size I went down to 15kB. Times hundred times a day - it had saved me a lot of mobile data.

Pihole solely works by intercepting dns requests and returning fake ip info for "known ad serving hostnames".

This is a proxy that works as a "man in the middle"

So, for example, it could block ads served from the same domain as the content or even in the content by using a regular expression that matches the <div> with the ad in it. Like the ads embedded in a google search result.

Is there a similar project that does dns filtering? That would workaround HTTPS.

Run your own DNS server, such as dnsmasq. For example I have a homegrown script that gets a few adblocking lists and constructs a dnsmasq config out of them (dedupe + merge + map to black hole IP).

dnsgate does that: https://github.com/jakeogh/dnsgate

This is roughly the equivalent:

https://pi-hole.net/

A decade or so ago when I was working on residential gateway/set-top box project, I used Privoxy to filter certain words from URLs via C/Linux programming.

Privoxy was an offshoot from IJB, the Internet Junkbuster which I guess started around 1996.

If interested in reading about the aversion to web ads and concern over privacy in the 1990's and early 2000's, check out IJB's old website. It is loaded with information. The IJB code is still around in at least a few places. While I prefer local DNS and local SSL-enabled proxies to filter ads, believe it or not ijb, as old as it is, still compiles and runs.

To quickly get the list of urls from archive.org, something like:

        curl -o ijb.urls https://web.archive.org/cdx/search/cdx?url=www.junkbuster.org\&fl=timestamp,original

        exec sed '
        /^[12][0-9]* h/!d;
        /^[12][0-9]* h/{
        s/^/http:\/\/web.archive.org\/web\//;
        s/ /\//;
        #^M is "\r"
        s/^M//g;
        }' ijb.urls

I always had a really tough time configuring this in a way that worked for me when I tried it. It's hard to see what it's doing. But, useful if you can set it up to work for you.

Isn't privoxy native code?

I wouldn't recommend running this, as it most likely is an excellent way to get a remote exploit on your machine.

Can you elaborate on this? What do you mean by native code, other than it being C? Why would a loopback listening daemon be an excellent way to get a remote exploit?.

It parses and rewrites HTML in C. And it is an esoteric tool. Throw AFL at it for a day, 100% chance of segfault.

You can download the source, take a look at it, compile it, and use it.

Reply on news.ycombinator.com