Client-side proxies – a better way to individualise the Internet? (2000) | Hacker News Reader

Client-side proxies – a better way to individualise the Internet? (2000)

people.dsv.su.se

43 pointsuserbinator11 comments

Client-side proxies were a great deal of fun. I started in 1997 with the idea that I could intercept java applets and inspect/modify the byte code. For example I scanned for "new Sound()" and created a gui frame with play/stop buttons for each sound object created. I also blocked some URLs from loading (e.g. ads).

By 1998 I realized that the ad blocking part was much more interesting to me than modifying java applet bytecode, and I started modifying HTML and other content. Instead of just blocking the ads, I removed that HTML so that the layout would change. But I also experimented with altering images, so for example I changed Slashdot's color from green to blue by changing the bytes in the GIFs. I altered headers (e.g. blocking cookies), and also used the Referer header to change the page (e.g. highlighting search words if the Referer was a search engine url). Seeing that I could make the web much faster, I started doing other things, like caching and DNS prefetching (long before Chrome existed). DNS lookup was really slow, in part because Netscape used a single thread implementation (if I remember right).

By 2000 I was using the proxy to implement parallel DNS lookup, chunked encoding, gzip encoding, keep-alive, request reuse. Most of these weren't yet available in browsers, but I implemented them in the proxy and presented a regular HTTP/1.0 connection to the browser. For example if the browser saw three GIFs to the same URL, it'd request that three times. The proxy would intercept these, make one request with gzip, and then feed that data to the three connections the browser made. Web surfing was so much faster with this proxy.

I continued using my client-side proxy for many years after that but over time it became less appealing and I no longer use them today.

I have my proxies written up at http://theory.stanford.edu/~amitp/proxy.html ; also see WebCleaner, a proxy based on mine https://webcleaner.sourceforge.net/

Interesting how the world has changed since the 2000s here - nowadays the ecosystem is far better, so it's much easier to set up tools to mess around with this, but the use of HTTPS everywhere makes it more difficult in more advanced cases (e.g. you'll often need to fight certificate configuration in individual clients).

In part because of that, browser extensions have become the main way to go for this kind of local web modification, but now there's new restrictions slowly coming in there too.

If you want to mess around with HTTP-level rewriting for yourself though, I maintain a Node.js library for easily writing tiny custom HTTP & HTTPS-intercepting proxies that makes it very easy: https://github.com/httptoolkit/mockttp/. Others have built more specific tooling on top too, like this web page modification proxy: https://github.com/OnkelTem/wmod-proxy

There's a walkthrough for setting up a quick local proxy & rewriting your own browser traffic here: https://httptoolkit.com/blog/javascript-mitm-proxy-mockttp/

Personally using client-side forward proxy every day for many years. In 2000, probably would have thought this would use too much system resources. Today, the resource usage is next to nothing.

I've been using Proxomitron since the turn of the century. Even back then it didn't use much resources, although as with all tools that use backtracking-based matching engines, pathological cases of complex filters can result in https://en.wikipedia.org/wiki/ReDoS .

FWIW, I always compile haproxy without PCRE. Have never been interested in backtracking or PCRE. I rarely need more than BRE.

Which tool do you use for the proxy? Do you do any content transformation?

Among the programs I use on a daily basis are socat, stunnel and haproxy. Content transformation is straightforwardwith haproxy+lua, and I continue to experiment with it, but in practice I usually modify JSON and HTML outside of the proxy using unix filters since I rarely use a graphical browser; I make many HTTP requests outside the browser and read HTML with a modified text-only browser. I am not a huge fan of browsers, TBH.

One limitation of haproxy is it only supports SOCKSv4. To get SOCKSv4a I use socat.

I think extension service workers might be the modern replacement that can deal with SSL https://developer.chrome.com/docs/extensions/mv3/service_wor...

tedu did a couple of such proxies:

- https://humungus.tedunangst.com/r/miniwebproxy

- https://humungus.tedunangst.com/r/medium-rare

Go makes it relatively easy to intercept TLS/HTTPS traffic with nothing but what's in the stdlib; you just need to teach your browser to trust your own CA.

I've done an intercepting proxy as an exercise, just to log all traffic and... It's scary, e.g. Chrome sends telemetry even when just looking at a new tab page.

Remember when "user agents" were supposed to fight for the users? I think this idea is even more relevant today than it was in 2000.

I ran Junkbuster back in the day, back before browser extensions that could do a better job existed, and I still run Squid to this day configured for large objects and long retention times, to cache my apt traffic.

Reply on news.ycombinator.com