ISP Advertisement Injection - CMA Communications | Hacker News Reader

ISP Advertisement Injection - CMA Communications

zmhenkel.blogspot.com

140 pointscramerica78 comments

I posted about this on reddit a few weeks ago[0]. Someone in the thread said they had contacted the Better Business Bureau, but I'm not sure what their process is or how far it's gotten.

There has also been a short email thread in which their official response is this:

> Mr. [redacted],

> CMA is in the process of trying to find ways to drive income from our internet service in new ways. These new ways would allow us to expand our service offering and maintain the cost of the current residential and business internet services.

> We’ve been testing a new service which allows us to overlay / insert some local advertisement on certain web pages. A company called Route 66 is our partner. Right now, you’re barraged with a lot of internet advertising, popups, etc… This has become part of the internet experience. At the core, we’re simply trying to better customize some of this experience. And possibly give you access to highly relevant local advertising.

> Having said that, I’ve recently become a little more familiar with what some of these ads look like and how they operate. I will concede that I’m not sure they strike the perfect balance between being information and non-invasive. Like I mentioned, we’re involved in a test and the feedback we’re getting from the test is helping us to refine and improve how (or if) we’ll continue here. So I’m stopping short of saying that we’ll be ceasing this type of internet advertising experiment. But I do want you to know that your feedback has resulted in the beginning of a pretty intense internal dialogue.

> Thanks for your feedback.

> [redacted]

> CMA Communications

It's absolute insanity and a major breach of trust that they'd inject their own content into webpages I visit. I'm permanently using a remote VPN for all outgoing traffic through CMA.

[0]: Didn't know exactly where the post belonged, so I put it in /r/self: http://www.reddit.com/r/self/comments/19zhl6/my_isp_is_injec...

The kernel of their user-hostile decision is right here:

Right now, you’re barraged with a lot of internet advertising, popups, etc… This has become part of the internet experience. At the core, we’re simply trying to better customize some of this experience. And possibly give you access to highly relevant local advertising.

"Hey, we're just going to take a piece of the shitty ad-laden pages you visit. Maybe you'll see something you like, but chances are low and beside the point."

The BBB has exactly 0 power over anyone, so I really doubt that will fix anything.

Complaining to the BBB is slightly less useful than posting an angry comment on a YouTube video.

Addendum: as soon as they started doing this, all of my port forwarding broke. I'm not sure if this is a coincidence or if it was on their end, but the technicians were of no help.

Immediately cancel your account and switch. If you are forced to use them as an ISP due to municipal/geographical regions complain to your city manager.

The only way to slap these companies back into line is with your wallet. If you can't do that then a couple complaints to the city manager can go a lot farther than you think, especially in smaller areas where there isn't a lot of support staff in city hall.

Considering just how big the outcry was over ISPs delivering advertisements instead of an NXDOMAIN response, and how many ISPs are continuing to mess with DNS over ten years after the execrable practice started, I haven't got much faith in the power of the market. Has there been just one ISP that stopped hijacking DNS over customer complaints?

Sonic.net ftw! Not available everywhere but great ISP who stands up for a free and open Internet.

Looks like Sonic only provides service in CA.

And as far as i know, only in certain parts of the bay area. I wish i could get it in cupertino.

"knowingly and with intent to defraud, accesses a protected computer without authorization, or exceeds authorized access, and by means of such conduct furthers the intended fraud and obtains anything of value, unless the object of the fraud and the thing obtained consists only of the use of the computer and the value of such use is not more than $5,000 in any 1-year period"

Injecting or replacing ads in other people's content on the wire: 'knowingly and with intent to defraud', 'exceeds authorized access', 'furthers the intended fraud'

Ad revenue from doing so: 'obtains anything of value'

Forget copyright infringement: a case could be made that CFAA applies here.

Wouldn't this depend on the TOS of the ISP? Perhaps by signing the contract you "authorise" this.

This sort of thing doesn't surprise me any more. AFAIK DNS on every major ISP in the UK is broken, there is no NXDOMAIN. Unresolvable domains are simply redirected to a specific IP address which happens to host a page of ads and a search bar on port 80. This might not matter to most people, but it's a huge PITA when I'm testing some things.

I'm not aware of any similar arguments being tested in court, so this is all conjecture.

That said, barring an explicit definition of 'Internet service' in your service contract, it's commonly understood that requesting a page from example.com, all the data your ISP returns implicitly is sourced from example.com. Introducing your own content in between is therefore fraud, as you've mis-represented the origin of the content.

I believe the owner of an involved web server would have standing as well, not just the users.

But in such a case you could make the same argument for ad supported wifi connections or proxies.

You do enter into agreement (usually), but do you have a valid contract when using those connections?

FWIW, I haven't noticed this on any of BT, Sky or Virgin.

NXDOMAIN is hijacked on Virgin Media for sure, unless you are using some other DNS. I haven't noticed them injecting ads though.

Not to be needlessly contrarian, but this doesn't appear to be the case - at least for me, for any of the domains I tried:

>nslookup notarealdomainthatshouldresolve.co.uk Server: cache1.service.virginmedia.net Address: 194.168.4.100

* cache1.service.virginmedia.net can't find notarealdomainthatshouldresolve.co.uk: Non-existent domain

NXDOMAIN used to be hijacked by Virgin Media. But it certainly is not any more, unless their DNS service changes significantly across the country.

Maybe we should all start packing our own TOS's in request headers... :)

They go as far as even replacing existing ads with their own, this seems criminal, especially when they are directly impacting google/microsoft/apple by removing their ads and replacing them with their own.

A picture speaks a thousand words here; the author did a great job supplying plentiful screenshots to emphasize how wrong this practice is. I read about this in the past but wasn't too moved until I scrolled through all those screenshots and thought, wow, this is not good for ad publishers OR brands OR anybody. This is only good for the greedy ISP.

This is an interesting problem. On a broadcast channel, when a local station attempts to replace the ads the network has put in their shows, with their own ads, the network has some leverage to shut down that process. But on the Internet there are a bunch of web sites and they don't have any leverage at all. They could do an IP filter, which is to say put up a page "This site unavailable on this ISP's network" when a request came in from a CMA communications IP block. That would cause a support headache for CMA with all their customers calling into complain. The other defense would be to create a web page that doesn't cache (it pulls the actual page content through AJAX calls gets around any local script injection). Lastly there seems to be "product" here where you bundle up an EC2 instance and some friendly software on the PC that spins up a VPN tunnel for all of your traffic.

"on the Internet there are a bunch of web sites and they don't have any leverage at all"

They could use TLS...

HTTPS everywhere would solve this, and the Comcast Javascript injection - I wonder how many more people will deploy things like this before that happens?

You shouldn't have to encrypt your data to stop your ISP from actively 1) scanning and 2) corrupting it.

What is the FCC good for?

Indeed. What's with the Internet being a set of tubes?

And tubes based on the honor system at that.

If you don't encrypt your data and verify the sites to which you connect (both of which https does) then anybody between you and them can intercept and alter the transmission.

I agree you shouldn't have to do it, but you need to worry about more than just your ISP.

Just assume any unsecured internet connection is actively hostile, and you'll be better off.

It is not just about your ISP, it is about all the ISPs between you and the website.

Transit and peering providers aren't going to do that. Certainly not tier 1 networks.

1) They can't get away with it 2) That'd slow everything to a crawl, inspecting traffic is expensive at the scales most large providers operate at.

You shouldn't have to, I agree, but apparently you do have to. "Should" counts for nothing.

We hope ads/no-ads arms race would end there. But I could easily see some unscrupulous/greedy ISPs then resorting to setting up SSL proxies to MITM your ostensibly secure traffic, as some private organizations (schools, corporations) already do.

That's a good way to get yourself the CA death penalty.

So? If Comcast does it, what are they going to do?

A few financial institutions holding large quantities of Comcast's paper (and their executives' IRAs) pick up the phone and explain how they feel about having their secure websites' identity impersonated.

They'd have to have their certs installed on your computer, or be an existing CA. Schools and corps (including the one I work for) can do this because they have admin control over destination machines.

You're absolutely right, I didn't mean to imply what schools & corps currently do was shady in any way (as long as you're aware that they're doing it).

Actually, the "HTTPS Everywhere" plugin [1] for firefox and chrome is an incomplete solution. The reason is simple; not all sites support HTTPS, so the plain HTTP-only sites are still vulnerable.

Injecting a script into insecure HTTP is just one of many abuses possible by ISP's. Replacing images on the fly is another. Recompressing (degrading) images/video is another. Messing with DNS responses is another, and so on...

A far better working solution is to use a VPN service since when it's configured correctly, it will encrypt all traffic passing through your ISP. Of course, this is really just moving the trust problem, rather than solving it, but at least using a VPN service makes it your decision who to trust. I use Tunnelr.com [2] since by reputation, similar interests, and years of traded emails, I know the people who run it.

[1] https://www.eff.org/https-everywhere

[2] http://tunnelr.com

tunnelr.com... "Powered by OpenBSD"... I like it already.

I meant actually demanding that content providers deploy HTTPS everywhere instead.

Sorry about that. It seems my reading comprehension skills are still flawed. ;)

And here's your annual reminder of the upside-down-ternet http://www.ex-parrot.com/pete/upside-down-ternet.html

It would solve it for many sites, but there are plenty that don't have full https support (and some that have none at all)

https is the wrong solution. that is for preventing others from seeing what you're sending/receiving, not verifying the integrity of what is sent/received. well, it does do that too, but it adds extra unneeded overhead by encrypting everything. Besides, the ISP can easily man-in-the-middle any connection you make and then inject their ads into the webpage, even if you use https.

The correct solution is signing the webpage (but not necessarily encrypting it). More technically, that means the server/website would hash the source of the webpage, and then send the webpage, the signed hash, and if needed, the cert it used to sign the hash. Upon receiving both the webpage and the signed hash, the browser would then check to make sure that the signature can be trusted (using a chain of trust the same way we do with certs for https pages already), hash the webpage source it received, and then verify that that hash matches the signed hash it received from the website.

It doesn't matter if any of that is sent in plaintext, because there is no sensitive information, and as long as the hashing algorithm used is strong (ie sha2 family, not md5), then the isp can do fuck all to inject javascript.

Ha-ha, and let's call this HTTPSEC http://cr.yp.to/talks/2013.02.07/slides.pdf !

(Spoiler: HTTPSEC is not a real thing, it's what we'd had if people who invented DNSSEC, or people like the parent commenter, designed something like TLS).

wow, that is an annoying slide deck. a lot of the issues seemed to be dns-specific that don't exist with http (slide 110: "Attacker forges many UDP request packets from victim’s IP address to many HTTPSEC servers.", also exist for https (slide 106: "Each HTTPSEC key/signature is another file to retrieve. Often your browser needs a chain of keys from several servers. Could be a serious slowdown."), or were self contradictory (slide 13 says httpsec "allow[s] for verification of the origin, authenticity, and integrity of data' obtained through http", but slide 123 says "The data signed by HTTPSEC doesn’t actually include the web pages that the browser shows to the user". I think the HTTPSEC you are referencing is partly a straw man, because that httpsec does the opposite of what I proposed, which was signing the webpage itself, not the redirect info. I do have some questions about issues brought up in the slides:

is the delay/cost of signing versus encrypting data really so huge that it's infeasible to sign dynamic pages?

also, why does each non-existent http page need it's own 404? Wouldn't a static 404 response be just fine?

It was meant to be a jab at DNSSEC, so yeah it's probably nonsensical if you try to read it as a real HTTP proposal.

If TLS could negotiate certificates instead of supporting one and only one, the backbone of any sane "virtual host" system, then https: wouldn't be a big deal. It'd be the default.

Now you need a separate IP (expensive) or port (annoying) for each virtual host configured with a different SSL cert. This has to stop, but it will not be easy to fix.

RFC 3546, which includes TLS Server Name Indication, has existed since June 2003. The problem preventing deployment is the lack of client support, especially Internet Explorer on Windows XP and Android 2.x [1].

1. http://en.wikipedia.org/wiki/Server_Name_Indication#No_suppo...

> Besides, the ISP can easily man-in-the-middle any connection you make and then inject their ads into the webpage, even if you use https.

How would they do this without triggering certificate warnings? Or are you simply saying that everyone ignores certificate warnings?

"You're almost done setting up your internet connection! For your security, please add our CA to your computer. This handy-dandy program that you can download from our website or from a usb stick our installer has with him/her will automatically do this for you.

At ShitISP we care about your cyber safety. In order to prevent viruses and other Bad Things from infecting your computer and the other computers on our network, you will be unable to do some things on the internet until you install our certificate.

Thank you for helping us keep your computer and our network safe!"

> adds extra unneeded overhead by encrypting everything

Baloney. :-) Or rather, please cite something in the last 5 years showing that the overhead of the symmetric encryption is a significant cost in HTTPS.

assuming you just want to browse websites and not transfer information that needs to be private, the extra rtt for establishing the ssl connection is the uneeeded overhead. for something like streaming a movie, that won't matter because the ssl connection will be long lived. for something like web browsing, where most websites require a new tcp connection for each GET, ssl is painful.

sure, the extra rtt is preferable to javascript injection, but signing the webpage is sufficient to prevent javascript injection and it wouldn't add extra rtt delay (aside from fetching a cert in the trust chain, which https can also suffer from in the exact same way).

depending on the algorithms used, on-the-fly signing of dynamic pages might be (read: almost certainly is) more painful than ssl/tls in terms of computation time, but to the user would still be quicker for most cases than the rtt delay added by ssl/tls.

Note that the primary HTML resource for most web pages takes multiple roundtrips to transmit completely. If you sign it only at the end, you've made the browser feel slower. So you'd have to avoid the RTT by accepting, processing, and presenting the data immediately but only later authenticating it retroactively. The bad guy can just drop or delay the legitimate signature.

This is inherently error-prone. It gives the developers a big, convenient, and reassuring assumption which the attacker is able to violate. For a complex and evolving endpoint like a web browser, I don't think you'd ever see the end of security bugs. More: https://www.ietf.org/mail-archive/web/tls/current/msg04017.h...

Furthermore, retroactive authentication still doesn't preclude the encryption: https://www.ietf.org/mail-archive/web/tls/current/msg08722.h...

The issues with bad guys dropping the sig or delaying it also equally applies equally to ssl during the handshake. But waiting for the sig and the whole page to arrive combined wit the size of modern webpages is a problem I hadn't fully considered. I suppose per packet signatures might work to fix the delay issue, but then you'd have to violate abstraction layers and the added cpu time would be completely untenable. At that point you might as well just copy ssl's dh handshake followed by a block cypher but only provide integrity and not privacy, which is stupid. yeah, ok I'm convinced, stupid idea due to practical reasons. just use https with the extra rtt's.

It's not a stupid idea, it's a problem that smart folks have been banging their head against for a long time now. Take a look at the "TLS Snap Start" proposal to see the lengths to which one must be willing to go to avoid that round trip.

But some low-hanging fruit remains. Improvements to clients and servers that increase TLS session resumption rates would help too.

If the ISP can MITM the HTTPS connection (hint: they can't because they can't provide a valid certificate which matches the domain which has been signed by a trusted CA), they can MITM any signing system you may come up with.

Google has already provided statistics showing that HTTPS adds a negligible amount of CPU load to servers (and most websites aren't CPU bound anyway).

Technically, isn't this copyright violation? People who inject ads into a page are creating a derived work without permission of the rights holders. I'm sure Apple didn't okay that addition to their HTML.

It would be interesting to see a lawsuit along those lines.

If they had only replaced ads and not created new ads at the bottom of the page, nobody would have noticed.

It's been long enough for me to state, but I used to work for a contractor hired by CMA Communications.

ISPs of this size try and maximise as much profit out of their customers and being that a lot of CMA's sites were over provisioned and are barely able to provide telephony service without incompetence-y along the way, it is not shocking that ads being injected into pages is a new thing for them.

To see these bullshit ads showing up on random pages is far from surprising.

The first screenshot has the Google squiggle. Am I right there? That somehow Google ads are being injected?

Because that seems like something that Google would not tolerate.

This was inevitable.

That's why I bought Google stock after they got into Android, as Android makes it possible for Google to now step in & protect against the MITTM attacks by ISP's blocking their ads. The OS gets the final word before it displays content to the user & it can detect & block these.

Now, they just have to deploy the fix to Android...

And what happens when Google are the one doing the MITTM attack?

It all comes down to who do you trust.

Has Google ever modified other websites' pages like this?

Yes, I believe they call it Adsense.

https://en.wikipedia.org/wiki/AdSense "How AdSense works

(1) The webmaster inserts the AdSense JavaScript code into a webpage."

So this is the webmaster modifying his own page. It's not Google injecting Javascript into someone else's page like CMA Communications or Comcast is said to be doing.

The sad thing about this is in many places (at least in the US) there are few if any alternatives. In my city, I can either get Time Warner Cable or AT&T DSL. I'm 20 miles from Verizon's office but FIOS is illegal in my city. So if the ISP starts screwing with the content then you have virtually no alternative.

I'm interested to know why FiOS is illegal in your city? I don't know of any regulation that would make it.

Some odd law in Texas. But I think FIOS is no longer being expanded anyway in favor of wireless.

Sprint is another (lesser?) offender -- their mobile broadband injects a script in every page that loads compressed versions of images until told otherwise. (Annoying, but at least it's well-intentioned.) I've long since blocked the IP, but it gave me a bit of a scare to see unfamiliar code in my own websites.

Ugh, I hate that attitude of "Macs can't get malware" that this guy exudes so smugly.

Yep. I rolled my eyes at this, before I even got to the meat:

"I laughed to myself briefly, thinking: “who uses Bing?”, and then realized I was a computer science grad student who had managed to get malware on a Mac, so I wasn’t in a position to judge."

My parents use CMA Communications. I can confirm that they receive injected ads as well.

I looked into switching then to another ISP, but the only one available is 1.5Mbps DSL vs their 15Mbps connection now.

Top contender for "worst publicity"

Damn, that is low. Time to get a vpn.

Another reason to use ABP...

Every website should use https

Reply on news.ycombinator.com