Hacker News Reader

Comment by martin_bech | Hacker News Reader

I’ve worked at a major ISP, for a decade, and spotting something like this should be so easy to spot. There are tools on monitoring of load all the time, and areas are routibely getting split etc. to improve bandwith, so I think your ISP are basicly amateurs..

View on HN

The problem is that most companies aren't going to tell you that their peering circuits are running hot or that their internal network or access layers to the end user are running warm at peak. ISPs all do stat muxing and the line is "we make money when customers don't use the service".

They'll be happy to deal with the last mile segment, but anything beyond that is murky and most companies I know aren't going to share much. Helps to have friends on the inside leak some graphs, though.

> I’ve worked at a major ISP, for a decade, and spotting something like this should be so easy to spot

MRTG graph, ISP circa 1995. Colorized.

See a flat line? that's congestion. Now figure out where it is coming from. Sorry, we have been doing this for thirty years so I'm kind of cranky. It is not a rocket science.

Alternatively, load has gone up across the board in a short period of time, so that preventive scaling has fallen behind and are in recovery mode.

Yes it can, but why would it take several techs, to spot something like load, which is the first thing you would do, it should take no more than 10s to look it up in a tool.

A "last foot" tech might not even have access to those tools, much less know how to use them.

Rolling out that tech has got to be more expensive than checking the load first.

Dunno how it is in the States, but here in UK rolling out the tech is basically the first thing they do after the unavoidable "have you tried turning it on and off again" phone call. They just don't trust the customer to have any clue and maybe don't want to waste time doing troubleshooting at their end when it's "probably" a downstream issue.

Network Operations should be raising known problem issues to front line call centre staff.

Network congestion issues shouldn't be handed off to field techs to check local loop (last mile) and CPE (Customer Premises Equipment.

I'm pretty sure it's standard practice at these companies to never let front line call center staff acknowledge known problems. Sometimes, the automated phone menu will give you a recorded generic message that they are currently experiencing a service issue, but that's intended to convince you to hang up and patiently wait for them to sort their shit out. I've never had a front-line rep be at all useful in diagnosing a real problem.

Yeah true.

I guess I need to remember the ISP I work for here in Australia (front line tech support, and then network operations physical security and infrastructure) was widely recognised as the best ISP in Australia multiple years running, so I shouldn't use it as a baseline expectation.

So how was life at internode or aussie?

Internode.

Yeah, was a good place to work. I was in their Adelaide data centres when iiNet acquired the company.

In NZ you sign up with an ISP, but your local connection is usually handled by the same physical equipment (DSLAM for ADSL, etc) which is owned by a single network provider.

I’m not sure what the incentives are for an ISP to try to get the provider to fix issues, or even if they would e.g. https://company.chorus.co.nz/what-we-do is notoriously bad for service and the copper network is being deprecated. Locally https://www.enable.net.nz/about-enable/ are doing a good job of service, because they are well subsidised by the government and seem to be effectively operated.

> There are tools on monitoring of load all the time

On some days my connection resets 5 times within an hour, which is quite annoying since retraining the connection takes a minute or two. When I call support about it they have zero monitoring in place that would let them know about the recent history of the connection quality, they can only do spot tests of SNR on demand, which of course doesn't show any transient events. According support forum posts of other users they'd have to explicitly enable "long term monitoring" based on user input to get that information.

Of course SNR line quality is an issue separate from congestion, but still, automatic monitoring appears to be limited.

how can i as a subscriber find out whats the capacity?

It used to be possible to determine the downlink capacity and even current usage with a DVB-C receiver and some Linux software, since DOCSIS is essentially just IP encapsulated in MPEG transport streams on a digital TV channel.

More recent versions of DOCSIS have moved away from that layer of backwards compatibility, so you would probably need some specialised equipment, if it is possible at all (I don't know at what layer exactly encryption happens).

Not amateurs, liars.

So Frontier?

Reply on news.ycombinator.com