Hacker News Reader

Comment by lxgr | Hacker News Reader

Having had to debug many of such cable issues in the past, it's baffling to me that cable companies aren't proactively monitoring for things like this.

They have all the data available on their end, as far as I can tell! (Unless DOCSIS modems somehow don't have a standard "signal receive report" functionality?)

View on HN

Telcos used to monitor their copper outside plant for moisture. This was called Automatic Line Insulation Testing in the Bell System. The ALIT system ran in the hours before dawn. It would connect to each idle line, and apply, for tens of milliseconds, about 400 volts limited to very low current between the two wires, and between each wire and ground, measuring the leakage current. This would detect moisture in the cable. This was dealt with by hooking up a tank of dry nitrogen to the cable to dry it out.

Here's a 1960s vintage Automatic Electric line insulation test system at work in a step-by-step central ofice. [1] Here's the manual for automatic line insulation testing in a 5ESS switch.[2] 5ESS is still the major AT&T switch for copper analog phone lines. After that, it's all packet switching.

For fiber, of course, moisture doesn't affect the signal.

This led to an urban legend: "bell tap". While Western Electric phones were designed to not react to the ALIT test signal, many cheap phones would emit some sound from the "ringer" when the 400V pulses came through, some time before dawn.

[1] https://www.youtube.com/watch?v=Wt1GGdDa5jQ

[2] https://www.manualslib.com/manual/2755956/Lucent-Technologie...

Great comment, thanks!

(I've sent a quick email suggesting it be added to https://news.ycombinator.com/highlights :)

If you're really into telephony history, the Internet Archive has "The History Of Engineering and Science in the Bell System" (3 volumes) online.

If you have to build reliable distributed systems, it's worth understanding how this was done in the electromechanical era of telephony, where the component reliability was much worse than the system reliability. "Number 5 Crossbar"[1] is worth reading, but hard to follow if you have no idea how telephone switching worked and are unfamiliar with the terminology.

Number 5 Crossbar, in current terms, was a collection of microservices. There was a big, dumb switch fabric, and "markers" which told it what to connect. Other microservices included trunks, originating registers (which listen to incoming dial digits), senders (which sent dial digits to the next switch), billing punches (which recorded toll call data for later billing), translators (which held routing tables), and trouble recorders (which logged errors.) Central offices had at least two of each resource, for redundancy. Resources were "seized" as needed from resource pools, with a hardware timeout and alarms to prevent resource lockup. If something went wrong in setting up a call, it was retried once, using different resources. If it failed on the second try, the caller got a fast busy and there was an alarm and a trouble recorder dropped a trouble card. Markers did not have persistent state. They started each call with a reset. So they could not get stuck in a bad state.

In the entire history of the Bell System, no electromechanical switching office was ever down for more than 30 minutes for any reason other than a natural disaster or a fire. It's worth understanding how they did that.

[1] https://telephoneworld.org/mdocs-posts/number-5-crossbar-sys...

Not truly related to the post content, but there is something about the way these old manuals are formatted/printed that immediately inspires confidence in the contents.

Maybe because you know that someone spent a lot of time on it before it was published since no adjustments could be made after the fact.

> trouble recorders

This feels like a term a sci-fi author would invent in an alternate history setting to replace "error log" and I find it very humorous.

No, just practical.

The previous version was a panel of blinking lights called the "trouble indicator". When an alarm sounded, someone had to go to the panel and record by hand which lights were on. There were about 200 lights. So the trouble recorder, which recorded that info automatically, was added in larger central offices as an upgrade.[1]

[1] https://hackaday.com/2022/12/02/stack-trace-from-the-1950s-p...

> Wait! Was that an old adding machine?

At 02:40.

And yes, it is an adding machine.

Ah, so that's why there were always nitrogen tanks on NYC sidewalks.

Yup, here's a tom scott video on the very same: https://www.youtube.com/watch?v=juZqGU9iuq0

We still have a land line. When a call comes through the phone often gives a gentle "peep", then a pause, then goes full-on ring. I've started to react to the "peep".

But every evening, mostly around 21:00 or so, the phone gives a gentle "peep" without then ringing.

I wonder if it's a line test?

Crap electronic ringer, probably. If you put a scope on the line, you should be able to see what's happening. Remember to be prepared for higher voltages, up to 400V.

There are various weird, obsolete signals in analog phones. Ring pulse alerting signal. ALIT test. Polarity reversal. Ring to ground. Ground start. Caller ID (1200 baud FSK between the first and second rings) DSL. Basic talk and ring was standardized around 1900, and everything else is backwards compatible. Ringers are supposed to ignore all that stuff. People who implement Asterisk PBXs are into this.

Here are some actual waveforms, if anybody cares.[1]

[1] http://www.adventinstruments.com/Products/AI-5120/Screenshot...

In my observation, to a first approximation, cable operators take off-the-shelf equipment, connect it, power it on, and bill customers for it. They don't really have the r&d capability to innovate and create new monitoring solutions quickly.

It might happen that an equipment manufacturer sees an opportunity and builds something, but then they have to go into a long sales cycles to convince operators to use it. Operators are in a duopoly situation in most places, so quality of service is kind of a secondary concern for them - customers may get annoyed, but as long as the competition is not vastly superior, few actually switch. It is not a market prone to innovation.

Reply on news.ycombinator.com