I also worked at an ISP. Customers would sometimes call to complain about how much we must suck because their shiny new 56k modem wouldn’t train up past 28.8, and it couldn’t possibly be on their end because they lived in a neighborhood of brand new McMansions. That’s when we had the diplomatic mission of explaining how Ma Bell saved a whole lot of money at the expense of the customer’s phone quality.
As long as we’re swapping horror stories, I can summarize another one in 1 word:
Winmodem.
shudder
Did residential dial up support from 95-00, and got to experience all those disasters.
Another fun one is when you serve at the edge of two different phone companies and they didn't have enough cross connect service between the two for all the long running dial up connections. Yea, your modem banks aren't full, but your customers are getting busy signals.
Memories! An old job bulk-bought a ton of cheap "home" PCs (Compaq Presario) to use as Citrix desktop clients; the included modem was a winmodem. We discovered the CPU in the PCs couldn't reliably run the Citrix software and maintain a modem connection at the same time. We ended up having to buy external modems (and later network cards) for all the presarios.
Thankfully we learned from the experience and stuck purchasing from the "business" product lines after that.
I worked for the first commercial ISP in Hungary in 1995. At the time phone lines were still a hot commodity -- it takes a bit of time to fix decades of neglect, after all (Hungary was a Soviet satellite until 1989 and the wait time for a landline was ~ten years in the 80s). So supply side for lines was low and prices were high. On the other hand, the potential income for the ISP was limited because, well, it's 1995, not many people or businesses even know what Internet is good for. So how on earth did they afford the necessary many lines?
Well, see, they didn't. The guys running the place got wind of a CO in an older district in Budapest has excess capacity so they rented a flat next to the CO and drilled the wall. Presto! Plenty phone lines without installation costs.
(I worked retail computer support at the time, and USRobotics did a neat sales presentation at Henry Ford Manor where, if we went to an "educational" session about the technology, we got a free modem. I already had ISDN by that time so I didn't really need it, but it was still neat. And a couple-hundred-dollar modem, for free, for a retail employee seemed like quite a perk.)
[1] https://en.wikipedia.org/wiki/History_of_AT%26T#Rise_of_SBC