An Eight in a Million chance (consultants, interns, MS Access) | Hacker News Reader

An Eight in a Million chance (consultants, interns, MS Access)

williamedwardscoder.tumblr.com

75 pointswillvarfar12 comments

CDR reconciliation is still this comedic mess, btw. Including the silly business with calls that cross midnight being sometimes split and sometimes not, and weird rounding behaviour, and so on. And on top of this, calls would be reported to be reconciled with arbitrary delays; just because they weren't going to report a January 1st call until January 20th doesn't mean they weren't going to bill for it on January 14th!

Indeed, the systems for generating the dollar value on our invoice were almost entirely separate from the systems for generating the line-items on our invoices, and they never did agree, and our upstream provider didn't even see that as a problem.

My boss at one point, if I understood him correctly, claimed that (in 2009 or 2010), the process for the telco to know how much to bill us involved human operators exporting to a reel of magnetic tape, moving the magnetic tape reel across the room, and importing it into a different database. He wasn't much of a one for leg-pulling, but I dunno.

I don't run in to many telecom folks these days. Are you still working in the industry?

I used to do a lot of telecom consulting for a large customer who we transitioned to a 100% SIP carrier. This particular carrier used AdTran TA 900 series gateways to provide T1-CAS/PRI to older phone switches. The project was particularly interesting because the carrier let me (a third-party analyst) have unheard of access to their infrastructure. Once I earned their trust, I had access to all of the AdTran devices, as well as their core switching and CDR reporting tools.

I actually just put wraps on a Ruby script that I use to analyze call concurrency from CDR exports (from CDRTool [1]). I dropped it in a Gist if you're interested. It takes a CDRTool query export (CSV data) and outputs a tab-delimited concurrency datafile that can be used pretty easily with gnuplot. The script could be easily adapted to any CDR report that uses single-line call entries.

https://gist.github.com/1706903

1 - http://www.ag-projects.com/CDRTool.html

PS - If you find this at all interesting, hit me up on Twitter to keep in touch https://twitter.com/#!/bradleyland

Nope, after I graduated I moved country and industry and wasted a good decade programming the mobile phones that help people create the CDRs that torment you.

These days I'm swung back more towards DBs and servers and stuff, but not a CDR in sight thankfully.

Liked your ruby; not a ruby ninja myself but it looks quite readable and almost pythonic :)

Speaking of rounding errors...

I worked telecom billing about 10 years ago... Our system consisted of a couple dozen fairly large C++ programs, and numerous DBF tables.

This system was originally written in Clipper(1), which IIRC defines numeric as an 18 digit text field. Well, when that particular program got ported to C++, the author simply used a 32-bit int in place of NUMERIC... Which needless to say worked just fine up until we started getting clients that had values larger than 2^31-1 for that field.

The scary part, is that it would just overflow, so you wouldn't get any errors... It wasn't until some guy in ops noticed a number that just seemed a little too odd.

1: Serendipity: The founder was a medical student that happened to know clipper, that happened to have a friend that had just started a telecom company and needed a billing solution (right after the Bell breakup).

My mobile provider used to just drop all calls at exactly 23:54 presumably to solve the crossing midnight problem. I know because my phone at the time displayed the start and end times of calls and I was doing mostly all my calls at that time.

I believe that Centurylink (formerly Qwest of Washington State, formerly US West) still provides 9-track magnetic tape CDR delivery as a tariffed service.

Makes sense - they have to be the worst telco in the US.

When I worked at AT&T some years ago they had an entire team of about 50 developers and testers working on identifying problems in AT&T's myriad billing and provisioning systems. This team developed applications that analysts used to reconcile billing with account features, usage, taxes, discounts, etc. This team was very successful in finding both under billings and over billings. One project identified something like $100M in annual under billings. Another project found so many switch/bill mismatches that they just decided to cancel it because they didn't have the manpower to manually correct all the problems or deal with all the customers calling to complain that their bill went up or their feature disappeared.

Interesting to learn that Amdocs started off as a billing system. I have worked with it in plenty of spots where it's used as a CRM but was unaware of its billing system roots.

I'm not sure why he called it an Israeli company though, it was founded in the US.

you made me google and yeap, its an American company.

The whole consultancy team they shipped over was definitely Israeli, but then Amdocs has big development offices there it seems so that's just where this particular team came from.

Now we know :)

Excellent story & writing, particularly the "cable unwind" bit.

great story! thanks for posting.

Reply on news.ycombinator.com