Comment by Spooky23 | Hacker News Reader

Comment by Spooky23 | Hacker News Reader

Parent Full thread

I had an even stranger experience. I was working on contract for a huge place to do some Oracle and systems management work in a small datacenter that was dedicated to a single purpose (it was a huge enterprise with about 80k sq ft of datacenter) around 2001. It was a beautiful setup, the wiring was nice, even the racks were really high end and setup thoughtfully. There were probably about 40 Sun boxes (e450s, e4500's) some AIX stuff, some windows stuff, about 3 racks of firewalls, tape robot, the works.

So I started setting up the management tools and noticed after a few days that nobody really came around much. Then I noticed that nobody was ever logging into these systems. Ever. It was a ghost datacenter running some WAN management stuff that nobody knew about -- some consultants set it up and left -- nobody knew what it was.

In my case, I did my job to the letter, I setup the management tools, got the servers and databases humming, etc. But the actual equipment was serving no discernable purpose. Total investment just in hardware and software was probably $5-6M

Though I've never worked for the large local telecom here in town I've had many friends and coworkers who have. I wish I had a dime for every time I've heard a story just like this (though smaller in scope). People have set up file sharing servers, game servers, whole warez team back ends...the works on hardware that was purchased & set up but never used. This is one of the primary reasons I've never wanted to work for Big Local Telco...

I worked with someone who got fired for hosting MP3s on a surplus corporate machine.

At least, that was the excuse they used when they fired him. I think the actual reason was that he was hard to get along with. However, it's difficult to fire him for that reason. Much easier to fire him over misuse of corporate equipment.

The guy told everyone about his MP3 server. Including summer interns.

> [...] whole warez team back ends

Are you able to elaborate on this? I've always wondered how the warez scene functioned; residential Internet connections are obviously insufficient, hacked resources too risky, and warez groups turn their nose up at any form of legitimate resources (e.g. dedi/colo hosting)

This leaves corporate infrastructure. All it takes is a small IT team with a big budget, lots of hardware and network resources, and management who neither know nor care how their resources are being utilized.

It's terribly romantic.

I know that my past employers networks have been abused for long periods for game servers and MP3 hosting. Y2K period in particular resulted in lots of insane investments in IT, with very lax management in most places. Nowadays, tools and copyright take downs make it difficult to be completely inept.

Hacked resources are used as well. I worked at a place that used an old SGI Indy as a mail relay. It wasn't really managed, and somebody rooted it to use as a distribution server for warez and porn. I suspect the admins kept it running as a source for their own warez and porn needs.

I got involved when a spammer got in and sent millions of emails, flooding our puny T1 with bounces.

Has anyone worked anywhere that didn't have a bit of a server that was dedicated to piracy? I haven't. Even working for a large public healthcare provider. Slightly worse, there was even a torrent client using their fibre. The trick was to find an IT illiterate colleague or one who was leaving it best of all, both these things in one person. Their logins wouldn't e revoked for months or years. Use their login to set up torrent clients. If you called IT support and just played dumb you always found someone who would let you install stuff (I need this program for my section on ankle casting). Keep calling back until they answer. Once a torrent was running, stay well away from the computer until no one else is around as grumpy IT guys would hang round to try and catch you. This was long ago now, I pay for my media now, with the exception of a few shows from the UK which I can't pay for.

One technique that managers use is to over-purchase.

Think about it this way:

If a need comes up, it's going to take months to get all the paperwork done to get new hardware.

If you have idle hardware, you're ready to go ASAP.

It's also a form of redundancy, albeit a primitive one. If equipment goes out, you have stuff you can swap out immediately.

Awesome story! Did you ever fantasize about setting up a big data set avant la lettre, or running your own hosting company off of it?

Slightly related, this reminds me of the 400,000 euro costing MRI-like machine my employer bought, without checking if it could connect to our healthcare software. It didn't.

What is like an MRI? And as someone who peripherally deals with this sort of proprietary crap regularly, they almost never connect to anything nicely. Things have got better recently with people starting to follow te DICOM spec, but brand new GE MRI, PET, CT and X-ray rooms will not interface properly with our radiology information system. They also have a bug whereby any image post processing is uniquely identified by time (hour, minute second). It turn out this isn't unique and will merge patient's imaging when you use an off-the-shelf and popular system like Intellipacs. Oops. Philips is better, just. Their scanners just dump these files all over the picture archive and communication system which are called PR files. No one knows what they are, they just come out the scanners and are spread everywhere. I'd like to think Siemens are better but I only have a small amount of experience with this manufacturer.

Reply on news.ycombinator.com