Hacker News Reader

Comment by harrytuttle | Hacker News Reader

Yes I've worked for a couple of outfits that did this over the years. It's usually down to who sold them the solution though. It's worst if the salesperson is external.

The funniest has to be the engineering company which I used to work with (not for thank fuck) back in the 90's (when 100Gb was big data!). They managed to bag 2x full 42U racks full of HP N-Class HPUX UNIX kit, PDU systems and separate disk arrays for just over £1,000,000 supplied with one full time UNIX monkey to keep the plates spinning.

Turns out they had only 10 CAD/engineering staff and about 8Gb of data in an Oracle DB which a single Sun Ultra desktop (<£10,000) could have handled without a problem at the time.

View on HN

After I moved from academia/gov't labs to private industry, I was absolutely flabbergasted at the waste. Many companies literally just buy whatever their vendors throw at them to solve a problem, and if a reasonably intelligent techie spent just an hour or two thinking through the problem, they'd realize that the solution was either far less complicated than the proposed kit or far more; either way, what's proposed is almost never right.

Another thing I haven't seen much of is acceptance testing. This is actually more surprising to me than the reliance on vendors for problem solving. It's an absolute no-brainer to say "I have problem X, which will be solved by solution Y, and the success will be measured by the set of tests Z. Unless the above happens, we will not pay the bill and you will take the hardware back." The vendors looked at me like I was from Mars when I started suggesting these.

I have not since complained about government waste in research computing budgets.

Outsourcing your own teams means there are no in-house experts to perform the oversight role you've described. Thus the principal-agent is amplified.

Yeah... it looks like my employer is going down the same route with an EMC Isilon NAS system.

We really just want high throughput for linear scans; possibly with a small amount of parallelism, which would argue for a fairly simple solution with as little cacheing as possible (ideally none whatsoever) ... but I cannot even tell (from the website & the literature that I can access) how these things are supposed to operate -- which is a worrying sign in itself.

Previously, I got good performance out of a SAN using a handful of NexSan SataBeasts -- which I would have recommended to use again, if anybody had bothered to ask.

:-)

Isilon is nice - expensive (it is EMC), but it gives good performance and is reliable.

When in the 90's? I'd expect it would have taken the full 2x 42U racks to keep the whole dataset in memory at one time. Not to say it's required, but that doesn't sound totally crazy for the time period.

Also, having an order of magnitude more hardware than you ought to need is totally understandable when hardware capability is increasing by an order of magnitude faster than hardware replacement cycles. It's when you're off by several orders of magnitude that it starts to become suspect.

1998. It had 4Gb of RAM in each node which wasn't that scary. Most of it was UPS and disk.

I had a Sun 1000E [1] for 3 years as a workstation from 1996. 8x 50MHz CPU, 2Gb of RAM, 16Gb disk array. It was made in'93 and was approx 12U for the unit and array. Things weren't that big in the 90's. 80's yes...

[1] http://en.m.wikipedia.org/wiki/Sun4d

Reply on news.ycombinator.com