Google uncloaks once-secret server | Hacker News Reader

Google uncloaks once-secret server

news.cnet.com

238 pointsars53 comments

While searching for the actual battery patent, I ran across the modular data center patent: http://www.google.com/patents?id=7TypAAAAEBAJ&printsec=a... and also this power supply research paper abstract: http://research.google.com/pubs/pub32467.html

I was also surprised at some of the patents awarded. While the claims were very specific, the ideas were very generic, such as mobile payments via cell phone. Take a look for yourself: http://www.google.com/patents?zoom=4&q=inassignee%3Agoog...

If anyone finds the battery/server patent, I would be extremely interested.

Not filed under Google but under Exaflop LLC (one quintillion floating point operations per second) [hat tip to: http://www.datacenterknowledge.com/archives/2008/02/15/googl...]

Patent application: 20080030078

"Each UPS includes a battery selectively connectable across a DC bus, and a AC-to-DC rectifier that converts an AC input voltage to a single output voltage on the DC bus. The regulated DC bus voltage may be close to the battery's fully charged voltage..."

http://appft1.uspto.gov/netacgi/nph-Parser?Sect1=PTO1&Se...

Pretty much every patent is a specific implementation of a generic idea.

This server appears to be from 2005; it looks like Google is still keeping some secrets.

The hard disks are dated Nov 2007. The motherboard is from 2005, but it's got decent specs http://www.gigabyte.com.tw/Products/Networking/Products_Spec... so they may still be using them.

Although I'm surprised they went with Xeon CPU's, and I don't see anything on gigabytes page about 12V only for the board.

It's quite possible that they got Gigabyte to do a custom design for them, given their volumes. Or, the 5v conversion is done in an extra device between the power supply and the mother board power socket (ATX or whatever it is).

We did indeed work with gigabyte on that board.

Given all Google's talk about power efficiency, it's debatable whether power-hogging NetBurst processors are "decent".

Note that the publicly available Gigabyte board you link to is the GA-9IVDT, the name on the board in the photo is GA-9IVDP. Moreover, they look nothing like each other.

I also can't tell where the 5V rail for the hard disks is coming from:

http://i.i.com.com/cnwk.1d/i/bto/20090401/Google_server-1.jp...

The only wires from/to the PSU are those running to the battery and the 12V and ground wires for the motherboard.

the psu is labelled 13.6 V, there seems to be a fair bit of the motherboard under the drives, possibly that holds a 12V to 5V DCDC convertor

I really appreciate the minimal "case" these are in. Saves material and makes the containers lighter.

and I'm sure keeps them cooler.

The "case" is the shipping container. Again, economies of scale.

A post at the 'Green Data Center Blog' (http://www.greenm3.com/2009/04/close-up-an-old-google-server...) says that the pictures are of a "Google Server which is ready for retirement".

I don't see how they can patent battery per motherboard design. Isn't that how laptops work?

OTOH, this design makes a lot of sense. One of the biggest failure mode for rack-wise failure I've seen is human error from maintenance crews: trip over or unplug the wrong a power cord (I've seen this happen twice already from different companies.) This design completely shields this type of errors.

But, it should not be patentable!

The battery-per motherboard makes eminent sense. Between economies of scale driving down the price of small AGM and gel-cell lead-acid batteries (1) and the increased efficiency of having the battery on board, it's an obvious win!

(1) - ironically, this is probably because of retail and commercial sales of Uninterruptible Power Supplies!

I don't understand: How can one battery per server be more efficient than a big, shared battery for all of them?

To hazard a guess I'd say it's to do with voltage step up/down efficiency.

I agree. Standard UPSes convert AC to DC and back, in order to power the server PSUs and these conversions are inefficient. If the battery is in the server (after the PSU), you can miss out these conversions and save a lot of power.

I'd also submit that low-power electronic components are far more readily available (i.e. cheap) than ones which can handle a lot of juice.

Could also be to do with management, particularly switching - switching large DC currents is problematic (i.e. it arcs). It's easier to manage on a smaller scale.

With the one battery per server, you eliminate one moving part. Well not moving, but you leave out the step of going from dc power to ac power. With their setup, you take 220 vac (note the label on the power supply) to the server and to dc. Any conversion step costs heat.

1) No transmission/conversion loss. It is probably soldered directly to the DC power regulator on the mainboard.

2) The servers are frequently removed for maintenance. Thus, the batteries will also be checked/replaced along with the machines they serve. As a bonus, there is always exactly the right amount of battery capacity in the system.

See how the hard drives are mounted back-to-back? I'm guessing that is designed to cancel the mechanical noise by putting the drives 180deg out of phase from each other, extending the life of the drives.

I believe that is patented (by Avid?) and that might be why Google was cagey about releasing specs.

I don't follow, if it's patented then didn't they already release the important details in the patent application?

I believe the implication is that "If it's patented by someone else, Google may have to pay if it comes out that they are using this technique"

Had the read the article carefully to see if i wasn't "getting" the joke. It's legit, and very cool that they were so innovative with their data centers.

The article says the battery backup was a secret but: http://www.webhostingtalk.com/archive/index.php/t-492779.htm... and at the time at least, he had linked to a picture of the board (but it's down now).

The battery integration is quite interesting, although I'd bet they are using it for purposes other than "having to buy one big UPS." One benefit I see to it is high availability, since the site UPS going out won't take down the servers, since they all have their own. Plus, I'd bet it makes the guys that work on the power buss sleep sounder at night, since its easier to work on without worrying about taking out the whole datacenter.

Since they're running the PSUs near capacity. the batteries probably provide some additional headroom under heavy load spikes.

Yes, there have been a few publicized incidents at large hosting centers where the central UPS either failed or didn't kick in when it was supposed to. By putting the battery in the server it eliminates that single point of failure.

This seems real despite the post date.

I've been in data centers hosting Google systems. This picture looks very similar if not identical to what I've seen, so I'd say it is most definitely real.

Could you verify if Google still uses Hitachi Deskstar drives?

I recall reading an article about the testing Google conducted to find the most reliable drives, but they refused to publish the findings.

They did actually publish their results. It's an interesting read:

http://research.google.com/archive/disk_failures.pdf

That paper does not mention any specific brands or models.

I wasn't ever close enough to see what kind of disks they were.

Based on their white paper on MTBF of hard disks, I'd guess that they avoid favoring any single vendor to lower the chance of a bad batch of drives all failing at once.

Anyone got a link to the "video tour Google presented of its data center containers"?

It's a video of the official video but passable (until Google releases the official video shortly):

http://www.youtube.com/watch?v=bs3Et540-_s

Article seems to no longer be available from the original link, but it's still up here: http://news.cnet.com/8301-1001_3-10209580-92.html?part=rss&#...

Battery pack is a good idea. I have a UPS that is just about dead (~1 minute of backup, but I keep it for spikes and drops). I thought it would be great if the PSU had a built-in UPS. The space between the PSU and drive bay would be a perfect spot, no need for the heavy switching hardware since the PSU can deliver the 12 or 13.6 volts needed for the battery to charge.

What battery chemistry are they using? It looks like a lead-acid package. Those are cheap, safe, and reliable. But they have very low energy density and there are disposal issues.

Was it really necessary to patent the built-in battery design? That doesn't really seem original enough to warrant a patent.

I for one am surprised that those are 2U servers and they haven't gone to 1U to double their space efficiency.

it looks like two of them may fit side-by-side, like this they get to use the big ram sticks and a cheap psu, as well as fans that will last a lot longer than those pesky 1u fans.

They may also use 'deep' racks to the machines have the PSU's in the middle and the fronts on the facing sides of the cabinets, one front, one back. That would make it twice as efficient as 1U, it would also explain why the cables go out the front.

I switched back and forth between 1U and 2U setups for the same motherboard, and I could run my system in higher ambient temperature with less airflow when using a 2U heatsink.

From the patent-looking picture in the article, it looks like the racks are 1-deep, so I don't think they're using 2-deep racks.

I've seen them two deep side by side. They had custom racks that reminded me of those racks of trays of food you'd see in large production kitchens. There were four systems per level/tray.

The server rack that they have at the Computer History Museum has them stacked 4 systems per tray as well.

1u limits airflow, probably too hot to stack HD that close, and 1u power supplies were probably cost prohibitive.

I dont think these are current.

No liquid cooling? How efficient is the power supplies?

One Server to rule them all

Reply on news.ycombinator.com