Thursday, March 13, 2014

Hot DC

This is what happens when the HVAC fails in your data center and no one sees the alerts from the environmental monitoring system. It hit 118 degrees. Luckily, a server shut itself down; prompting a call from the plant so that someone could figure out what was happening. I've never seen tiles curl up like that. C'est la vie. (And we all have our trials and tribulations.)



4 comments:

  1. Replies
    1. The good news is that most of the racks are empty or almost empty. I can't imagine what would have happened if they had been full of heat producing servers. You can tell which racks had servers in them - that's where the tiles curled.

      Delete
  2. I've had a similar thing happen in a datacentre I was involved with...

    A small datacentre for a prison, cooled by an oversized colling system (since we expected growth). Only problem was, the coolers temperature got too low and it went into a protection state. No problem, unless it never left that protective state... so no colling since friday afternoon.

    On monday morning I couldn;t reach the servers and a tracert stopped just outside the datacentre. Grabbed the keys and went for a check. Temperature was over the auto-shutoff temperature for Cisco routers and you could feel that on the outside of the DC through 2 doors...

    I opened the doors and put ventilators in the doorway, one pointing inwards and one outwards and that lowered the temperature enough to make the routers work again. The servers needed some more cooling though. ;)

    ReplyDelete
    Replies
    1. I feel your pain. You can see in the second picture a couple of box fans and the open door. Those boxes fans can be worth their weight in gold.

      Delete