October 9, 2006 :: Lightning Outage

[Jason]

Our server Lightning experienced just shy of 44 minutes of outage around 4pm EST today.

An account on that server was compromised and was attacking another server.  The datacenter took the server offline to investigate which resulted in the outage.

Without going into too many details, I'm very much upset with the process they followed today.  I certainly understand their need to protect their other clients, however, they didn't follow their normal procedures.  They've always provided at least 24hrs of notice for something like this and furthermore, they at least call the listed contact numbers.  For whatever reason, neither advanced warning nor a call to the correct number was given today. 

When the server was brought back online, we had the compromised account isolated in under 5 minutes.