Charlottezweb

June 13, 2014 :: Wildfire, extended outage

Discussion started on Server Updates & Outages

Wildfire experienced an outage this morning that was monitored at approximately 3 hours and 24 minutes in length:

http://uptime.charlottezweb.com/430879

The issue began around 3:01am Eastern (according to Pingdom, our monitoring company). 

Our techs observed the outage and attempted to login to initiate a reboot.  Unfortunately, they were unable to login due to the fact that I changed the credentials earlier this week.  The credentials were provided to them at that time however it was in a separate location awaiting verification so they had not yet updated their overall records.  I'm certainly not happy about this however I accept responsibility for not chasing that down sooner to ensure they were aware of this and had it fully tested and recorded.  I'm following up on this presently.

They attempted to call me when they discovered their lack of access and my phone was unfortunately not accessible to me at the time.  This compounded the problem and is also something I take full accountability for.   

It was a combination of two unfortunate scenarios that caused this to take much longer than it normally would've.

When I woke up and grabbed my phone, I immediately saw the multiple monitoring notifications and rebooted the server.  It was back online minutes later.

For our customers impacted, please accept my apologies and rest assured I'll take steps to ensure this doesn't happen again.  It's the first time, if memory serves, that anything like this has occurred in the over 12 years of running this business and is certainly unacceptable service in my book.

Thank you,
Jason

#1 - June 13, 2014, 06:38:22 AM

Members:

0 Members and 1 Guest are viewing this topic.