Sept 28-29, 2006 :: 2 Hurricane outages

[Jason]

We've had 2 instances of unrelated outages on Hurricane luck in the last 24 hours.  Fortunately neither of them are server issues to be concerned with.

Last night a request I put in to have an IP updated was made on the shared IP which should've never happened.  That resulted in all web requests being routed to a login page for just under 10 minutes until I had it corrected.  I accept responsibility for this happening though it came down to someone not reading a request properly unfortunately.

And then today -- about 15 minutes ago -- Hurricane experienced about 10 minutes of outage due to a disconnected fiber line at the datacenter.  The new datacenter where Hurricane is located is still adding new network providers (which is great news) but a technician unplugged a fiber line he shouldn't have which caused this issue.

Both events are completely unrelated, but I wanted to let you know the causes so you don't think it's a server problem.

Regards,
Jason

[Mark]

And then today -- about 15 minutes ago -- Hurricane experienced about 10 minutes of outage due to a disconnected fiber line at the datacenter.  The new datacenter where Hurricane is located is still adding new network providers (which is great news) but a technician unplugged a fiber line he shouldn't have which caused this issue.

Oops! lol imagine being that guy.

[Jason]

Indeed.  There are words for mistakes like that but I'll save them for your imagination

Although, as a result of this coming up with other clients at the dc, they're putting in some great failsafe systems there -- they're still moving equipment from the other dcs.  It should be an amazingly redundant dc once everything is done shortly. 

Of course, human error will get you every time. 

[Mark]

Of course, human error will get you every time. 

Yes sir.