October 30, 2011 :: Network disruption impacting multiple servers

Jason · October 30, 2011, 05:32:10 PM

For approximately 2 hours this afternoon, there were network disruptions that impacted some connections to a few servers.

From what was reported, it seems like Whiteout (Charlottezweb's main server), Supercell and Jetstream were impacted.

During the disruption, certain network paths to those servers were working whereas others were not. I was able to reach all servers throughout however I could not reach www.charlottezweb.com via browser. I had several customers also reporting their sites as unreachable but I could reach them. So it was a bit hit or miss during the issue timeframe.

The datacenter tracked the issue to a server that was on the same switch as these servers that was being flooded with traffic. They took that server offline temporarily to resolve the issue. The server is back online and they are monitoring traffic for anything abnormal.

I will keep an eye on things the rest of the afternoon should anything reoccur.

Thank you,
Jason

Diamond · October 30, 2011, 10:23:31 PM

Hi Jason,

We were one of the ones out during those two hours and we just went down again at approimately 10:09 pm. We're on Supercell.

Thanks.

www.pogohaven.com

Jason · October 30, 2011, 11:17:07 PM

Are you still having an issue reaching your site?

I was able to reach your site right after you posted this (I got the email notification) but Charlottezweb was down again for me. It's back up now.

Diamond · October 30, 2011, 11:32:20 PM

It just came back after a little more than an hour out. I couldn't get to Chalottezweb the first time we were down, but could this time.

Thanks Jason!

Jason · October 31, 2011, 08:57:11 AM

Quote from: Diamond on October 30, 2011, 11:32:20 PM
It just came back after a little more than an hour out. I couldn't get to Chalottezweb the first time we were down, but could this time.

Thanks Jason!

Thanks for confirming.

Jason · November 04, 2011, 11:19:45 AM

Friday, November 4

I've noticed a series of brief network disruptions again this morning at approximately 6am, 7am and 10:45am EST. Each one was typically only a few minutes in duration. Whiteout (Charlottezweb.com's dedicated server), Jetstream and Sirocco have alarmed today.

I've spoken to the datacenter on two of the occasions and they're currently investigating whether this is connected to the same issue we saw on Sunday. Either way, one thing I'm certain of is that it's a network issue and not an issue with any of our servers themselves. They've been up and running fine throughout the disruptions.

I will update this thread if I hear any information that would be helpful to share.

Thank you.

Pam · November 04, 2011, 04:25:59 PM

Jason,

I had several folks complaining of slow response earlier today.

Since I wasn't around at that time, I can't give you specifics but just wanted to make you aware.

I suppose the slowness could be connected to the issues today.

Thanks.

wizard 87 · November 05, 2011, 11:27:15 AM

Doing it again this morning. Started around 11. Not as bad as last Sunday but still an issue.

Jason · November 05, 2011, 01:42:19 PM

If you could send a tracert to my email (or open a ticket) when it's slow or especially when you can't reach it, that would help tremendously.

They didn't find any issue yesterday so I'm trying to track down tracerts as we see these issues so we can determine where the drop is.

If you need instructions on how to perform a tracert, please let me know.

Thank you!

Jason · November 05, 2011, 03:02:30 PM

I don't have specifics yet but I've been told we shouldn't be seeing any more issues related to this.

If you notice anything still (or going forward), please send me a tracert while the issue is happening.

Thanks!