PHP upgrade in progress (Resolved)
  • Priority - Medium
  • Affecting Server - Lightning
  • We're going to be performing a PHP upgrade on server Lightning.  There may be a short outage to the web service as we complete the process.

    Regards,

    Jason

  • Date - 06/10/2017 19:30 - 06/10/2017 20:19
  • Last Updated - 06/10/2017 19:31
Lightning web service issue (Resolved)
  • Priority - High
  • Affecting Server - Lightning
  • Lightning experienced a failure of its http (web) service that lasted approx 45minutes.  The server was online the whole time but sites weren't loading. 

    We're currently investigating the cause however we did have to suspend one account temporarily to allow the load to drop to where we could repair a few mysql databases that were compounding the problem.  It looks like we're stable as of that activity however I hope to get more information on the root cause.

  • Date - 05/31/2017 17:51 - 06/10/2017 19:30
  • Last Updated - 05/31/2017 19:12
Lightning - httpd/mysql issue (Resolved)
  • Priority - High
  • Affecting Server - Lightning
  • We're investigating an issue with http / mysql on server Lightning.  I will update this post as soon as it's resolved or we have more information.
    ---
    Update: This was resolved at approximately 7:04pm Eastern. This seems to be very similar (if not the same) issue that happened earlier this week on Monday.  We're looking for root cause now.  I will post to our forums once I have more information.

    Update March 6, 2017:  A forum posting with additional details has been posted here:  https://www.charlottezweb.com/forums/index.php?topic=2113.0

    As with earlier this week, if you're seeing any errors on your site on this server, running a database repair in cPanel will typically resolve it.  If not or if you'd like assistance, please email me or open a ticketI've created an article here on how to run a database repair in cPanel.

    Thank you,
    Jason

  • Date - 03/04/2017 23:57 - 03/10/2017 17:38
  • Last Updated - 03/06/2017 22:02
MySQL issue on Lightning (Resolved)
  • Priority - High
  • Affecting Server - Lightning
  • We're working on a mysql (database) issue on server Lightning currently.  This started after an http (web service) outage.

    I will post an update once we know more.

    Update: This was resolved at approximately 2:30pm Eastern however we're looking into the root cause.  If you're seeing any errors on your site on this server, running a database repair in cPanel will typically resolve it.  If not or if you'd like assistance, please email me or open a ticket.  I've created an article here on how to run a database repair in cPanel.

    Thank you!


    Regards,

    Jason

  • Date - 02/27/2017 19:20 - 02/28/2017 22:11
  • Last Updated - 02/27/2017 23:55
PHP upgrade scheduled (Resolved)
  • Priority - Low
  • Affecting Server - Landslide
  • We will be upgrading PHP on server Landslide on Friday around 6am Eastern time.  This may result in a brief outage of php service to activate once complete.

  • Date - 02/17/2017 06:00 - 02/17/2017 14:38
  • Last Updated - 02/16/2017 21:41
Server Upgrades - COMPLETE (Resolved)
  • Priority - Medium
  • Affecting Other - Multiple Servers
  • The retirement of server "Blackice" is complete as of approximately 8:10pm Eastern, November 2.

    Please visit this post for all details including the latest updates:

    http://www.charlottezweb.com/forums/index.php?topic=2089

  • Date - 11/02/2016 18:00 - 11/05/2016 14:31
  • Last Updated - 11/03/2016 00:31
Lightning Server Upgrade - October 1, 2016 (Resolved)
Vortex Server Upgrade - October 1, 2016 (Resolved)
  • Priority - Medium
  • Affecting Server - Vortex
  • Please see the following post regarding a scheduled upgrade to server Vortex.

    http://www.charlottezweb.com/forums/index.php?topic=2092.0

    Note that Charlottezweb.com is on this server and will experience two outages on Saturday as a result of this planned maintenance.

  • Date - 10/01/2016 05:00 - 10/01/2016 06:56
  • Last Updated - 09/29/2016 19:56
Wildfire server to be retired on September 15 (Resolved)
  • Priority - Medium
  • Update: This was completed per schedule.

    Reminder that server Wildfire will be shut down on Thursday, September 15, 2016 at approx 10pm Eastern.

    This is related to the server migrations that are discussed here:

    http://www.charlottezweb.com/forums/index.php?topic=2089

    If you have an account on this server and manage your own domain name(s), your action is required as discussed in the post above.

  • Date - 09/15/2016 22:00 - 09/16/2016 13:47
  • Last Updated - 09/16/2016 13:47
If your .ca domain is facing issues (Resolved)
  • Priority - Medium
  • Affecting Other - .CA Domains
  • CIRA.ca  (the agency that controls .ca domains) has been stepping up compliance audits lately. 

    If you use a .ca domain for your site and you find that your site isn't loading, please open a ticket immediately so we can investigate.

  • Date - 07/06/2016 00:25 - 07/07/2016 02:19
  • Last Updated - 07/06/2016 00:28
Charlottezweb.com migrating servers (Resolved)
  • Priority - Medium
  • UPDATE:  Charlottezweb's maintenance should now be complete.  If you spot any errors or issues, please contact us.
    -------------

    We are replacing server "Thundersnow" over the weekend of May 6-8, 2016 via a migration to a new server:  "Vortex."

    Charlottezweb.com is hosted on this server and will be impacted by this migration.

    I'm targeting a window of 4pm-6pm Eastern (New York) time on Saturday, May 7 to do the migration of Charlottezweb.com. 

    What does this mean?
    During this time, you may see our forums and the Client Area in a "locked down" maintenance mode.  This is to prevent the loss of data during transfer.

  • Date - 05/07/2016 16:00 - 05/09/2016 00:09
  • Last Updated - 05/07/2016 23:01
March 12 and 20, 2016 - Maintenance (Resolved)
  • Priority - Low
  • Affecting System - Datacenter Scheduled Maintenance
  • Please see this post on our forum regarding two upcoming datacenter maintenance windows.

    http://www.charlottezweb.com/forums/index.php?topic=2074

  • Date - 03/12/2016 01:00 - 03/20/2016 17:15
  • Last Updated - 03/08/2016 19:08
Scheduled - Server Reboots - February 18, 201 (Resolved)
  • Priority - High
  • Affecting Other - multiple servers
  • Many of our servers will be rebooted tonight starting around 10pm Eastern.  This is a semi-scheduled event however I have purposely not posted advanced notice.

    This is related to a largescale security issue that has been exposed in the past few days.  I will share more details on our forum once our reboots are complete and our servers are secure.

    Thank you for your understanding of your security.

    Regards,

    Jason

  • Date - 02/18/2016 22:00 - 02/18/2016 22:09
  • Last Updated - 02/18/2016 20:43
Blackice -- Emergency Server Maintenance (Resolved)
  • Priority - Medium
  • Our datacenter is currently working on server "Blackice" due to an unplanned emergency maintenance issue that arose.

    They expect the maintenance to be brief however I don't I have specific timeframes yet.

    I will update this posting as soon as more information is available.

    Regards,

    Jason

  • Date - 02/15/2016 22:30 - 02/15/2016 23:48
  • Last Updated - 02/15/2016 22:40
Wildfire Server Upgrade (Resolved)
  • Priority - Medium
  • Server "Wildfire" will be upgraded tonight to a more powerful configuration.

    The upgrade will involve two periods of service disruption.  The first is complete.

    Please review the following thread on our forums for full details:

    http://www.charlottezweb.com/forums/index.php?topic=2072.0

    Update 12pm Eastern, Feb 6:  The upgrade is still in progress.  I will update the forum post above once complete.
    Update 2:48pm Eastern, Feb 6:  The server appears to be in the final stages of the resize and is currently offline while this completes.
    Update 3:49pm Eastern, Feb 6:  The server is in the final stage of resizing its partioning.  Unfortunately there's not a way of knowning how much time is remaining at this phase of the process.  I will keep monitoring it for completion. 
    Update 4:30pm Eastern, Feb 6:  The resize has completed as of 4:21pm Eastern.

    Thank you,
    Jason

  • Date - 02/06/2016 02:00 - 02/07/2016 15:34
  • Last Updated - 02/06/2016 16:31
Scheduled Datacenter Network Maintenance (Resolved)
  • Priority - High
  • Affecting Other - Network Maintenance
  • The datacenter is working on a some scheduled network maintenance that has been causing intermittant network connectivity issues over the last few hours for two of our servers:

    Landslide
    Thunderbolt

    These servers have been unreachable at times starting when the maintenance began approximately 4 hours ago.

    The latest update is that they expect the maintenance to be complete within the next hour at which point full redundancy should be in place again.

    Thank you for your patience

  • Date - 11/21/2015 06:19 - 11/23/2015 14:51
  • Last Updated - 11/21/2015 06:22
Landslide - Maintenance in Progress (Resolved)
  • Priority - High
  • Affecting Server - Landslide
  • Landslide required emergency datacenter maintenance last night which caused approximately one hour and 45 minutes of complete downtime.  When the server was brough back online, MySQL did not properly recover and required additional maintenance as well as the repair of a number of databases that corrupted when the server went down.  Unfortunately the server did not send any MySQL alerts which led to an extended period of outage to sites that utilize MySQL.  This was restored in the past 45 minutes and we'll be investigating today why the notices were not sent and what we can do to better prepare for a scenario like this going forward.

  • Date - 11/18/2015 21:25 - 11/20/2015 21:27
  • Last Updated - 11/19/2015 07:30
COMPLETE - MySQL & PHP upgrade - Thunderb (Resolved)
  • Priority - Medium
  • Affecting Server - Thunderbolt
  • Update:  This work has been completed.  As PHP/MySQL have been upgraded, please check your sites and scripts and contact Support if you have any issues.  Thank you!

    We have scheduled upgrades to MySQL and PHP on server Thunderbolt to occur on Friday (October 30) between 6-8am Eastern.  During this time, there may be periods of disruption to MySQL and PHP.

    I will update this posting once complete.

  • Date - 10/30/2015 06:00 - 10/31/2015 20:43
  • Last Updated - 10/30/2015 08:31
COMPLETE - MySQL & PHP upgrade - Wildfire (Resolved)
  • Priority - Medium
  • On Sunday, October 25, we upgraded Apache, MySQL and PHP on server Wildfire. 

    On Wednesday, October 28, we will be doing another upgrade of both MySQL and PHP. 
     
    The reason for two updates in the same week is that we're incrementally stepping up (version-wise) so that we can test in between vs. making a significant jump between versions.

    I expect any interruption experienced to be similar to Sunday's which was a few minutes to restart services once we have the upgrades complete.

    I will update this thread as we proceed.  Thank you!

  • Date - 10/28/2015 06:00 - 10/29/2015 11:50
  • Last Updated - 10/29/2015 11:49
COMPLETE - MySQL & PHP upgrade (Resolved)
  • Priority - Low
  • Affecting Server -
  • Update:  This work has been completed.  As PHP has been upgraded, please check your sites and scripts and contact support if you have any issues.  Thank you!

    We have scheduled upgrades to MySQL and PHP on server Wildfire to occur on Sunday between 5:30-7:30am Eastern.  During this time, there may be periods of disruption to MySQL and PHP.

    I will update this posting as we proceed.

  • Date - 10/25/2015 05:30 - 10/27/2015 21:45
  • Last Updated - 10/25/2015 11:15
Multiple server outages - October 6 and 7 (Resolved)
  • Priority - High
  • Affecting Other - Multiple servers
  • As an update -- All servers are currently functioning normally but we have experienced issues with 7 of them over the past two days that have caused anywhere from 30 minutes to a few hours of outage at a time.  One server (Wildfire) has seen significant downtime today related to this.

    We believe the issue has been isolated to a known bug with the most recent release of cPanel.  We're implementing a workaround to address the issue until they officially release a fix.

    These are the servers that were impacted to varying degrees either yesterday, today or both:

    Avalanche, Blackice, Deluge, Heatwave, Sandstorm, Thundersnow, Wildfire

    Thank you for your patience.

  • Date - 10/07/2015 10:04 - 10/08/2015 17:49
  • Last Updated - 10/07/2015 14:49
Datacenter issue under investigation (Resolved)
  • Priority - Critical
  • Affecting Other - Multiple servers
  • Our datacenter is currently working on some issues causing very high loads to several of our servers.  I've been told it may take them upwards of an hour to have it addressed.  I will post more information as it's available but this is impacting servers Avalanche, Blackice and Wildfire as I type this.

    Update: mitigation is in place and services have been restored. Wildfire is still experiencing higher loads than normal but is stabilizing.

  • Date - 10/07/2015 04:31 - 10/07/2015 04:58
  • Last Updated - 10/07/2015 04:55
Wildfire and Avalanche - High Loads (Resolved)
  • Priority - High
  • Affecting Other - Wildfire, Avalanche
  • We're currently investigating high loads impacting servers Avalanche and Wildfire.  This is causing sites to be unresponsive at this time.

  • Date - 10/06/2015 07:27 - 10/06/2015 08:34
  • Last Updated - 10/06/2015 07:28
All Servers - Reboot in progress (Resolved)
  • Priority - Medium
  • Affecting Other - All Servers
  • We will be rebooting all servers over the next hour for a security patch implementation.  Details will be posted following completion.

    Update: All servers are complete except for Wildfire which we're working on now.

    Regards,

    Jason

  • Date - 07/27/2015 18:47 - 07/27/2015 19:30
  • Last Updated - 07/27/2015 19:13
DDoS attack in progress (Resolved)
  • Priority - High
  • Affecting Other - Blackice, Landslide
  • Two DDoS attacks against servers Blackice and Landslide resumed this morning.  We'll post more information as it becomes available.

  • Date - 05/30/2015 09:00 - 05/31/2015 09:44
  • Last Updated - 05/31/2015 09:15
Server Reboots Complete (Resolved)
  • Priority - Medium
  • Affecting System - All Servers
  • Almost all of our servers were rebooted this morning (around 7:30am Eastern) to enable a security patch critical for securing our environment. 

    The reboots all completed successfully and quickly.  Pingdom reported each server outage to be 1-2 minutes.

    Please visit this posting on our forum for full details.

    Thank you

  • Date - 05/16/2015 07:22 - 05/16/2015 22:18
  • Last Updated - 05/16/2015 08:06
Landslide - Apache maintenance within hour (Resolved)
  • Priority - Medium
  • Affecting Server - Landslide
  • We're starting some maintenance to the web service (Apache) on Landslide.  This will cause loads to be potentially elevated for the next 30 minutes (approximately) followed by a restart of the service.  This may result in under 5 minutes of outage.

    We're doing this to address what appears to be a bug that's caused some service interruptions over the past week.

  • Date - 03/29/2015 13:22 - 03/29/2015 14:13
  • Last Updated - 03/29/2015 13:25
Scheduled outages, multiple servers (Resolved)
  • Priority - Medium
  • Affecting Other - Multiple servers
  • This maintenance was successfully completed this morning. 

    Please see the following post regarding scheduled outages that will impact the following servers on Tuesday, March 3:

    http://www.charlottezweb.com/forums/index.php?topic=2033.0

    (This was originally scheduled for Monday, March 2 but the datacenter postponed it).

    Avalanche
    Blackice
    Deluge
    Heatwave
    Sandstorm
    Thundersnow
    Wildfire

  • Date - 03/03/2015 02:00 - 03/04/2015 08:33
  • Last Updated - 03/03/2015 06:52
Server Reboots in Progress (Resolved)
  • Priority - Medium
  • Affecting Other - Multiple Servers
  • We will be rebooting all servers over the course of the next 24 hours. Details will be shared after the reboots are complete.

  • Date - 02/07/2015 19:37 - 02/07/2015 21:19
  • Last Updated - 02/07/2015 19:38
High server loads: Landslide (Resolved)
  • Priority - Medium
  • Affecting Server - Landslide
  • We are currently investigating high loads impacting server Landslide. Customers may experience page timeouts and errors while this occurs.

  • Date - 01/22/2015 10:40 - 01/22/2015 17:11
  • Last Updated - 01/22/2015 10:43
High loads due to account restore (Resolved)
  • Priority - Medium
  • Sandstorm is currently experiencing high loads due to an account restore. We're presently monitoring this and expect loads to drop once complete. The loads are causing slow site load times and some interruption.

  • Date - 12/15/2014 17:30 - 12/15/2014 18:07
  • Last Updated - 12/15/2014 17:46
Wildfire Scheduled Maintenance Overnight (Resolved)
Reminder - Jetstream/Monsoon Retirement (Resolved)
  • Priority - Low
  • Affecting Other - Servers: Jetstream and Monsoon
  • Reminder that servers Jetstream and Monsoon will be retired (wiped and shutdown) later today.

    Please visit the forum for details:

    http://www.charlottezweb.com/forums/index.php?topic=2002.0

  • Date - 10/17/2014 18:00 - 10/18/2014 11:23
  • Last Updated - 10/17/2014 15:16
Reboots to occur October 1 (Resolved)
  • Priority - Medium
  • Affecting Other - Multiple servers
  • Update 1 - we've been advised these reboots will occur early morning October 1 between 1am - 8am Eastern
    Update 2 - All maintenance is now complete.

    We've received notice of some software maintenance occurring at one of our datacenters that will require reboots of many of our servers.

    Impacted servers will include:

    • Avalanche - COMPLETE
    • Blackice - COMPLETE
    • Deluge - COMPLETE
    • Heatwave - COMPLETE
    • Sandstorm- COMPLETE
    • Thundersnow - COMPLETE
    • Wildfire - COMPLETE
    Please visit this thread for details:

    http://www.charlottezweb.com/forums/index.php?topic=2004.0

  • Date - 10/01/2014 01:00 - 10/02/2014 09:47
  • Last Updated - 10/01/2014 08:00
Thundersnow hardware maintenance (Resolved)
  • Priority - Medium
  • Please see the following post on our forum for details around scheduled downtime for server Thundersnow this coming Saturday (August 30) at 1am Eastern.

    This maintenance will mean Charlottezweb.com's website will be down.

    http://www.charlottezweb.com/forums/index.php?topic=1995

  • Date - 08/30/2014 01:00 - 08/30/2014 10:11
  • Last Updated - 08/26/2014 20:23
Scheduled Reboots, Saturday, August 23 (Resolved)
  • Priority - Medium
  • Affecting Other - Jetstream, Monsoon, Sirocco
  • We've scheduled reboots to occur at approximately 5am Eastern on Saturday, August 23 for the following servers:

    Jetstream
    Monsoon
    Sirocco

    Details will be posted here:

    http://www.charlottezweb.com/forums/index.php?topic=1992.0

    Thank you

  • Date - 08/23/2014 05:00 - 08/23/2014 13:15
  • Last Updated - 08/22/2014 13:41
Jetstream under heavy load (Resolved)
  • Priority - High
  • Loads are very high on server Jetstream currently causing services to restart in response.

    This is causing an interruption in web pages at this time.

    We're currently investigating.

  • Date - 08/21/2014 09:38 - 08/21/2014 22:15
  • Last Updated - 08/21/2014 09:39
Sporadic network slowness and/or outages (Resolved)
  • Priority - Medium
  • Affecting Other - Multiple large network providers
  • If you are looking for information on service disruptions experienced today, please see the following post on our forum:

    http://www.charlottezweb.com/forums/index.php?topic=1992.0

  • Date - 08/12/2014 08:12 - 08/14/2014 12:48
  • Last Updated - 08/12/2014 19:53
Datacenter network instability (Resolved)
  • Priority - High
  • Affecting Server - Thunderbolt
  • The datacenter is working on a segment of their network where server "Thunderbolt" is located.   The server is up and running but is appearing inaccessible at times due to this issue.  I will post an update as soon as I hear further from them.

  • Date - 08/05/2014 07:57 - 08/05/2014 19:59
  • Last Updated - 08/05/2014 07:58
Thunderbolt Datacenter Network Maintenance (Resolved)
  • Priority - Medium
  • Affecting Server - Thunderbolt
  • Please see this post regarding scheduled datacenter network maintenace that will potentially cause service interruptions for server "Thunderbolt" early morning Sunday, August 3, 2014.

    http://www.charlottezweb.com/forums/index.php?topic=1990.0

    Thank you

  • Date - 08/03/2014 02:00 - 08/05/2014 07:51
  • Last Updated - 07/29/2014 22:28
Planned Server Reboots (Resolved)
  • Priority - Medium
  • Affecting Other - Multiple servers
  • We will be rebooting the following servers this coming Saturday, July 19th approximately between the hour of 4 to 5am Eastern (New York time) to implement a security update:

    Avalanche
    Blackice
    Deluge
    Heatwave
    Sandstorm
    Thunderbolt
    Thundersnow
    Wildfire 

    Please visit this thread for additional details or if you have any questions:

    http://www.charlottezweb.com/forums/index.php?topic=1987.0

  • Date - 07/19/2014 04:00 - 07/20/2014 17:34
  • Last Updated - 07/17/2014 10:26
Sandstorm scheduled outage - Sunday (Resolved)
  • Priority - Medium
  • Please visit this thread on our forum regarding a scheduled maintenance window this coming Sunday morning for server Sandstorm.  There will be an outage to all of Sandstorm's services for approximately 30 minutes at that time:

    http://www.charlottezweb.com/forums/index.php?topic=1984.0

  • Date - 07/13/2014 05:00 - 07/13/2014 16:54
  • Last Updated - 07/10/2014 08:06
Scheduled Avalanche MySQL maintenance (Resolved)
Emergency dc network maintenance (Resolved)
  • Priority - Low
  • Affecting Other - Servers: Jetstream, Monsoon, Sirocco
  • Please refer to this forum post regarding emergency datacenter (upstream) network maintenance that may cause latency for up to an hour overnight:

    http://www.charlottezweb.com/forums/index.php?topic=1982.0

    Potential impact is limited to servers:  Jetstream, Monsoon and Sirocco.

  • Date - 06/30/2014 23:30 - 07/01/2014 04:40
  • Last Updated - 06/30/2014 18:39
Wildfire - Server Upgrade (Resolved)
  • Priority - High
  • Our server Wildfire will be upgraded tomorrow starting at 5am Eastern.

    Update 6/20 at 8:30am Eastern:  The resize attempt failed this morning.  We'll reschedule once the root cause is corrected.

    Please visit this forum post for updates as they arrive:

    http://www.charlottezweb.com/forums/index.php?topic=1977

    Thank you.

  • Date - 06/20/2014 05:00 - 06/20/2014 21:41
  • Last Updated - 06/20/2014 08:44
Thunderbolt Scheduled Outage (Resolved)
  • Priority - Medium
  • Affecting Server - Thunderbolt
  • Please see this thread regarding a very brief service disruption scheduled for 5am Eastern Friday, June 20.

    http://www.charlottezweb.com/forums/index.php?topic=1978.0

    Thank you

  • Date - 06/20/2014 05:00 - 06/20/2014 05:48
  • Last Updated - 06/19/2014 17:28
Thunderbolt - possible dc maintenance overnig (Resolved)
  • Priority - High
  • Affecting Server - Thunderbolt
  • There were several disruptions of service to server "Thunderbolt" today due to emergency network maintenance done by the datacenter.

    I've just received an update that it's possible there could be further outages overnight as they continue work on this.

    "Additional complications have prevented us from completing the network maintenance .... that was originally planned to be completed in the early morning of June 18th. We are moving forward with a solution that will enable us to restore services to affected customers while vendor issues are resolved. Users losing service earlier today will see this improve although, until additional resources are restored, [service] will continue to run at a diminished capacity. This means customers with services in that Zone may continue to experience intermittent latency and connection issues. Fixing this is of critical importance to us and we are working diligently to ensure the service of each and every one of these customers is personally addressed. This will be updated with additional information as we continue to approach a comprehensive solution. Customers affected by these networking issues ... will also continue to receive emails with additional details as they develop."

  • Date - 06/18/2014 21:58 - 06/19/2014 06:51
  • Last Updated - 06/18/2014 22:01
Thunderbolt - Emergency dc maintenance (Resolved)
  • Priority - High
  • Affecting Server - Thunderbolt
  • We are seeing sporadic network instability today impacting server Thunderbolt.  This is due to a network issue our datacenter is working.  Here is their latest update that should end in the next 30 minutes:

    We attempted to complete an update to significantly improve our network
    performance late last night. Unfortunately, an unexpected issue arose in
    Lansing Zone B due to additional complications with our most recent
    maintenance. We are moving forward with an enhanced solution that will
    return users in this section to peak performance, however this will
    incur another brief round of latency and temporary loss of connectivity
    for those involved. It is our aim to have this completed within the next
    90 minutes by proceeding with this fix immediately.

  • Date - 06/18/2014 16:22 - 06/18/2014 19:33
  • Last Updated - 06/18/2014 16:24
Thunderbolt - Datacenter maintenance (Resolved)
  • Priority - High
  • Affecting Server - Thunderbolt
  • Thunderbolt is has had some intermittant outages (two in the past hour and a half, approximately) related to some emergency network maintenance the datacenter is completing.   Their maintenance window ends at 6am so hopefully the interruptions have completed.

    In the event we experience further issues to Thunderbolt within the hour, this could be related.

    The current time is 5:24am Eastern.

    Thank you for your patience.

  • Date - 06/18/2014 04:16 - 06/18/2014 07:56
  • Last Updated - 06/18/2014 05:25
Wildfire - high loads due to backup (Resolved)
  • Priority - High
  • An image (backup) process is running on all our servers right now however Wildfire is experiencing some issues with it causing high loads and unresponsiveness.

    Update - a crash during the backup corrupted a database for one site that was causing it to consume server memory extremely quickly.   This should now be sorted.

    We'll continue monitoring for the morning.

    Thank you

  • Date - 06/17/2014 09:34
  • Last Updated - 06/17/2014 18:56
Informational: Wildfire outage (Resolved)
Wildfire reboot in progress (Resolved)
  • Priority - High
  • Wildfire is currently rebooting after hanging.

    Update 1:50pm Eastern:  The datacenter is working on a network-related issue that seems to be tied into this.  The issue may not be with Wildfire directly but with the segment of the network it's on.  I will update this posting as details are made available.

    Thank you,
    Jason

  • Date - 05/15/2014 13:36 - 05/15/2014 14:24
  • Last Updated - 05/15/2014 13:49
Wildfire reboot in progress (Resolved)
  • Priority - High
  • Wildfire became unresponsive and is currently being rebooted.

  • Date - 05/14/2014 11:14 - 05/14/2014 11:22
  • Last Updated - 05/14/2014 11:14
Reboot in progress (Resolved)
  • Priority - High
  • Wildfire is being rebooted now.

  • Date - 05/03/2014 19:23 - 05/03/2014 19:38
  • Last Updated - 05/03/2014 19:23
Avalanche Planned Follow-up Maintenance (Resolved)
  • Priority - Medium
  • Note:  This is now complete.  I'll leave this posting up for the rest of today.

    There was an extended outage to server Avalanche on Friday, April 25 caused by multiple hardware failures on the parent server.  Whereas this has been addressed, we will be migrating Avalanche to a new parent server starting at 4am Eastern, Sunday, April 27, 2014.

    For details of this planned maintenance, please visit:

    http://www.charlottezweb.com/forums/index.php?topic=1967.0

    For details around the extended outage on Friday, April 26, please visit:

    http://www.charlottezweb.com/forums/index.php?topic=1968.0

    A notification of the information above has been emailed to all customers with active accounts on Avalanche.

  • Date - 04/27/2014 04:00 - 04/29/2014 06:33
  • Last Updated - 04/27/2014 11:25
Avalanche Server Outage (Resolved)
  • Priority - High
  • Initial Issue Description - This server initially became unresponsive and required a reboot just before 8am Eastern today.  The reboot completed but the startup required a file system check which lengthened the time to get it back online. 

    Update 9:50am Eastern - the dc confirmed it is running a file system check after the reboot.  These can take some time to complete.  Given how long it's been already (approx 50 minutes since the reboot), it should complete fairly soon.

    Update 10:17am Eastern - the file system check appears to have completed and services are starting to come up now.  Loads are very high due to the downtime and everyone trying to reconnect repeatedly so it will take some time for it to stabilize.

    Update 10:35am Eastern - I see that the datacenter has initiated another reboot just now.  I'm awaiting an update from them on why that was initiated.

    Update 10:45am Eastern - I've spoken to the datacenter.  The loads after reboot spiked so high that it became unresponsive again. After the current reboot we're going to temporarily block web traffic so they can get in and diagnose what's driving the loads.

    Update 11am Eastern - The server is up however we're blocking all HTTP traffic while they investigate the load issue.  This means that your sites are still down but the server itself is up and running again.  I hope to have their update shortly.

    Update 11:50am Eastern - The firewall has been updated to re-allow HTTP traffic again.  Loads are very high (but are dropping) due to the influx of inbound traffic.  Hopefully we will see stabilization soon.  I know that many sites are not loading due to the resources required to generate them.  This should correct itself as the server stabilizes.

    Update 1:20pm Eastern - The datacenter is still actively working on this server.  Whereas it's technically up, it's essentially unable to serve sites due to high load.
     
    Update 2:50pm Eastern - Unfortunately the server required another reboot due to load spike (unresponsiveness) and it's running an automated file system check again.  As mentioned before, a file system check can take anywhere from 15 minutes or so to much longer. Earlier today it took just over an hour before access was restored.  The tech onsite I'm working with is connected to the server so he'll know the moment that completes so he can login and attempt to isolate the load cause before it spikes again.  

    Update 3:20pm Eastern - The datacenter is now aware of an issue with the parent server that Avalanche runs from.  They are performing emergency maintenance right now on it.  Hopefully that is the cause of what we've been experiencing today (vs. an issue with Avalanche itself).   We'll know if that's the case once that maintenance is complete.
     
    Update 3:35pm Eastern - The datacenter is quite confident the issue we're seeing is due to the parent issue and not with Avalanche.  Once they complete their emergency maintenance (currently in progress) we'll know for sure.  They will also provide a root cause for us. 

    Update 4:50pm Eastern - The datacenter maintenance on the parent server is still in progress.  I will update this once they've advised of completion. 
     
    Update 6:20pm Eastern - I've been advised that they're currently rebuilding (cloning) the parent server.  If all goes as planned they're hoping to have that restored in the next 1-2 hours.
        
    Update 9:25pm Eastern - Here is the present status as of right now.  The parent server suffered from a hard drive hardware failure that could not wait for a normal maintenance window or process to address.  That led to the need to do a complete rebuild/clone of the parent as I mentioned above.  Given the massive amounts of data involved in something like this, it's taking time to restore everything.  The parent server has a configuration that uses four hard drives.  The first drive has been restored.  At present, they are rebuilding the second drive and that should take approximately another hour.  When that drive is complete, there's a chance Avalanche will be part of the first and second drive space to where we could be back online while drives 3 and 4 rebuild in the background.  If we're part of the 3rd/4th drive space, however, we would remain down for the timeframe needed to rebuild drives 3 and 4.  That could be several more hours.  
     
    To summarize - the parent server suffered a hardware problem related to storage and a rebuild of that is ongoing.
     
    Update 9:45pm Eastern - Avalanche is back online.  I'm able to load a few sites but they're a little slow which is to be expected given traffic and load after being down all day.  I don't know if this means the maintenance that impacts us is complete or not.  I will attempt to confirm.
         

    Update 8:10am Eastern, April 26 - Avalanche has remained up overnight while the datacenter continued their rebuild.  It was fortunate that we were setup to where we fell into the first half of the restore.  Whereas the parent is up and not under an immediate threat of disruption, they would like to migrate us to a new parent that's more stable.  We've had to do this in the past on a few of our other instances.  It's a normal proces they do when they need to upgrade or perform (non-emergency) maintenance on a parent.  This involves an initial brief outage (usually around 15 minutes) and then the server is back up and online while data transfers in the background to the new parent.  The transfer can take anywhere from a few hours to 10+.  During this time there's typically no performance issue experienced and sites will be online.  When the transfer is complete, a second outage occurs to sync up and finalize the move.   They would like to schedule this for 12am Monday morning however I'm looking into possibly doing it sometime late tonight so it completes over the weekend.  I don't want to impact another business day (especially a Monday) if avoidable.  I will confirm here once I have full details.  I will also be moving all of these updates into a thread on our forum and will send a summary email to everyone with an account on Avalanche.
          
    Thank you for your ongoing patience.  Hardware-related issues like this are never pleasant.
     
    Regards,
    Jason

  • Date - 04/25/2014 09:15 - 04/26/2014 16:30
  • Last Updated - 04/26/2014 08:16
Thundersnow: PHP/MySQL Upgrade (Resolved)
Sandstorm: PHP/MySQL Upgrade (Resolved)
Avalanche: PHP/MySQL Upgrade (Resolved)
Thunderbolt - Reboot for security update (Resolved)
  • Priority - Medium
  • Affecting Server - Thunderbolt
  • We will be performing a security update and rebooting server "Thunderbolt."

    The reboot will likely occur within the hour however I made the decision to proceed now vs. scheduling this for off-hours due to the security issue involved.

    Please visit this thread for details:

    http://www.charlottezweb.com/forums/index.php?topic=1955.0

  • Date - 04/09/2014 16:37 - 04/10/2014 07:34
  • Last Updated - 04/09/2014 16:41
Emergency (extended) maintenance (Resolved)
  • Priority - High
  • This maintenance is complete.   I will be posting a full write-up with details on what occurred later today in this post on our forum:

    http://www.charlottezweb.com/forums/index.php?topic=1953.0

    -------------


    We're currently working on Tsunami.  PHP-driven sites are currently not loading related to an emergency software upgrade that was needed. 

    Update 12:15am Eastern -- this is still being actively worked.  We hope to have resolution shortly.

    Thank you for your patience.

  • Date - 03/26/2014 22:21 - 03/27/2014 04:00
  • Last Updated - 03/27/2014 04:23
Planned server outage (Resolved)
Crash - Pending investigation (Resolved)
  • Priority - High
  • Sandstorm has crashed.  Before I reboot, I'm waiting on the datacenter to login and gather logs as this has happened a few times already this week.

  • Date - 01/28/2014 12:59 - 01/28/2014 13:20
  • Last Updated - 01/28/2014 13:00
Reboot in progress (Resolved)
  • Priority - High
  • Sandstorm is currently being rebooted after becoming unresponsive.

  • Date - 01/27/2014 21:37 - 01/27/2014 21:42
  • Last Updated - 01/27/2014 21:38
MySQL Upgrade (Resolved)
  • Priority - Medium
  • An upgrade of MySQL is scheduled for Blackice for this coming Sunday morning.  Please visit this thread on our forum for further details:

    http://www.charlottezweb.com/forums/index.php?topic=1944.0

  • Date - 01/19/2014 05:00 - 01/19/2014 11:07
  • Last Updated - 01/15/2014 20:15
Heatwave / Deluge outage (Resolved)
  • Priority - High
  • Affecting Other - Servers Heatwave & Deluge
  • Servers Heatwave and Deluge became inaccessible around 10 minutes back.  I'm looking into a possible network/datacenter issue now.

    Will update once details are known.

    Thanks

  • Date - 12/23/2013 09:11 - 12/23/2013 09:22
  • Last Updated - 12/23/2013 09:12
Tsunami reboot in progress (Resolved)
  • Priority - High
  • Tsunami quit responding and is in the process of being rebooted.

  • Date - 12/04/2013 19:33 - 12/04/2013 20:11
  • Last Updated - 12/04/2013 19:34
Load spike due to backup (Resolved)
  • Priority - High
  • Affecting Server -
  • Wildfire is currently running at very high loads due to a backup process.  It should stablilize shortly.

    Update:  The server became unresponsive a few moments ago and is currently rebooting.

    Sandstorm was experiencing this as well but has now stabilized.

  • Date - 11/19/2013 11:29 - 11/19/2013 11:49
  • Last Updated - 11/19/2013 11:47
Scheduled network maintenance (Resolved)
  • Priority - Low
  • Affecting Other - Multiple servers
  • Scheduled maintenance this coming Thursday (11:30pm Eastern) potentially impacting multiple servers.

    Please visit this thread for details:

    http://www.charlottezweb.com/forums/index.php?topic=1934.0

  • Date - 10/31/2013 23:30 - 11/02/2013 09:14
  • Last Updated - 10/28/2013 17:57
Two scheduled DC outages for Deluge (Resolved)
  • Priority - Medium
  • Deluge will be undergoing a server move starting at 4am, Sept 25.  There will be a reboot followed by normal operation (while data transfers) and then a second reboot later in the day to complete.   This is ONLY for our server "Deluge."

  • Date - 09/25/2013 04:00 - 10/02/2013 06:07
  • Last Updated - 09/24/2013 21:38
High loads on Tempest (Resolved)
  • Priority - Medium
  • We're working on controlling high email-related loads on Tempest right now.  You may be seeing slow server performance as a result of the mail processing.

  • Date - 09/11/2013 15:56 - 09/11/2013 17:21
  • Last Updated - 09/11/2013 15:57
Emergency datacenter network maintenance (Resolved)
  • Priority - Medium
  • Affecting Other - Multiple servers
  • We've received notification of an emergency datacenter maintenance window that may impact any of the following servers for a period on Friday, September 6, 2013 starting at 10pm Eastern.

        Jetstream
        Monsoon
        Sirocco
        Supercell
        Tempest
        Tsunami

    Please visit this thread on our forum for additional details:

    http://www.charlottezweb.com/forums/index.php?topic=1915.0

  • Date - 09/06/2013 22:00 - 09/07/2013 09:14
  • Last Updated - 09/05/2013 17:17
Reboot in progress (Resolved)
  • Priority - High
  • Server "Tempest" is in the process of being rebooted.  

  • Date - 07/22/2013 08:55 - 07/22/2013 09:16
  • Last Updated - 07/22/2013 08:55
High loads on Monsoon (Resolved)
  • Priority - High
  • We're working on isolating high loads on Monsoon.  Mysql is currently not responding due to this.  Sites not using mysql are online but running slowly at this time.


    We hope to get this sorted quickly.

  • Date - 07/19/2013 12:07 - 07/19/2013 14:11
  • Last Updated - 07/19/2013 12:08
Planned outage / maintenance (Resolved)
  • Priority - Medium
  • Server: Sandstorm
    Date:  Monday, June 10, 2013 -- starting at 11pm EST

    There will be two planned outages to server "Sandstorm" as part of a scheduled maintenance this next Monday, June 10 starting at 11pm EST. 

    Please see this forum post for full details:

    http://www.charlottezweb.com/forums/index.php?topic=1897.0

    Thanks,
    Jason

  • Date - 06/10/2013 23:00 - 06/11/2013 15:47
  • Last Updated - 06/05/2013 20:27
Monsoon server maintenance - Emergency (Resolved)
  • Priority - High
  • Monsoon has been experiencing symptoms of primary hard drive failure today.  We're going to be starting emergency drive replacement shortly.

    Please visit this thread for full details and updates:

    http://www.charlottezweb.com/forums/index.php?topic=1894.0

  • Date - 05/06/2013 08:34 - 05/06/2013 22:02
  • Last Updated - 05/06/2013 16:07
High loads due to quota maintenance (Resolved)
  • Priority - Medium
  • We're running a quota fix on Jetstream which is driving high loads.  It should stabilize shortly.

  • Date - 04/23/2013 11:26 - 04/23/2013 11:35
  • Last Updated - 04/23/2013 11:26
Scheduled MySQL maintenance - Tuesday, 5am ES (Resolved)
  • Priority - High
  • On Tuesday, April 23 at 5am EST, we will be doing required MySQL maintenance on server "Supercell." 

    This maintenance will require MySQL services to be stopped for approx 30-40 minutes.  Worst-case, we may be looking up to an hour though I don't expect it to take that long.

    Any sites or scripts on Supercell that use MySQL databases for functionality will experience an outage during this time.

    If you have any questions, please visit this thread:

    http://www.charlottezweb.com/forums/index.php?topic=1890.0

    Thank you,
    Jason


  • Date - 04/23/2013 05:00 - 04/23/2013 06:43
  • Last Updated - 04/21/2013 10:40
Planned datacenter network maintenance (Resolved)
  • Priority - Medium
  • Affecting Other - Multiple servers
  • One of our datacenters will be performing some network maintenance on Sunday, April 21th from approx 2am - 3am EST.

    This may impact the following servers:

    Jetstream
    Monsoon
    Sirocco
    Supercell
    Tempest
    Tsunami


    Window duration: 1 hours

    Anticipated impact: 15 minutes


    Description:
    Upstream Network Maintenance
     
    Impacted services:
    Atlanta Gold and Silver network connections
     
    Details of Maintenance:
    Hardware upgrades will be carried out on equipment carrying connections for nLayer and some peering links, resulting in a short disruption of connectivity for traffic passing through these connections.  Some network instability and suboptimal routing may occur during the scheduled window as traffic is rerouted through other upstream providers.

  • Date - 04/21/2013 02:00 - 04/21/2013 10:32
  • Last Updated - 04/17/2013 18:10
Server unresponsive, working with dc (Resolved)
  • Priority - Critical
  • Supercell is currently experiencing issues.  We're working with the datacenter now.

  • Date - 04/16/2013 15:41 - 04/16/2013 16:07
  • Last Updated - 04/16/2013 15:42
Reboot in progress (Resolved)
  • Priority - High
  • We're rebooting Avalanche right now. 

    Update:  The reboot did not resolve.  We're working with the datacenter now.  Updates to follow as I get them.

    Thanks,
    Jason

  • Date - 03/08/2013 18:25 - 03/08/2013 18:41
  • Last Updated - 03/08/2013 18:36
Investigating high loads (Resolved)
  • Priority - High
  • Avalanche is currently experiencing high loads.  We're investigating this now.

    Thank you,
    Jason

  • Date - 03/05/2013 00:18 - 03/06/2013 19:54
  • Last Updated - 03/05/2013 00:19
Server reboot (Resolved)
  • Priority - High
  • Avalanche is being rebooted right now.

  • Date - 02/21/2013 19:41 - 02/21/2013 19:48
  • Last Updated - 02/21/2013 19:41
PHP upgrade on Supercell (Resolved)
Pingdom monitoring page presently offline (Resolved)
  • Priority - Medium
  • Affecting Other - Pingdom Uptime Status Site
  • The website we use for uptime stats provided by Pingdom is presently not loading.

    Their main site is up as is my admin area but the "public-facing" stats page is not loading.

    I've reported this issue to them to address.

  • Date - 01/12/2013 16:01 - 01/12/2013 17:11
  • Last Updated - 01/12/2013 16:47
Multiple Reboots (Resolved)
  • Priority - Critical
  • There have been multiple reboots of server "Avalanche" in the past 24 hours.  Full detail are available on our forum here:

    http://www.charlottezweb.com/forums/index.php?topic=1874.0

    Regards,
    Jason

  • Date - 01/12/2013 15:40 - 01/12/2013 17:11
  • Last Updated - 01/12/2013 16:45
Reboot in progress (Resolved)
  • Priority - High
  • Server "Avalanche" is currently in the progress of being rebooted.

  • Date - 01/11/2013 19:33 - 01/11/2013 19:35
  • Last Updated - 01/11/2013 19:33
Charlottezweb.com site migration (Resolved)
  • Priority - Medium
  • I'm in the progress of migrating Charlottezweb.com (the website) to a newer server.  This does not impact any customer accounts -- this is only for Charlottezweb.com.

    The Client Area (billing/support system) and Forum will be locked during the move.  If you need to contact support, please email support@charlottezweb.com.

  • Date - 01/06/2013 07:23 - 01/06/2013 10:13
  • Last Updated - 01/06/2013 07:25
Reboot in progress (Resolved)
  • Priority - High
  • Server "Monsoon" is currently being rebooted.  

    ----------
    Update 9:15am EST:  
    This is still being worked by the datacenter.  The initial reboot did not complete successfully.
    ----------
    Update 9:25am EST: 
    The server failed an automatic file system check upon the first reboot.  We're going to run a force version of the same command now.  
    Realistically, this could take upwards of 30 minutes to an hour typically.

  • Date - 12/11/2012 08:25 - 12/11/2012 10:26
  • Last Updated - 12/11/2012 09:25
PHP upgrade (Resolved)
Planned Datacenter Maintenance (Resolved)
Server reboot (Resolved)
  • Priority - High
  • Avalanche became unresponsive -- We're rebooting it right now.

  • Date - 11/07/2012 21:03 - 11/07/2012 21:52
  • Last Updated - 11/07/2012 21:03
Reminder: Blizzard server shutdown Oct 30 (Resolved)
  • Priority - Medium
  • Affecting System - Blizzard
  • Reminder that our server Blizzard is being retired tomorrow.


    Please visit this thread for full details:


    Thanks,
    Jason

  • Date - 10/30/2012 01:00 - 11/01/2012 09:44
  • Last Updated - 10/29/2012 10:47
Server reboot (Resolved)
  • Priority - Critical
  • Tempest has been rebooted after becoming unresponsive.  It's back up and accessible however sites aren't loading yet.  We're currently investigating.

    Thanks,
    Jason

  • Date - 09/20/2012 05:50 - 09/20/2012 06:39
  • Last Updated - 09/20/2012 06:37
PHP/Apache change - brief maintenance planned (Resolved)
  • Priority - Medium
  • I'm scheduling a 30 minute maintenance window for tomorrow evening at 10pm EST for our server Tsunami.  We're going to be altering a setting that has been causing one of our storage partitions to fill up incorrectly.  It's possible there may be a 5-15 minute service disruption to web services.

    If you have any questions, there's a forum notice here:

    http://www.charlottezweb.com/forums/index.php?topic=1840.0

    Thank you,
    Jason

  • Date - 09/11/2012 20:00 - 09/11/2012 23:00
  • Last Updated - 09/10/2012 21:08
Emergency Maintenance (Resolved)
  • Priority - Medium
  • Affecting Other - Multiple servers
  • One of our datacenters just started some emergency maintenance that required a reboot of our servers.  This was not planned and unfortunately I wasn't advised until I logged into their site to confirm.

    They are essentially rebooting the servers after doing the work required on their end.  Two servers were down about 10 minutes (rebooting) and are up again now.  It looks like our other servers with them are now in the process of rebooting.

    They are very good about scheduling maintenance and providing advance notice so I assume this was for something they felt was urgent.

    Here is the forum thread where I'll post an update once complete:

    http://www.charlottezweb.com/forums/index.php?topic=1838.0

    Thank you,
    Jason

  • Date - 09/07/2012 18:39 - 09/07/2012 18:54
  • Last Updated - 09/07/2012 18:47
Planned Maintenance (Resolved)
Sporadic connectivity issue (Resolved)
  • Priority - High
  • Starting just before midnight last night, Tsunami has been facing some sporadic network issues. Services may appear to be up/down as we continue our investigation.

    Update 6:45pm EST:  I've posted a summary of this issue here: 
    http://www.charlottezweb.com/forums/index.php?topic=1828.0

    Update 11:35pm EST
    We believe this to be resolved.  If you're still experiencing issues, please open a ticket, post in the thread above or email us.

  • Date - 08/06/2012 10:09 - 08/06/2012 23:36
  • Last Updated - 08/07/2012 09:58
Load / Mysql Issues (Resolved)
  • Priority - High
  • We are working on issues with Wildfire currently related to high loads and mysql. Details will follow as we have them.

    Update: Mysql service was restored awhile ago but loads remain high. We've found an issue that we're working to address now.

    Thank you for your patience!

    Regards,
    Jason

  • Date - 07/27/2012 22:36 - 07/28/2012 13:24
  • Last Updated - 07/27/2012 23:50
Reboot (Resolved)
  • Priority - Critical
  • This server is currently being rebooted.

  • Date - 06/10/2012 18:37 - 06/10/2012 18:55
  • Last Updated - 06/10/2012 18:38
Server reboot in progress (Resolved)
  • Priority - High
  • Avalanche is currently being rebooted.

  • Date - 06/07/2012 08:20 - 06/07/2012 08:29
  • Last Updated - 06/07/2012 08:20
Jetstream - Hard Drive Replacement (Resolved)
  • Priority - Critical
  • This coming Friday evening (April 13, 2012) at approximately 11pm EST, our server "Jetstream" will begin planned emergency maintenance to replace its primary hard drive.

    For full details or to ask questions, please visit this thread for full details.

    An email notification has been sent to all customers on Jetstream directing them to the forum post above.

  • Date - 04/13/2012 23:00
  • Last Updated - 04/14/2012 07:54
Wildfire outage - High loads, slow response t (Resolved)
  • Priority - High
  • Wildfire is still experiencing high loads. This is being investigated. Please view this thread for full details or discussion:

    http://www.charlottezweb.com/forums/index.php?topic=1792.0

  • Date - 03/16/2012 13:29 - 03/16/2012 15:43
  • Last Updated - 03/16/2012 17:12
Planned upstream network maintenance (Resolved)
  • Priority - Low
  • Affecting Other - Multiple servers hosted in the Atlanta datacenter
  • Please see this forum post for full details:

    http://www.charlottezweb.com/forums/index.php?topic=1791.0

  • Date - 03/16/2012 23:00 - 03/21/2012 11:40
  • Last Updated - 03/13/2012 20:11
Regional network slowness (Resolved)
  • Priority - High
  • Affecting Other - External network speeds
  • I'm getting reports from some customers of slow or unreachable sites. It appears there's a larger (perhaps regional) network issue presently.

    All servers are up and running fine however depending on your location, you may be facing issues reaching them. This is not a server or even a datacenter issue from what I'm seeing in tracerts. I'm seeing slowness reaching our servers in Atlanta and if I trace to our severs in Michigan, it's also very slow until it gets a few states north of here at which point the speeds increase dramatically. Because of that, I believe it's a network/provider issue somewhere in this general southeast region.

    I will add to this post if I can find information online about anything currently happening.

  • Date - 03/11/2012 11:42 - 03/11/2012 13:02
  • Last Updated - 03/11/2012 11:47
Wildfire and Avalanche - Network issue (Resolved)
  • Priority - Critical
  • Update: Our servers Avalanche and Wildfire are responding very slowly due to a network issue at the datacenter.

    Details here:

    http://www.charlottezweb.com/forums/index.php?topic=1788.0

  • Date - 03/09/2012 00:26
  • Last Updated - 03/09/2012 08:05
Planned upsteam network maintenance for XO (Resolved)
  • Priority - Low
  • Affecting Other - AtlantaNAP (Netdepot/GNAX)
  • For more details (including potentially impacted servers) or to post questions, please visit this post on our forum.

    One of our datacenters has advised us today of a pending upstream network maintenance window for this coming Friday night at approx 11pm EST. Normally we don't see any impact from these but I like to post them in the event you experience slower response times during the maintenance window.

    Maintenance Notification:
    Facility: AtlantaNAP
    Time: 23:00 EST on Friday March 2nd.
    Window duration: 5 hours
    Anticipated impact: Routing instability

  • Date - 03/02/2012 23:00 - 03/03/2012 04:00
  • Last Updated - 03/04/2012 15:43
Avalanche - Scheduled Datacenter Maintenance (Resolved)
  • Priority - Medium
  • This has been rescheduled for midnight Sunday evening (12:01am EST, Monday, March 5)

    This coming Saturday morning (March 3, 2012) at approximately 4am EST, Avalanche will experience some scheduled datacenter maintenance that will result in two periods of outage separated by some potentially slower response times in between. Each outage should be under 30 minutes however it could be less.

    For full details or to ask questions, please visit this thread for full details.

    An email notification has been sent to all customers on Avalanche directing them to the forum post above.

  • Date - 03/05/2012 00:00 - 03/05/2012 10:31
  • Last Updated - 03/03/2012 05:08
Tsunami - Service outage (Resolved)
  • Priority - Critical
  • We're working on an issue with Tsunami currently. It looks like a process just used the full temp partition.

  • Date - 02/24/2012 20:19 - 02/24/2012 20:55
  • Last Updated - 02/24/2012 22:39
Blizzard - Outage (Resolved)
  • Priority - Critical
  • http://www.charlottezweb.com/forums/index.php?topic=1778.0

  • Date - 02/20/2012 15:45
  • Last Updated - 02/20/2012 15:55

Server Status

Below is a real-time overview of our servers where you can check if there's any known issues.

Server Name Online Server Load Uptime
Downpour
Landslide
Lightning
Stormsurge
Thunderbolt
Vortex

Charlottezweb utilizes third-party monitoring by Pingdom which is publicly available on our Pingdom site.

Pingdom tests connectivity to all of our servers from global monitoring stations every 1 minute meaning there's no doubt about your site's uptime.

*Server Load: Server load is a measure of the amount of computational work that a computer system performs. A general rule of thumb is that a server can safely support a load of 1 per core (processor). For example, if you have a quad-core (4 processor) server, a target load should be 4 at most. Our servers have at minimum 2 if not 8+ cores and should typically report loads under 2. It is possible that load spikes may occur throughout the day as resource-intensive processes run. Wikipedia Reference