Tues, Sept 5, 2006 :: Cyclone httpd restarts

[Jason]

There have been about 4 httpd restarts this afternoon -- two of which were noted by Alertra.

Similarly, there was one yesterday very early morning.

09/05/2006 12:55:01 00:05:00 Connection to HTTP service failed 
09/05/2006 12:04:54 00:04:59 Connection to HTTP service failed 
09/04/2006 05:19:44 00:04:59 Connection to HTTP service failed 

These are due to mysql load usage and we're aware of the situation.

I'm keeping my eye on things this afternoon and making plans towards a solution.

In the meantime, the outages are standard in terms of a service restart.  They are a nuissance and certainly not par for the level of service I aim to provide so this will be handled as quickly as possible.

(The outages are typically 2-5 minutes in length and are automated restarts to keep the server from crashing).

Regards,
Jason

[Mark]

Haven't noticed, been in my e-mail most the day. Thanks for the heads up.

[WSA]

I've noticed.  It's actually doing it right now.

[Jason]

Update for Cyclone clients...

The loads are due to a new account that has very quickly outgrown the amount of resources the server can efficiently provide given the existing user base on it.  During normal times, this isn't an issue, but during high traffic times of the day, like right now for example, this is causing Hawkeye to restart httpd -- resulting in the several minute outages.

I'm in the process of ordering our next server which is where this account will be moved to as soon as it's humanly possible.  

In the meantime, if we get a few more restarts, I'll discuss other more immediate options to clearing this up.  I appreciate your patience.  Once that account's volume is moved to the new server, the service levels we've grown to expect from Cyclone will be back to their normal fast speeds. 

I've been in discussion with the client for the past several days and we're working through it together now.  For everyone's good, we'll likely be closing his forum down for a few days or redirecting traffic elsewhere.

Please let me know if you have any questions.  I'll keep this thread updated with anything new I get.

[Jason]

Update.  The account in question has been temporarily relocated to another server while our newest server is being built.

Loads have dropped back down to where they should be (which is a BIG drop from the past few days).

I appreciate everyone's patience and understanding.  You will be able to track it on next week's Alertra report, but I think we probably had close to ten short (2-5 minute) outages in the past 2 days on Cyclone.  They won't all show up on the report because some were shorter than the 5 minute interval Alertra uses.

We should be good to go now.  Thanks everyone!