Blizzard has been worked on today by Cpanel.net technicians to fix an ongoing issue. They are working on rebuilding the service which has been causing mysql to be up and down over the past few hours.
I authorized the fix to happen today not expecting more than a few minutes of outage so I am following up now to get a status on what's being done.
I will keep this thread updated as we investigate.
Cpanel is going to hopefully be wrapping this up quickly. They're going to be recompiling php momentarily which should wrap this up based on what they're advising.
A little background on this issue: This server has had a misconfiguration for the past few months where adding a database user to a database through cpanel only works in the "X" them. It wasn't working in RVskin or in X3. This isn't anything other than a nuissance so I haven't prioritized correcting it and the previous info I received on it wasn't helpful. Today I decided to get it resolved and have been working with Cpanel support directly to do so.
They found what they believe to be the problem and were going to correct that. From what they described, there should've been 2-3 minutes for mysql to restart and that was the assumption I was under in terms of any outage.
Apparently, that's not the case and mysql has been failing to load for at least a good 1-2 hours now. Other services are up -- just mysql has been impacted for the most part.
In order for their change to be complete, they wanted to recompile php so it will resync and that's what's being done now.
My apologies for this delayed outage. We're a bit at their mercy for speed and I was never given the impression this would cause such a significant issue or I would've scheduled it for a more ideal timeframe than midday on a Friday.
I will keep this thread updated as I receive more from them though I expect a resolution shortly.
Thanks for the updates
Thank you for the updates.
Please send along my sentiments to Cpanel:
GRRRRRR (what follows the growl should not be posted)
Honestly, I'm going to give them another several minutes and then I'm going to ask them to logout so we can do this ourselves. A recompile should take a matter of minutes.
My site is back :)
Yes, looks like the ones I've checked are functioning now.
I'll look for an update from cPanel and post here once I have confirmation.
Ok, cPanel has fixed the problem I initially reported and we're through with anyone doing anything further on this box.
Again, I apologize for the extended mysql downtime. Their initial diagnosis should've been a very minor change so I didn't think twice about approving it for immediate production.
On a positive note, this is likely tied into another error that has been happening on rare occasion for a few users regarding mysql unavailability errors. Essentially, before this fix, whm (cpanel) was reporting the current version of mysql as being different than what we were truly running. We've uninstalled/removed past versions and files and rebuilt it to bring everything back into sync.
Thanks Jason, my site is fine now! good work ;)