Charlottezweb

Current News => News & Announcements => News Archive => Topic started by: Jason on June 19, 2006, 03:19:06 PM

Title: Thunder/Lightning :: Conversion to PHPSUEXEC
Post by: Jason on June 19, 2006, 03:19:06 PM
I will be enabling phpsuexec on Thunder and Lightning within the next 48 hours due to security concerns.   Thunder will be upgraded this evening.

Thunder:  Monday, June 19, 2006
Lightning:  Tuesday, June 20, 2006 or Wednesday* 

(Lightning may be pushed back depending on how smoothly Thunder transitions)

It is important that you read below to understand the impact this may have on your account.  Pay special attention to the section:  "What do I need to do to prepare?"

What is phpsuexec?
I found this text on multiple hosting sites so I'm not sure of the original author:
Quote
"When PHP runs as an Apache Module it executes as the user/group of the webserver which is usually "nobody". Under this mode, files or directories that you require your php scripts to write to need to have 777 permissions (read/write/execute at user/group/world level). This is not very secure because besides allowing the webserver to write to the file it also allows anyone else to read or write to the file.

With PHP running as CGI with suexec enabled your php scripts now execute under your user/group level. Files or directories that you require your php scripts to write to no longer need to have 777 permissions. In fact, having 777 permissions on your scripts or the directories they reside in will not run and will instead cause a 500 internal server error when attempting to execute them to protect you from someone abusing your scripts. Your scripts and directories can have a maximum of 755 permissions (read/write/execute by you, read/execute by everyone else). PHP running as CGI/suexec is much more secure than the older Apache module method."

To cut to the chase, this script will GREATLY enhance our ability to keep the server secure and troubleshoot if the need arises.  In the event that any sort of script compromise occurs, spam is being sent, an accout is compromised, etc, phpsuexec provides more containment from a security perspective and also allows us to very quickly pinpoint these problems.  We're seeing issues from time to time on Thunder and Lightning that are taking us longer than needed to diagnose because of a lack of this.  I've been researching this for months (our latest server, Cyclone, was converted back in February), many hosts have moved this way already and I think it's certainly the time to get onboard. 

What do I need to do to prepare?
Before your server is converted, you will want to adjust permissions on your files.  Due to the way phpsuexec works, if you have files that are chmodded to 777 (which is often the standard for many scripts), you will get server error messages after phpsuexec is enabled.   You will want to lower those to 755.

I found this general troubleshooting info all over the web (not sure of the original source):
Quote
Troubleshooting
HELP my php script doesn't work or I have an error message.

1. Check that the php script that you are attempting to execute has permissions of no more than 755 - 644 will work just fine normally, this is not something that will need to be changed in most cases.

2. Check that the directory permissions that the script resides within is set to a maximum of 755. This also includes directories that the script would need to have access to also.

3. Check that you do not have a .htaccess file with php_values within it. They will cause a 500 Internal server error, when attempting to execute the script.

The php_values will need to be removed from your .htaccess file and a php.ini put in its place, containing the php directives as explained above.

How will this impact future clients or new scripts I install?
Once phpsuexec is in place and you've converted any existing scripts, you should hardly notice it's there.  It's not a new concept so most all major scripts should've been supporting it a long time ago.  If you have anything custom in your account, you may need to make sure it's workable.  Otherwise, there should be little to no noticeable impact to new users or new script additions.  The initial "learning phase" of switching permissions on existing scripts is typically the hard part.  Even then, it may be a breeze.  We'll find out.  :)

Why such short notice?
We've been looking into enabling phpsuexec on all servers since 2005.  However, it's been easier to fight small fires as they arise rather than making the jump and inconveniencing you with having to alter your sites.  This process has become time consuming and more importantly it means that finding problems when they arise takes longer -- resulting in longer outages in some cases.  An event in the last day has prompted me to react with less notice than I would normally hope to provide.  I always try to give as much of an advanced warning as I can pass on, however, this is a situation that I believe is best addressed immediately.

Just in case some of you don't get the notice before the upgrade tonight, I will be running some global commands on Thunder to adjust permissions before the upgrade.  That should hopefully prevent errors with users running files chmodded to 777.

Will you be supporting this?
Our server Cyclone had phpsuexec enabled back in February and it has performed quite well.  It makes it possible to see who is doing what on the server quite clearly which is key to fast troubleshooting if needed.  You can read about how the change on that server went here:

http://www.charlottezweb.com/forums/index.php?topic=356.15

I will be posting again to this thread once I start the installations.  I will be emailing a notice of this thread to all clients this afternoon so that everyone can post their comments/concerns here.

Please post your support questions to the forum (here) so that future users can use them going forward.  :)


--------------------------

More information on phpsuexec:

http://www.nsdesign.net/cgi-bin/newdesk/new/cgi-bin/kb.cgi?do=read&id=94&lang=en

http://www.cablan.net/cablan/What_is_PHPSuexec.449.0.html

https://emaxhosting.com/support/index.php?_m=knowledgebase&_a=viewarticle&kbarticleid=411
Title: Re: Thunder/Lightning :: Conversion to PHPSUEXEC
Post by: CountryLady on June 19, 2006, 04:08:28 PM
Thanks for keeping us all aware of what's going on and why, Jason -- its very rare in today's cyber world. Website and server security is a primary concern for all of us. Knowing you're there to help anyone who has difficulties is MOST reassuring.

Cheers for You, Jason~! I'm so happy I found Charlottezweb Hosting.


Title: Re: Thunder/Lightning :: Conversion to PHPSUEXEC
Post by: The Librarian on June 19, 2006, 05:11:43 PM
Hi Jason,

I dont know what server I'm on but I thought we had already done this back in Feb? So maybe I am ok.  I wont be around much this week as I am moving house, so maybe you can tell me if I am on these servers or not, and then I can start worrying later?

thanks
Title: Re: Thunder/Lightning :: Conversion to PHPSUEXEC
Post by: Jason on June 19, 2006, 05:29:22 PM
Quote from: ello on June 19, 2006, 05:11:43 PM
I dont know what server I'm on but I thought we had already done this back in Feb? So maybe I am ok.  I wont be around much this week as I am moving house, so maybe you can tell me if I am on these servers or not, and then I can start worrying later?

You're on Cyclone so this won't impact you.

Cheers,
Jason
Title: Re: Thunder/Lightning :: Conversion to PHPSUEXEC
Post by: Jason on June 19, 2006, 06:53:39 PM
We are going to begin the Thunder conversion now...
Title: Re: Thunder/Lightning :: Conversion to PHPSUEXEC
Post by: Jason on June 19, 2006, 07:28:28 PM
Thunder's conversion to phpsuexec is complete.  Please check your sites for functionality and post here if needed.

Thanks,
Jason
Title: Re: Thunder/Lightning :: Conversion to PHPSUEXEC
Post by: Jason on June 19, 2006, 07:45:28 PM
Note:  If you have chmodded your files to 755 (at the highest) and are still experiencing problems, make sure you don't have any php commands located in your htaccess file.  If you rename your htaccess file, that will likely fix your issues if you're having any.

Title: Re: Thunder/Lightning :: Conversion to PHPSUEXEC
Post by: Powerbob on June 20, 2006, 01:28:11 AM
Hi, well a little more warning would have been very helpfull. Instead I find the site down, and am getting sms after sms telling me  members cannot access the site! >:( I am not a very happy bunny this day!

As to the explanation, well I am sorry but for us non-geeks its a joke! I am still not clear as to what to do to bring my site back on-line.

Some help here would go a long way to making amends!


Powerbob (ofbboard.com stiil down)
Title: Re: Thunder/Lightning :: Conversion to PHPSUEXEC
Post by: Kris on June 20, 2006, 01:32:00 AM
Man,

I don't like this at all.
(just emailed you too)

I'm abroad for business and get sms that all sites are down.

Next time you upgrade, you should just warn everyone at least one week before. Resources have to be planned etc. Putting a message here and a few hours later upgrade is just not done.

I like your hosting and service but the last month you've done this already twice... Last time I had to upgrade a lot of my sources and I lost a lot of data because of that... :(

Might be time changing hosts...
Title: Re: Thunder/Lightning :: Conversion to PHPSUEXEC
Post by: Jason on June 20, 2006, 08:07:21 AM
Quote from: Powerbob on June 20, 2006, 01:28:11 AM
Hi, well a little more warning would have been very helpfull. Instead I find the site down, and am getting sms after sms telling me  members cannot access the site! >:( I am not a very happy bunny this day!

As to the explanation, well I am sorry but for us non-geeks its a joke! I am still not clear as to what to do to bring my site back on-line.

Some help here would go a long way to making amends!


Powerbob (ofbboard.com stiil down)

Hi Powerbob.  I completely understand your frustration, but time was not an option in this case unfortunately.  I'll be happy to go into details here after we sort this out.

Make sure your files are no greater than 755 from a chmod perspective.  I ran a script to do this for everyone yesterday before the upgrade, so that is not likely your problem. 

If that doesn't do it, make sure you don't have an htaccess file in your public_html folder.  If you do, try renaming it.  That will likely solve the problem.

If not, please post here or email me directly with a path to the installation having issues.

I'm going to look into your account for you now.

- - - - -

Quote from: Kris on June 20, 2006, 01:32:00 AM
Man,

I don't like this at all.
(just emailed you too)

I'm abroad for business and get sms that all sites are down.

Next time you upgrade, you should just warn everyone at least one week before. Resources have to be planned etc. Putting a message here and a few hours later upgrade is just not done.

I like your hosting and service but the last month you've done this already twice... Last time I had to upgrade a lot of my sources and I lost a lot of data because of that... :(

Might be time changing hosts...

Hi Kris,

I understand your frustration, but if you knew my reasoning, you'd understand.  In fact, the server itself was almost shut down this morning due to the reason behind my actions.  You pay me to keep your site online.  You have to trust that my actions are always going to be geared towards making the best decisions to keep that possible.  As I know you're aware, I always provide as much time as possible -- typically a week or so -- but that was not an option this time as I'll be happy to explain later.  I can't go into further detail at this time, but I will be happy to do so once this has been completed. 

As for the last upgrade to php5 -- I gave about a week's notice.  Your need to upgrade then was because of an outdated script.  You would've faced that same issue no matter who your host is. 

Have you followed the instructions above?  If your permissions are correct, have you renamed/removed any htaccess files you have in place?

I'm going to look into your account for you now.

Regards,
Jason
Title: Re: Thunder/Lightning :: Conversion to PHPSUEXEC
Post by: Powerbob on June 20, 2006, 08:14:35 AM
Well I found several files and dirs at 777 which I changed by hand to 755. 6 hours of frustrating looking and changing :'(


Powerbob
Title: Re: Thunder/Lightning :: Conversion to PHPSUEXEC
Post by: Jason on June 20, 2006, 08:18:59 AM
Kris,

Your forum is up.  Please confirm.

Powerbob,

Looking into your's now.

Title: Re: Thunder/Lightning :: Conversion to PHPSUEXEC
Post by: Jason on June 20, 2006, 08:21:58 AM
Powerbob,

Your forum is up.  Please confirm.

Regards,
Jason
Title: Re: Thunder/Lightning :: Conversion to PHPSUEXEC
Post by: Kris on June 20, 2006, 09:21:29 AM
Quote from: Jason on June 20, 2006, 08:18:59 AM
Kris,

Your forum is up.  Please confirm.

Powerbob,

Looking into your's now.


I've done it myself.
Have to keep my clients happy.
Title: Re: Thunder/Lightning :: Conversion to PHPSUEXEC
Post by: tmlfever on June 20, 2006, 02:51:28 PM
Off topic sorry, Jason did you get my PM?
Title: Re: Thunder/Lightning :: Conversion to PHPSUEXEC
Post by: Jason on June 20, 2006, 02:56:57 PM
It appears that the overall conversion of Thunder to phpsuexec has gone better than I expected.  The immediate threat requiring this change last night was on that server and prevented me from delaying -- even by a few days. 

I will move Lightning's conversion to Wednesday night for everyone on our remaining server.  I will look to start the fix there around 7 or 8pm EST.  There is less of an immediate need to react on that machine, though just to be sure, I'd like to have it updated as soon as possible.

If you have any concerns, please feel free to post them here or email me directly.

Once that server has been updated, I will provide more details into the change and why we acted as quickly as we did. I apologize again for the few of you who had incompatabilities we worked out today.  I understand your concern and wish I could've provided more lead time but I unfortunately have to keep the server (and all clients on it) in mind when making decisions like this.  When I went to bed, I was under the impression that all sites were up and fine because I went down the account listing and viewed all sites on Thunder one by one.  I missed the two sites above due to confusion on one of them due to the error message and the fact that the other one is a site under a different name than what my account list shows.  Had I known then that they were down, we could've corrected it within a few minutes.  I know it doesn't make up for the outage, but 2 sites out of around 90 on this box -- both of which were corrected within 10 minutes once I was aware this morning -- is much better than the server going down for an extended period of time when it could've been prevented by a proactive approach on my part.

FYI, I have discovered that Menalto's Gallery script will likely need to be adjusted after the conversion to work properly.  I can do this for you in a matter of minutes if you let me know ahead of time that you're running it.  For those of you who are web savvy, you will essentially have to clear out the htaccess file it uses (Remember:  htaccess files with php code in them will cause your site not to work after phpsuexec) and then rerun Gallery's configuration wizard.  File ownership may need to be adjusted as well (which I'll have to do for you) so if you let me know in advance, I'll take care of it for you.  I fixed about 7 instances of it last night pretty quickly.

Thank you,
Jason
Title: Re: Thunder/Lightning :: Conversion to PHPSUEXEC
Post by: Jason on June 20, 2006, 03:00:13 PM
Quote from: tmlfever on June 20, 2006, 02:51:28 PM
Off topic sorry, Jason did you get my PM?

Yes I did.  I will respond now.  Thanks!
Title: Re: Thunder/Lightning :: Conversion to PHPSUEXEC
Post by: bitwiz44 on June 20, 2006, 10:36:44 PM
Umm.. i have a few 777's the when I try to change it I get "Fatal error" and it doesnt change.

Why? and What server am i on?
Title: Re: Thunder/Lightning :: Conversion to PHPSUEXEC
Post by: Jason on June 20, 2006, 10:44:20 PM
Quote from: bitwiz44 on June 20, 2006, 10:36:44 PM
Umm.. i have a few 777's the when I try to change it I get "Fatal error" and it doesnt change.

Why? and What server am i on?

You are on Lightning.

Are you using an ftp client to change your files or cpanel's built-in file manager?  I would highly recommend a standalone ftp client.

Title: Re: Thunder/Lightning :: Conversion to PHPSUEXEC
Post by: bitwiz44 on June 20, 2006, 10:59:53 PM
Quote from: Jason on June 20, 2006, 10:44:20 PM
Quote from: bitwiz44 on June 20, 2006, 10:36:44 PM
Umm.. i have a few 777's the when I try to change it I get "Fatal error" and it doesnt change.

Why? and What server am i on?

You are on Lightning.

Are you using an ftp client to change your files or cpanel's built-in file manager?  I would highly recommend a standalone ftp client.



cpanel. um.. Ok.  :-\
Title: Re: Thunder/Lightning :: Conversion to PHPSUEXEC
Post by: Jason on June 20, 2006, 11:02:46 PM
Quote from: bitwiz44 on June 20, 2006, 10:59:53 PM
cpanel. um.. Ok.  :-\

Cpanel's built-in filemanager is widely viewed as a good option for quick fixes on the fly but isn't a highly reliable option for everyday use.  There are plenty of free options out there if you don't want to purchase a paid solution.

I will be converting everyone with 777 files down to 755 like I did for Thunder so you don't necessarily need to be concerned with that part as much.  I would recommend you check to see if you have any htaccess files in your account.  If so, you may want to make sure they don't have any php commands in them.  If so, clear that out, or rename the htaccess file itself (save a backup for reference).

Cheers,
Jason
Title: Re: Thunder/Lightning :: Conversion to PHPSUEXEC
Post by: Powerbob on June 21, 2006, 07:25:27 AM
Quote from: Jason on June 20, 2006, 08:21:58 AM
Powerbob,

Your forum is up.  Please confirm.

Regards,
Jason

Yes my Forum is back up and working now thanks


Powerbob
Title: Re: Thunder/Lightning :: Conversion to PHPSUEXEC
Post by: bitwiz44 on June 21, 2006, 03:35:15 PM
well.. Thanks. however I changed my attachment directory to 775 and now the members cannot upload an attachment or avatar.

Is this something  once you are done that will go away?
Title: Re: Thunder/Lightning :: Conversion to PHPSUEXEC
Post by: Jason on June 21, 2006, 03:46:33 PM
Quote from: bitwiz44 on June 21, 2006, 03:35:15 PM
well.. Thanks. however I changed my attachment directory to 775 and now the members cannot upload an attachment or avatar.

Is this something  once you are done that will go away?

You mean 755 right?

I'm not 100% but I'm pretty certain that will go away after the upgrade tonight.  I had a few issues with a forum after converting to 755 that vanished once phpsuexec was in place.
Title: Re: Thunder/Lightning :: Conversion to PHPSUEXEC
Post by: bitwiz44 on June 21, 2006, 04:31:24 PM
Quote from: Jason on June 21, 2006, 03:46:33 PM
Quote from: bitwiz44 on June 21, 2006, 03:35:15 PM
well.. Thanks. however I changed my attachment directory to 775 and now the members cannot upload an attachment or avatar.

Is this something  once you are done that will go away?

You mean 755 right?

I'm not 100% but I'm pretty certain that will go away after the upgrade tonight.  I had a few issues with a forum after converting to 755 that vanished once phpsuexec was in place.


opps. I was doing 775.  :-\
Title: Re: Thunder/Lightning :: Conversion to PHPSUEXEC
Post by: Jason on June 21, 2006, 07:47:37 PM
Lightning is being converted now.
Title: Re: Thunder/Lightning :: Conversion to PHPSUEXEC
Post by: Jason on June 21, 2006, 08:06:23 PM
Quote from: Jason on June 21, 2006, 07:47:37 PM
Lightning is being converted now.

Lightning has been converted.  Please check your site(s) for functionality and post here if you have any questions.

All servers have now been migrated.

I will email all clients now and followup with an "explanation post" here later once I'm sure everyone is good to go.
Title: Re: Thunder/Lightning :: Conversion to PHPSUEXEC
Post by: Jason on June 21, 2006, 09:26:42 PM
I've gone through site by site for Lightning and all looks pretty good.  I fixed about 8 forums -- which I must mention -- if you're running old versions of forum software, now is a good time to look into upgrading  (email me if you have questions).  I fixed a few file permissions with some oscommerce installs and that was about it.  I'm trying to fix a part of one site, but everything else that I can see is looking good.

Note:  Don't take my word for it -- please check your own sites for any scripts that aren't clearly linked off of your homepages.

More info to follow...
Title: Re: Thunder/Lightning :: Conversion to PHPSUEXEC
Post by: Jason on June 22, 2006, 10:46:52 PM
I have posted more info on why this implementation occurred so quickly in the client's-only area of our forum:

http://www.charlottezweb.com/forums/index.php?topic=464.0

(If you are a client and can't view the above link, you need to signup for the forum so that I can grant you membership to the private areas)

Thanks,
Jason