Plesk Virtuozzo containers Upgrade Tool

Refers to:

  • Plesk 11.5 for Linux

Created:

2016-11-16 13:17:13 UTC

Modified:

2016-12-21 20:22:12 UTC

0

Was this article helpful?


Have more questions?

Submit a request

Plesk Virtuozzo containers Upgrade Tool

Description

Plesk PVC Containers Upgrade Tool is the utility that allows you to determine whether it would be possible and safe to upgrade containers with Plesk for Linux. The utility performs the upgrade in accordance with the Service Team best practices. The utility is able to:

  • Manage the overall upgrade project stages on a HW Node or a subset of Containers: check, start, monitor, finish.
  • Collect and consolidate all the necessary logs on a Hardware node.
  • Perform the upgrade via Major/Version-free EZ-templates or Autoinstaller in the most secure/recommended way
  • Manage the upgrade using proper commands, handle the pre-upgrade stage correctly.
  • Analyze the source environment before the upgrade to identify possible known upgrade issues
  • Block the upgrade process in case a Plesk database inconsistency is detected during the revertable PREP-stage of the upgarde.
  • Verify template-set against installed components, use the safest EZ-templates set to upgrade.
  • Run several operations at once with per VE locks. (Scheduled upgrade)
  • Choose custom Plesk repository to be used for a container validation/upgrade.

Please feel free to discuss the tool in the special thread on the Plesk Forum .

Navigation

Prerequisites

  • The tool is written in shell and is designed to be executed on a Hardware Node. Wget utility should be installed on the hardware node. The tool should be executed with super-user privileges.
  • The full set of relevant Plesk EZ-templates (Major or Version Free) must be installed on a Hardware Node and should be up-to-date. (see recommended workflow section)
  • Plesk repository (http://autoinstall.plesk.com/ by default) should be available from the Hardware node and Containers.

User Interface

The utility is managed by three main commands: check, upgrade and summary with various additional options:

\# ./plesk\_pvc\_upgrade.sh
Usage: plesk\_pvc\_upgrade.sh <action> [options]
plesk\_pvc\_upgrade.sh help [command]
plesk\_pvc\_upgrade.sh version
plesk\_pvc\_upgrade.sh summary --action=check|upgrade [--ctid=<ctid>] [--logdir=<logdir>] [--groupby=container|issue]
plesk\_pvc\_upgrade.sh check --release=<release> [--ctid=<ctid>] [--source=<source>] [--mailto=<mailto>] [--logdir=<logdir>] [--type=major|versionfree|autoinstaller] [--postcheck-script=<postcheck-script>] [--ignore-warnings]
plesk\_pvc\_upgrade.sh upgrade --release=<release> [--ctid=<ctid>] [--source=<source>] [--mailto=<mailto>] [--logdir=<logdir>] [--type=major|versionfree|autoinstaller] [--postcheck-script=<postcheck-script>] [--postupgrade-script=<postupgrade-script>] [--ignore-warnings]

For more details see help for each action:

\# ./plesk\_pvc\_upgrade.sh help <action>

Recommended workflow


Install/update the full set of Plesk EZ templates.

All available Plesk EZ templates are located as rpm packages on the ftp repository ftp://download1.swsoft.com/Plesk/PP11/11.5/
For example ftp://download1.swsoft.com/Plesk/PP11/11.5/CentOS6/vztemplates/
Just download and unpack the pp11.5 for major versions or a pp archive for versionfree EZ templates and install them with the "vzpkg update template" command.

Also you can obtain/update templates via the vzup2date utility.


Analyze selected containers to get the list of possible pre-upgrade issues

Run the utility with the check command:

\# ./plesk\_pvc\_upgrade.sh check --release=PLESK\_11\_5\_30 --ctid=504 513 514

The Utility checks containers one by one and briefly shows performed actions in the output. All necessary information will be also available in the utility’s summary reports (see below). Get the summary report grouped by containers or issues:

\# ./plesk\_pvc\_upgrade.sh summary --action=check --ctid=504 513 514 --groupby=container

The Utility shows all detected issues and the overall result for each container:

Container '504' result:
[RESULT]: SKIP: Plesk version PLESK\_11\_5\_30 or later is already installed.

Container '513' result:
[RESULT]: SUCCESS: No issues were detected.

Container '514' result:
[WARNING]: Unable to get data from KA server
[WARNING]: Unable to find "main" IP address in psa database. Please, check /en/112417 for more details.
[WARNING]: Your autoinstaller version 3.12.1 is outdated. Please refer to article /en/112166 on how to obtain latest version of autoinstaller.
[RESULT]: WARNING: A number of issues will be encountered during upgrade.

All issues are divided into several levels:
[SUCCESS] - Means that known issues were not found and the Container is ready for the upgrade.
[SKIP] - Means that the Container will not be upgraded (for example because the container is stopped or Plesk is not installed or it already has all necessary Plesk version components installed).
[WARNING] - Issues that might affect the upgrade procedure or post-upgrade functionality. We strongly recommend to review and eliminate all such issues if possible. There is a set of warnings that does not require to be resolved before the upgrade (like business logic related issues)
[CRITICAL] - Issues that potentially can cause dramatic damage. The Utility doesn’t allow the upgrade for Containers with such issues.
[FAILED] - Means that the utility fails to perform some action or some prerequisites are not met.

Since some issues appear on several containers it’s possible to show the list of the affected containers for each issue with the help of the option --groupby=issue:

\# ./plesk\_pvc\_upgrade.sh summary --action=check --ctid=504 513 514 --groupby=issue
[CRITICAL]: Checking for a password in my.cnf files
ctid:510 executed\_in:17:05:38 hostname:ct510

[WARNING]: Checking autoinstaller version (/en/112166)
ctid:515 executed\_in:12:40:54 hostname:ct515.tld
ctid:516 executed\_in:12:42:03 hostname:ct516.tld
ctid:510 executed\_in:17:05:38 hostname:ct510.tld

[WARNING]: Checking for a Plesk mail drivers conflict (/120284)
ctid:515 executed\_in:12:40:54 hostname:ct515.tld
ctid:516 executed\_in:12:42:03 hostname:ct516.tld

[WARNING]: Checking for a Plesk mail drivers conflict with sendmail (/118841)
ctid:515 executed\_in:12:40:54 hostname: ct515.tld

Deal with all reported issues for containers selected for upgrade:

For example apply resolution from the provided KB 118841 on container 515:

\# vzctl exec 515 rpm -e sendmail-cf sendmail-doc sendmail

Please make sure that all issues are eliminated.

Perform the upgrade for the list of prepared containers and check results:

Now it is time to start the upgrade process. The command to upgrade containers using Plesk Major EZ templates is:

\# ./plesk\_pvc\_upgrade.sh upgrade --release=PLESK\_11\_5\_30 --ctid=501 502 503 504 505 506 507 508 509 510 --mailto=admin@hw-node.tld --type=major

The command to upgrade containers using Plesk Version Free EZ templates is:

\# ./plesk\_pvc\_upgrade.sh upgrade --release=PLESK\_11\_5\_30 --ctid=501 502 503 504 505 506 507 508 509 510 --mailto=admin@hw-node.tld --type=versionfree

The command to upgrade containers using Plesk Installer:

\# ./plesk\_pvc\_upgrade.sh upgrade --release=PLESK\_11\_5\_30 --ctid=501 502 503 504 505 506 507 508 509 510 --mailto=admin@hw-node.tld --type=autoinstaller

Option --type can be skipped. In this case the utility will try to identify and keep the type from the source container:

\# ./plesk\_pvc\_upgrade.sh upgrade --release=PLESK\_11\_5\_30 --ctid=501 502 503 504 505 506 507 508 509 510 --mailto=admin@hw-node.tld

By default the utility does not allow upgrade for containers that have some warnings at the "check" stage. But there is a set of warnings that does not require to be resolved prior to the upgrade (like business logic related issues). To proceed with the upgrade despite the warnings there is an option --ignore-warnings :

\# ./plesk\_pvc\_upgrade.sh upgrade --release=PLESK\_11\_5\_30 --ctid=501 502 503 504 505 506 507 508 509 510 --ignore-warnings --mailto=admin@hw-node.tld

( --mailto=<email> option will send email notifications with overall operation result for each Container)

The utility upgrades containers one by one and briefly shows performed actions in the output. All necessary information is available in utility’s summary reports similar to reports from the "check" stage. The following commands show reports for all containers that have been processed already:

\# ./plesk\_pvc\_upgrade.sh summary --action=upgrade --groupby=container
# ./plesk\_pvc\_upgrade.sh summary --action=upgrade --groupby=issue

The detailed utility log with all involved actions is located here:
/var/log/plesk\_pvc\_upgrade/last/<CTID>/ct\_upgrade.log.

Also several logs from containers are stored in /var/log/plesk\_pvc\_upgrade/last/<CTID>/ directory.

Using custom actions

The utility allows to specify custom scripts that will be executed after the " check " or the " upgrade " operation for each container with options --postcheck-script=<full path> and --postupgrade-script=<full path>

\# ./plesk\_pvc\_upgrade.sh check --release=PLESK\_11\_5\_30 --ctid=504 513 514 --postcheck-script=/root/postcheck.sh

or

\# ./plesk\_pvc\_upgrade.sh upgrade --release=PLESK\_11\_5\_30 --ctid=504 513 514 --postcheck-script=/root/postcheck.sh --postupgrade-script=/root/postupgrade.sh

The following parameters are passed to the custom scripts as environment variables:
PP\_RELEASE =PLESK\_11\_5\_30 - release used for check or upgrade
PP\_LOGDIR =/var/log/plesk\_pvc\_upgrade/507/2014-03-03\_211405 - logs directory.
PP\_LOGFILE =/var/log/plesk\_pvc\_upgrade/507/2014-03-03\_211405/ct\_check.log - main log file.
PP\_SOURCE\_URL =http://autoinstall.plesk.com - upgrade source.
PP\_ACTION =check - type of action (check/upgrade).
PP\_ACTION\_STATUS =1 - result of the action.
PP\_STATUS\_OK =0 - exit code for SUCCESS result.
PP\_STATUS\_WARNING =1 - exit code for WARNING result.
PP\_STATUS\_CRITICAL =2 - exit code for CRITICAL result.
PP\_STATUS\_FAIL =3 - exit code for FAIL result.
PP\_STATUS\_SKIP =4 - exit code for SKIP result.
PP\_MAILTO =admin@node.tld - email to send report to.
PP\_CTID =504 - container ID.

The following example creates container’s "clone" before the upgrade and reverts the container to the initial state if the upgrade finishes with FAILED or CRITICAL status. Also these scripts execute "bootstrapper repair" if the upgrade finishes with WARNING.

\# ./plesk\_pvc\_upgrade.sh upgrade --release=PLESK\_11\_5\_30 --ignore-warnings --ctid=514 517 --postcheck-script=/root/postcheck.sh --postupgrade-script=/root/postupgrade.sh

/root/postcheck.sh

set -x
vzmlocal -C $PP\_CTID:$(( 1000000 + $PP\_CTID ))
rc=$?
#add timeout to let container to start
sleep 60
if [[ $rc != 0 ]] ; then
exit $PP\_STATUS\_CRITICAL
fi

/root/postupgrade.sh

set -x
if [ $PP\_ACTION\_STATUS = $PP\_STATUS\_CRITICAL -o $PP\_ACTION\_STATUS = $PP\_STATUS\_FAIL ] ; then
vzlist $(( 1000000 + $PP\_CTID ))
rc=$?
if [[ $rc != 0 ]] ; then
echo "[CRITICAL] Container's clone is not found. Rollback is not possible"
exit $PP\_STATUS\_CRITICAL
fi
vzctl stop $PP\_CTID
vzctl umount $PP\_CTID
vzctl destroy $PP\_CTID
vzmlocal -C $(( 1000000 + $PP\_CTID )):$PP\_CTID
rc=$?
if [[ $rc != 0 ]] ; then
echo "[FAILED] Rolling back container CTID:$PP\_CTID from clone CTID:$(( 1000000 + $PP\_CTID )) is failed"
exit $PP\_STATUS\_FAILED
fi
echo "Container was rolled back successfully"
vzctl start $PP\_CTID
fi
if [ $PP\_ACTION\_STATUS = $PP\_STATUS\_WARNING ] ; then
echo "Trying to execute bootstrapper repair"
vzctl exec $PP\_CTID "/usr/local/psa/bootstrapper/pp11.5.30-bootstrapper/bootstrapper.sh repair"
rc=$?
if [[ $rc != 0 ]] ; then
echo "[WARNING] Bootstrapper repair was finished woth non zero exit code: $rc"
fi
echo "Bootstrapper repair finished successfully"
fi
echo "Trying to remove container's clone with CTID:$(( 1000000 + $PP\_CTID ))"
vzctl stop $(( 1000000 + $PP\_CTID ))
vzctl umount $(( 1000000 + $PP\_CTID ))
vzctl destroy $(( 1000000 + $PP\_CTID ))
rc=$?
if [[ $rc = 0 ]] ; then
echo "Container's clone with CTID:$(( 1000000 + $PP\_CTID )) have been removed"
exit $PP\_STATUS\_SUCCESS
else
echo "[WARNING] Removing container's clone with CTID:$(( 1000000 + $PP\_CTID )) is failed"
exit $PP\_STATUS\_WARNING
fi

The custom script in the example below fetches all domains and stores their name and status in Plesk and http code (specifically, in files domains\_status\_pre and domains\_status\_post in the container’s logs directory before and after the upgrade). The scripts compares the files and in case changes are detected/files are not identical, it reports the WARNING status for the custom post upgrade action:

\# ./plesk\_pvc\_upgrade.sh upgrade --release=PLESK\_11\_5\_30 --ctid=520 --postcheck-script="/root/check\_domains.sh" --postupgrade-script="/root/check\_domains.sh" --ignore-warnings

Part of the container upgrade log:

[18:42:54]\[post\_script]\[OUTPUT]:
[18:42:54]\[post\_script]\[OUTPUT]: Executing the following command: /root/check\_domains.sh
[18:43:09]\[post\_script]\[OUTPUT]: 2c2
[18:43:09]\[post\_script]\[OUTPUT]: < a10-52-240-204.qa.plesk.ru 0 200
[18:43:09]\[post\_script]\[OUTPUT]: ---
[18:43:09]\[post\_script]\[OUTPUT]: > a10-52-240-204.qa.plesk.ru 0 00
[18:43:09]\[post\_script]\[OUTPUT]: [WARNING] domains statuses before and after upgrade are different. For more details see # diff /var/log/plesk\_pvc\_upgrade/520/2014-03-06\_183617/domains\_status\_pre /var/log/plesk\_pvc\_upgrade/520/2014-03-06\_183617/domains\_status\_post
[18:43:09]\[post\_script]\[OUTPUT]: Finished with errors
[18:43:09]\[post\_upgrade\_script]\[WARNING]: Running the post-upgrade command specified by the user. DESC: Custom post-upgrade command has been finished, however several errors occurred.

/root/check\_domains.sh

if [ $PP\_ACTION = "check" ]; then
vzctl exec $PP\_CTID 'MYSQL_PWD= cat /etc/psa/.psa.shadow mysql -u admin -BNr psa -e "select name, status from domains where htype in (\\"vrt\_hst\\",\\"frm\_fwd\\",\\"std\_fwd\\") order by name"' > $PP\_LOGDIR/domains\_list
echo "domain status HTTP\_CODE" > $PP\_LOGDIR/domains\_status\_pre
for i in \ cat $PP\_LOGDIR/domains\_list | awk '\{print $1\}'\ ; do
grep -E "^$i[[:space:]]\ " $PP\_LOGDIR/domains\_list |xargs echo -n >> $PP\_LOGDIR/domains\_status\_pre
curl -sL --connect-timeout 3 -w " %\{http\_code\}\\\\" -o /dev/null "http://$i" >> $PP\_LOGDIR/domains\_status\_pre
done
fi
if [ $PP\_ACTION = "upgrade" ]; then
echo "domain status HTTP\_CODE" > $PP\_LOGDIR/domains\_status\_post
for i in \ cat $PP\_LOGDIR/domains\_list | awk '\{print $1\}'\ ; do
grep -E "^$i[[:space:]]\
" $PP\_LOGDIR/domains\_list |xargs echo -n >> $PP\_LOGDIR/domains\_status\_post
curl -sL --connect-timeout 3 -w " %\{http\_code\}\\\\" -o /dev/null "http://$i" >> $PP\_LOGDIR/domains\_status\_post
done
diff $PP\_LOGDIR/domains\_status\_pre $PP\_LOGDIR/domains\_status\_post
rc=$?
if [ $rc -ne 0 ] ; then
echo "[WARNING] domains statuses before and after upgrade are different. For more details see # diff $PP\_LOGDIR/domains\_status\_pre $PP\_LOGDIR/domains\_status\_post"
exit $PP\_STATUS\_WARNING
fi
fi

Useful tips

To get summary results for a limited number of containers use option --ctid with the "summary" command

\# ./plesk\_pvc\_upgrade.sh summary --action=upgrade --ctid=515 520

To get the list of containers affected by the selected issues in "grouped by issues" report:

\# ./plesk\_pvc\_upgrade.sh summary --action=upgrade --groupby=issue | awk -v var='<REGEX1>|<REGEX2>' '$0 ~ ".\*"var, /^$/'

\# ./plesk\_pvc\_upgrade.sh summary --action=upgrade --groupby=issue | awk -v var='FAILED|CRITICAL' '$0 ~ ".\*"var, /^$/'
[FAILED]: vzpkg execution has been failed
ctid:514 executed\_in:00:38:04 hostname:ct514.tld
ctid:517 executed\_in:00:43:26 hostname:ct517.tld
[CRITICAL]: Checking for a password in my.cnf files
ctid:510 executed\_in:18:03:00 hostname:ct510.tld
[CRITICAL]: Post upgrade procedure.
ctid:514 executed\_in:00:38:04 hostname:ct514.tld
ctid:517 executed\_in:00:43:27 hostname:ct517.tld

To filter out irrelevant issues in the "grouped by issues" report:

\# ./plesk\_pvc\_upgrade.sh summary --action=check --ctid=503 519 --groupby=issue | awk -v var='<REGEX1>|<REGEX2>' '\{if($0~".\*"var)flag=1;if(!flag)print;if($0~/^$/)flag=0\}'

\# ./plesk\_pvc\_upgrade.sh summary --action=check --ctid=503 519 --groupby=issue | awk -v var='Unable to get data from KA server|Checking autoinstaller version' '\{if($0~".\*"var)flag=1;if(!flag)print;if($0~/^$/)flag=0\}'
[WARNING]: Checking for main IP address (/en/112417)
ctid:519 executed\_in:11:12:45 hostname:ct519.tld
[SKIP]: The container is not running.
ctid:503 executed\_in:17:00:46 hostname:ct503.tld

To have only the newly installed template registered in containers

In order to have only the newly installed template registered in containers after the upgrade, please make sure the already installed templates are removed from the container's config (/etc/vz/conf/<CTID>.conf) prior to the upgrade . (plesk\ , pp\ ):
TEMPLATES=" .postgresql .mysql .devel .plesk9.5-horde .plesk9.5-qmail .plesk9.5 .plesk9.5-postfix .pp11-migration .pp11 .pp11-webmail .pp11-postfix"
>
TEMPLATES=" .postgresql .mysql .devel "

Once done, start the upgrade procedure with the help of the corresponding utility indicating the type of the templates you want to be installed (otherwise, the utility would launch the autoinstaller):

\# ./plesk\_pvc\_upgrade.sh upgrade --release=PLESK\_11\_5\_30 --ignore-warnings --ctid=520 --type=major

Only new Plesk templates would remain after the upgrade:
TEMPLATES=" .postgresql .mysql .devel .pp11.5-migration .pp11.5 .pp11.5-webhosting-addons .pp11.5-bind .pp11.5-webmail .pp11.5-postfix"

Known Issues

  • If there is no connection to the Parallels KA server from the container, the key checking action takes time (~1min for every container).
  • We do not recommend executing several utility instances at once for the same containers as it can affect summary reports generation.

The logs directory location can be changed by the --logdir option:
\# ./plesk\_pvc\_upgrade.sh check --release=PLESK\_11\_5\_30 --ctid=504 513 514 --logdir=/var/log/custom\_dir

Feedback

Please feel free to discuss the tool in the special thread on the Plesk Forum .

Have more questions? Submit a request
Please sign in to leave a comment.