How to recalculate AWStats web-statistics using domain logs for previous months in Plesk

Question

How to recalculate AWStats web-statistics using domain logs for previous months?

Answer

There are two techniques to recalculate/regenerate AWStats statistic files:

1. From AWStats txt files, which are still within the AWStat directory:

   Instructions to check

   1. Connect to the server via SSH.

   2. Check the directory:

      # ls -lahn /var/www/vhosts/system/example.com/statistics/webstat-ssl/ | grep awstats | awk '{print $9}'
      awstats082018.example.com-https.txt
      awstats092018.example.com-https.txt
      awstats102018.example.com-https.txt
      awstats112018.example.com-https.txt
      awstats122018.example.com-https.txt

   Note: In this case the months 08(August), 09(Semptember), 10(November), 11(October), 12(December) can be retrieved for the year 2018.

2. From the old logs, which are present on the server.

Note: In both, old logs and AWStats txt file, are deleted, the recalculation will not be possible.

Installation

How script works

• General Functionality

  1. For each specified domain (or all domains, if the -A option is used), the script will memorize the log file's start date.

  2. Then, it will remove AWStats data files for the months covered by the log files and rebuild them with corresponding information.

  3. Finally, the script will build static pages with statistics summary for each month covered by the log files.

• -F
  If the -F option is used, all AWStats data will be removed and new files will be generated using only the information stored in the log files.

• -R
  if the -R option is used, the script will gather all data of the mentioned domain and will generate all months, where the AWStats txt data is existing.

• Merging Web-Statistics

  In order to merge web-statistics which was created from scratch with AWStats statistics from another directory, the corresponding .txt file should be copied from the webstat directory. For example, statistics of a domain for December were created from scratch. In this case, the following files should be copied from /var/www/vhosts/system/example.com/statistics/webstat/:

  • A .txt file dated by December 2018: awstats122018.example.com-http.txt

  At the end it required to run the script with -R to build the HMTL files:

  # ./rebuild_awstats.sh -R example.com

Output example

For every processed domain, the script will report this message:

CONFIG_TEXT: === example.com
Saving: webstat* --> webstat*.3
Logs begin on Apr 18 2018
access_log* (new/old/corrupted): 9/0/0
access_ssl_log* (new/old/corrupted): 71/0/0
Rebuilding static pages: 2018-04 2018-05 2018-06 2018-07

Restoring from a backup

The message "_Saving: webstat* --> webstat*.3 _" in the output above shows that the original directories are saved with a numeric suffix (the suffix is increased if the destination directory already exists). This makes it possible to roll-back to old AWStats statistics if needed.

In case the script was launched to recalculate statistics from scratch (with -F flag), which removes old web-statistics that is not stored in log files, it is possible to restore it from the backup that was created by the script.

Troubleshooting

If the script fails with one of the following error messages, visit this KB article for troubleshooting:

CONFIG_TEXT: ./rebuild_awstats.sh: line 210: /usr/share/awstats/tools/logresolvemerge.pl: No such file or directory
ERROR: failed to merge access_log*. Skipping domain.

OR

CONFIG_TEXT: ERROR /usr/share/awstats/wwwroot/cgi-bin/awstats.pl cannot be executed.

OR

CONFIG_TEXT: ERROR /usr/lib/cgi-bin/awstats/awstats.pl cannot be executed