How to recalculate AWStats statistics from logs for previous months

Question

How to recalculate AWStats web statistics from domain logs for previous months?

Answer

Download, unpack and execute the script on the server:

# wget https://support.plesk.com/hc/article_attachments/115003493785/rebuild_awstats.tar.gz
# tar -xzvf rebuild_awstats.tar.gz
# ./rebuild_awstats.sh

If no arguments are given, the script will display help message.

Note: for Ubuntu 14.04 as additional step the following symbolic link should be created:

# ln -s /usr/lib/plesk-9.0/logresolvemerge /usr/share/doc/awstats/examples/logresolvemerge.pl

For more information how the script works see below.

Algorithm

1. For each specified domain (or all domains, if the -A option is used), the script will memorise the log file's start date.
2. Then, it will remove AWStats data files for the months covered by the log files and rebuild them with corresponding information.
3. Finally, the script builds static pages with a statistics summary for each month covered by the log files.

Alternatively, if the -F option is used, all AWStats data will be removed and new files will be generated using only the information from the log files.

In order to merge statistics which was created from scratch with AWStats statistics in another folder, the corresponding .txt file and folder should be copied from webstat directory. For example, statistics of example.com domain for December were created from scratch, in the way the following files should be copied from /var/www/system/example.com/statistics/webstat/ :

• awstats122016.example.com-http.txt file
• 2016-12 folder

Note: it is very important to copy .txt file otherwise new statistics (from scratch) can be erased.

Help message

# ./rebuild_awstats.sh

Usage: ./rebuild_awstats.sh [options] [<domains...>]

Options:
    -A
    --all-domains
        Process all domains. If this option is not specified, then list of
        domains to process must be provided.

    -F
    --from-scratch
        Remove contents of webstat/ and webstat-ssl/ directories before
        rebuilding statistics pages (originals are saved with numeric
        suffix). Statistics will be rebuilt from logs only, only for
        period covered by log files. If this parameter is not used, then
        statistics is recalculated beginning on the month on which log files
        start (if log starts in the middle of the month, then statistics
        for first half of the month will not be present.)

    -h
    --help
        This message.

Output example

For each processed domain, the script will report message like:

=== example.com
Saving: webstat* --> webstat*.9
Logs begin on Oct 27 2012
access_log* (new/old/corrupted): 9/0/0
access_ssl_log* (new/old/corrupted): 71/0/0
Rebuilding static pages: 2012-10 2012-11 2012-12 2013-01 2013-02

As you can see, the original directories are saved with a numeric suffix (the suffix is increased if the destination directory already exists). As a result, it is possible to roll-back to a previous version if it will be required.

Additional information

In case script was launched to recalculate statistics from scratch (with -F flag), which removes old statistics that are not stored in log files, you can restore it from the backup that is created by script.

The backup is stored in the directories with names webstat.N and webstat-ssl.N, where N is a number. This folder is specified in the output when the script is launched:

Saving: webstat* --> webstat*.11

Use this article #213408789 if the script fails with the following error:

./rebuild_awstats.sh: line 210: /usr/share/awstats/tools/logresolvemerge.pl: No such file or directory
ERROR: failed to merge access_log*. Skipping domain.
ERROR /usr/share/awstats/wwwroot/cgi-bin/awstats.pl cannot be executed.