Articles in this section

How to regenerate AWStats web-statistics using domain logs for previous months in Plesk

kb: how-to Plesk for Linux

Applicable to:

  • Plesk for Linux

Question

How to regenerate AWStats web-statistics using domain logs for previous months?

Answer

Warning: Before using the script to regenerate statistics, it is strongly advised to create a backup of the subscriptions containing the domains the script will be used for.
If both Apache and nginx are enabled and the script is executed, logs processed by this script will not be sorted by timestamp, which can lead to incorrect values in resulting AWStats statistics.

There are two techniques to regenerate AWStats statistic files:

  • From AWStats formatted txt files, which are still within the AWStat directory:

    Instructions to check:
    1. Connect to the server via SSH.
    2. Check the directory:

      # ls -lahn /var/www/vhosts/system/example.com/statistics/webstat-ssl/ | grep awstats | awk '{print $9}'
      awstats082018.example.com-https.txt
      awstats092018.example.com-https.txt
      awstats102018.example.com-https.txt
      awstats112018.example.com-https.txt
      awstats122018.example.com-https.txt

    Note: In this case the months 08(August), 09(Semptember), 10(November), 11(October), 12(December) can be retrieved for the year 2018.

  • Or from the existing/old web access logs, which should be present on the server.

Note: If both, old logs and AWStats txt files are deleted, the recalculation is not be possible. However, if these files can be restored from available backups then recalculation is possible.

Execution

How to use the tool
  1. Connect to a Plesk server via SSH
  2. Download the script:

    # wget https://raw.githubusercontent.com/plesk/kb-scripts/master/rebuild-awstats/rebuild-awstats.sh

  3. Grant execute permissions for the script:

    # chmod 755 rebuild-awstats.sh

  4. Run this command to show help message:

    # ./rebuild-awstats.sh
    Rebuild AWstats static pages from available log files.

    Usage: ./rebuild-awstats.sh [options] [<domains...>]

    Options:
    -A
    --all-domains
    Process all domains. If this option is not specified, then list of
    domains to process must be provided.

    -F
    --from-scratch
    Remove contents of webstat/ and webstat-ssl/ directories before
    rebuilding statistics pages (originals are saved with numeric
    suffix). Statistics will be rebuilt from logs only, only for
    period covered by log files. If this parameter is not used, then
    statistics is recalculated beginning on the month on which log files
    start (if log starts in the middle of the month, then statistics
    for first half of the month will not be present.)

    -R
    --rebuild
    Rebuilds the HTML files for AWStats.

    -h
    --help
    This message.

    Look for additional details here: <https://kb.plesk.com/en/115476>

Click on a section to expand

How script works
  • General Functionality
    1. For each specified domain (or all domains, if the -A option is used), the script will memorize the log file's start date.
    2. Then, it will remove AWStats data files for the months covered by the log files and rebuild them with corresponding information.
    3. Finally, the script will build static pages with statistics summary for each month covered by the log files.
  • -F
    If the -F option is used, all AWStats data will be removed and new files will be generated using only the information stored in the log files.
  • -R
    if the -R option is used, the script will gather all data of the mentioned domain and will generate all months, where the AWStats txt data is existing.
  • Merging Web-Statistics

    In order to merge web-statistics which was created from scratch with AWStats statistics from another directory, the corresponding .txt file should be copied from the webstat-ssl directory. For example, statistics of a domain for December were created from scratch. In this case, the following files should be copied from /var/www/vhosts/system/example.com/statistics/webstat-ssl:

    • A .txt file dated by December 2018: awstats122018.example.com-https.txt

    Then finish copying this file to the required domain location. to be merged. At the end it is required to run the script with -R to build the HMTL files:

    # ./rebuild_awstats.sh -R example.com

Click on a section to expand

Output example

For every processed domain, the script will report this message:

CONFIG_TEXT: === example.com
Saving: webstat* --> webstat*.1
Logs begin on: Jun 15 2022
access_log* (new/old/corrupted): 293/0/0
access_ssl_log* (new/old/corrupted): 1651/0/254
Rebuilding static pages:
2022-06 2022-07 2022-08 2022-09 2022-10 2022-11 Successfully rebuild all html files.

The message Saving: webstat* --> webstat*.1 in the output above shows that the original directories are saved with a numeric suffix (the suffix is increased if the destination directory already exists). This makes it possible to roll-back to old AWStats statistics if needed.

In case the script was launched to regenerate statistics from scratch (with -F flag), which removes old web-statistics that is not stored in log files, it is possible to restore it from the backup that was created by the script.

After recreation is complete, recalculate the statistic using one of the articles:
How to recalculate statistics for all domains in Plesk?
How to recalculate statistics for a particular domain in Plesk?

Was this article helpful?

Comments

0 comments

Please sign in to leave a comment.