Is there a way to anonymize IP addresses in logfiles before log rotate?

Question

Is there a way to anonymize IP addresses in log files before log rotate?

Answer

Currently, this feature is yet to be implemented in Plesk. You can anonymize logs only on log rotation. For this, please refer to the following article: How to anonymize IP addresses in log files in Plesk?

There is a Feature Request through our User voice channel, you can take a look at this new request by referring to the following link: User Voice Feature Request
Remember, the top-ranked suggestions are likely to be included in the next versions of Plesk.

As workaround, you can create a shell script to anonymize these logs.

PLESK_WARN: This script is not officially tested by Plesk and should be used at your own risk because these addresses are used in security tools like fail2ban and modsecurity. Even running it on a crontab, it is still possible to affect the performance of these tools.

To use it, create a shell script as below in /usr/local/bin/anonymize-logs2:

CONFIG_TEXT: #!/bin/bash
# (c) Matt Bagley, under the GPL2
# given log file(s), it will anonymize the logs and update (only if needed)
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin

if [ "$1" == "" ] || [ "$1" == "-h" ] ; then
   echo "Usage: $0 logfile.log [logfile2.log] [logfile3.log] ..."
   exit 1
fi

td="$(mktemp -d)"
temp_log=$td/file
temp_log2=${temp_log}-pass2
temp_log3=${temp_log}-pass3

clean_up() {
   rm -f $temp_log $temp_log2 $temp_log3
   rmdir $td
   exit
}
trap "clean_up" 1 2 3 4 5 15

for each in $@ ; do
   #echo Looking at $each
   # does it exist?
   if ! [ -f $each ] ; then
      echo "File not found: $each"
      continue
   fi
   # non-zero?
   if ! [ -s $each ] ; then
     continue
   fi
   # compressed or not?
   compressed=0
   if [ -n "$(echo $each | grep '.gz$')" ] ; then
     compressed=1
   fi
   # expand log
   if [ $compressed -eq 1 ] ; then
     zcat $each > $temp_log
   else
     cat $each > $temp_log
   fi

   # make sure that none of the lines start with '-'. ipv6loganon does not like this
   # and that no lines have "::: " in them
   cat $temp_log | sed 's/^- /0.0.0.0 /g' | sed 's/:::* /:: /g' > $temp_log2

   # anonymize it (ipv6loganon only does files that have IP address first)
   if [ -n "$(head -n 10 $temp_log2 | awk '{print $1}' | egrep '(\.|:)')" ] \
   && [ -z "$(head -n 10 $temp_log2 | awk '{print $1}' | sed 's/[a-fA-F0-9\.:]*//g')" ] ; then
   # echo Running ipv6loganon on $each
     cat $temp_log2 | ipv6loganon --anonymize-careful > $temp_log3
     cat $temp_log3 > $temp_log2
     rm -f $temp_log3
   fi
   cat $temp_log2 | sed 's/\([0-9]*\.[0-9]*\.[0-9]*\)\.[0-9]*/\1.0/g' \
                               | sed 's/\([0-9a-fA-F]*:[0-9a-fA-F]*:[0-9a-fA-F]*:\)[0-9a-fA-F:]*/\1:/g' \
                               | sed 's/:::*/::/g' > $temp_log3
   cat $temp_log3 > $temp_log2
   rm -f $temp_log3

   # verify that it's not empty
   if ! [ -s $temp_log2 ] ; then
      echo "$each was processed as empty"
   continue
   fi
   # diff to see if we changed anything
   if [ -n "$(diff -q $temp_log $temp_log2)" ] ; then
     # if we did, zip and copy file back
     temp_log_ext=""
     if [ $compressed -eq 1 ] ; then
       gzip $temp_log2
       temp_log_ext=.gz
     fi
     mv $each ${each}-old
     echo Replacing $each
     cat ${temp_log2}${temp_log_ext} > $each
     # set the time to the same as the previous file
     touch --reference=${each}-old $each
     # clean up
     rm -f ${each}-old ${temp_log2}${temp_log_ext}
   fi
   rm $temp_log $temp_log2 $temp_log3 -f
done
clean_up

This script uses the ipv6loganon, then you need to install the ipv6calc:

• To Debian/Ubuntu:

  # apt install ipv6calc

• To CentOS/Almalinux:

  # yum install ipv6calc

Then add a weekly cron job for it in /etc/cron.weekly/anonymize-logs:

CONFIG_TEXT: #!/bin/bash

/usr/local/bin/anonymize-logs2 /var/log/httpd/*.gz /var/log/nginx/*.gz /var/log/maillog*.gz

The nice thing about this is that it also anonymizes the log even if it’s been archived to a .gz file.