Articles in this section

See more

Plesk websites return Apache error 403 on May 26, 2023: client denied by server configuration

Avatar
Alexander Tsmokalyuk
Updated
Follow
Plesk for Linux kb: technical

Symptoms

Starting from May 26, 2023 Plesk domains are not accessible in a browser with either the 403 Forbidden error or with the 123 Apache Test Page.

Cause

With the update of the aum package provided by Atomicorp, the Apache module mod_evasive was installed, and this module false-positively blacklisted the server's IP address - errors like below are found in the log file /var/log/messages or in the Apache log:

CONFIG_TEXT: mod_evasive[177717]: Blacklisting address 192.0.2.2: possible DoS attack

The issue is resolved in the further update of the aum package 6.0.48-29386.

Resolution

  1. Connect to the server using SSH.

  2. Update aum to the latest version and remove the Apache module mod_evasive:

    # aum -uf ; yum -y remove mod_evasive ; aum -u

  3. Go to Tools & Settings > Web Application Firewall > Settings, switch to other ruleset, press Apply, switch back to initial ruleset and press Apply.

As a workaround, apply one of the following:

Manually whitelist the server's IP in mod_evasive

  1. Connect to the server using SSH.

  2. Open the configuration file /etc/httpd/conf.d/mod_evasive.conf for editing. For example, use the vi text editor:

    # vi /etc/httpd/conf.d/mod_evasive.conf

  3. Scroll to the bottom of the file and uncomment the DOSWhitelist directives, for example:

  • Before:

    #DOSWhitelist 127.0.0.1
    #DOSWhitelist 192.0.2.2

  • After:

    DOSWhitelist 127.0.0.1
    DOSWhitelist 192.0.2.2

    Note: Replace the IP address 192.0.2.2 with the exact IP address of the server that is present in the error message from /var/log/messages.

  1. Save changes and close the file.

  2. Restart Apache:

    # service httpd restart

To completely disable mod_evasive:

  1. Connect to the server using SSH.

  2. Execute the following command to disable mod_evasive:

    # sed -i -e '/LoadModule evasive20_module/s/^/#/g' /etc/httpd/conf.d/mod_evasive.conf

  3. Restart Apache:

    # service httpd restart

When the issue with the aum package is fixed on the Atomicorp side, to re-enable the module execute the below command

# sed -i '/#LoadModule evasive20_module/s/^#//g' /etc/httpd/conf.d/mod_evasive.conf

And restart Apache to apply changes:

# service httpd restart

Comments

10 comments

Sort by Date Votes
  • Avatar
    Roman Krakov

    Server's external IP also have to be whitelisted (DOSWhitelist xxx.xxx.xxx.xxx) to resolve the issue.

    1
    Comment actions Permalink
  • Avatar
    Martin Gojowsky (Edited )

    Yes exactly. We also struggled with this until we saw in the httpd service status message that our external IP has been banned by mod_evasive.

     

    Edit: a hint has been added to article now, thank you.

    0
    Comment actions Permalink
  • Avatar
    Stephan Mol (Edited )

    Same here, spent almost two hours trying to find out what was going on. Fortunately this article now exists, great job on posting it so quickly! Although too late for us others will find this very useful. Do not forget to also add the servers IPv6 address if applicable. The external/resolvable servers IP address(es).

    No one changed at thing and suddenly this morning customers were facing intermittent downtime. Via mentioned line coming by colored in red (is why it stood out) on a journalctl tail I ended up reading about mod_evasive at https://www.howtogeek.com/devops/how-to-configure-mod_evasive-for-apache-ddos-protection/ and trying to fix it by adding the servers IP addresses to this DOSWhitelist, and that solved the incident.

    I purposely have auto updates of both Plesk and OS packages disabled, to have the control myself on a set interval and try to prevent surprises like this from happening. So find this out about the automatic installation of these OS package was quite irritating causing this incident:

    ...

    May 24 03:40:36 Updated: 1:aum-6.0.48-29078.el7.art.x86_64
    May 25 03:33:42 Updated: 1:aum-6.0.48-29360.el7.art.x86_64
    May 26 03:55:27 Updated: 1:aum-6.0.48-29375.el7.art.x86_64
    May 26 03:56:12 Installed: mod_evasive-1.10.1-22.el7.x86_64
    May 26 03:56:15 Installed: mod_qos-11.70-1.el7.x86_64

    ...

    Apperently the WAF is resposible for that.

    0
    Comment actions Permalink
  • Avatar
    Kimon Nouskalis

    We have the same problem. I have disable the Fail2Ban and atomicorp WAF but the problem still exists.

    I have whitelist the IP of the server restart the httpd service and waiting forward to see that is OK.

    0
    Comment actions Permalink
  • Avatar
    Brandon Ingham

    This bug is horrible. It's made all of my websites display 403 errors if reloaded too quickly, and using the above instructions made my Apache unable to start.

    Please can you just do the normal thing and release an update that fixes all of these WAF issues? I don't want to be bombarded with security rules that I don't even know are there, and don't work by default.

    0
    Comment actions Permalink
  • Avatar
    Justin Henderson

    Same as Brandon Ingham. The current "fix" makes it so Apache will not start. My temporary workaround after applying was to disable ModSecurity altogether...

    0
    Comment actions Permalink
  • Avatar
    Taras Ermoshin

    Hello @Brandon Ingham and @Justin Henderson

    The article was updated with additional resolution step, step 3, please check it.

    You can also check another article about a similar recent issue related to Atomic rule set: https://support.plesk.com/hc/en-us/articles/14801826549527.

    If both solutions don't help, consider contacting support.

    0
    Comment actions Permalink
  • Avatar
    Martin Gojowsky

    Would it be better to remove mod_evasive or to stay with the whitelisting suggested as workaround before? Why do think now it´s better to remove instead of properly configuring it?

    0
    Comment actions Permalink
  • Avatar
    Davide Manzi

    Hi, updating aum gives to me version 6.0.48-29375 instead (on Centos 7  7.9.2009, Plesk Obsidian 18.0.5.2 #3).

    After following this article, I restored original /etc/https/conf.d/mod_evasive.conf that had the "LoadModule evasive20module" commented due to the problem occurred last week on mod_evasive, but restarting apache it does not start, so I had to comment the LoadModule again.

    Detailed error is:

    Cannot load modules/mod_evasive24.so into server: /etc/httpd/modules/mod_evasive24.so: cannot open shared object file: No such file or directory

    The module mod_evasive24.so is no more present on the server after the update.

    Could you please help?

    0
    Comment actions Permalink
  • Avatar
    Peter van der Kleij

    This also solved our problem.
    On 1 specific website (as far as we know) IP addresses were blocked due to errors that arose on the site in the 'website'\logs\error_log file.

    Errors appeared on all kinds of files, the error messages looked like this. (ip addresses and site names are fictitious).

    • [Wed May 31 21:55:22.218085 2023] [:error] [pid 12996:tid 139638714857216] [client 123.123.123.123:58850] client denied by server configuration: /var/www/vhosts/123.nl/httpdocs/ wp-content/plugins/woocommerce/assets/js/frontend/add-to-cart.min.js, referer: https://123.nl/

    in plesk, the tool fail2ban, i removed all registrations from the jail 'plesk-apache'. (This was super slow by the way. Have removed ip addresses in batches of 10-20 pieces, each batch took at least 5 minutes.)

    In the 'website'\logs\error_log file. no more registrations came, so problem seems solved.

    0
    Comment actions Permalink

Please sign in to leave a comment.

Have more questions? Submit a request

Related articles