Applicable to:
- Plesk for Linux
Question
How to block custom bots on Plesk server?
Answer
It is possible to add the required bots into Fail2ban "apache-badbot" jail:
- Log in the server via SSH
- Create backup of apache-badbots.conf file:
# cp -p /etc/fail2ban/filter.d/apache-badbots.conf /root/apache-badbots.conf_back
-
Edit /etc/fail2ban/filter.d/apache-badbots.conf file and add the required bot.
In the example below ClaudeBot is added:
Fail2ban will start blocking access from both Nginx and Apache. The settings can be checked at Tools & Settings > IP Address Banning (Fail2Ban) > Jails > plesk-apache-badbot:
CONFIG_TEXT: [plesk-apache-badbot]
enabled = true
filter = apache-badbots
action = iptables-multiport[name=BadBots, port="http,https,7080,7081"]
logpath = /var/www/vhosts/system/*/logs/*access*log
/var/log/httpd/*access_log
maxretry = 1
To ensure the jail is working, check Tools & Settings > IP Address Banning (Fail2Ban) > Banned IP addresses or fail2ban log /var/log/fail2ban.log.
Some custom bots are not blocked in certain cases using this method, the failregex can be modified for them, see the article https://support.plesk.com/hc/en-us/articles/29089393480343.
Also, the particular User-Agent can be added rather than modifying the failregex as was advised in third-line https://webpros.atlassian.net/browse/PPS-17230?focusedCommentId=1497685:
add the modified user agent to the badbots = section, as it appears in actual requests:
|\(compatible; Bytespider; spider-feedback@bytedance\.com\)|
Comments
5 comments
Hello,
I entered the above code.
However I get an error message.
What am I doing wrong?
Does anyone know another solution?
------
Error: Unable to add the jail filter: f2bmng failed: ERROR:__main__:File contains no section headers.
file: '<stdin>', line: 1
'badbotscustom = SputnikBot|omgili|socialmediascanner|Jooblebot|SeznamBot|Scrapy|CCBot|linkfluence|veoozbot|Leikibot'.
------
Thanx
Please add a copy and past line for badbotscustom = examplebot|examplebot| to the explanation - Not just a screenshot
Hello Thomas,
This article describes where users can put their own custom values. The ones on the screenshot are just an example and should not be copied.
this entry already exists in Plesk.. you just need to edit it
Does not work, even the bytedance and bytespider are in the jail filter, still receive massive crawl..
Please sign in to leave a comment.