Applicable to:
- Plesk Onyx for Linux
Symptoms
-
Mail delivery does not work periodically. The following messages can be found in
/var/log/maillog:CONFIG_TEXT: hostname postfix/smtpd[2177845]: warning: read TCP map reply from 127.0.0.1:12346: unexpected EOF (Connection timed out)
hostname postfix/smtpd[2177845]: warning: read TCP map reply from 127.0.0.1:12346: unexpected EOF (Connection timed out) -
These
unexpected EOFerrors itself are not an issue, they usually occur when smtp limits are reached, for example in/var/log/maillog:CONFIG_TEXT: Jul 22 13:44:07 cp postfix / master [6027]: warning: service "smtp" (25) has reached its process limit "100": new clients may experience noticeable delays 2.
OR
CONFIG_TEXT: Jul 22 18:16:40 cp postfix / smtpd [21470]: warning: Connection concurrency limit exceeded: 51 from unknown [203.0.113.2] for service smtp
Cause
SMTP connections exceed limits, causing SRS checker service to time out.
Resolution
-
For service "smtp" (25) has reached its process limit "100" - check if default_process_limit needs to be increased or block malicious connections as described here: Postfix keeps stopping, preventing the sending of mail
-
For "Connection concurrency limit exceeded: 51", check the limit for smtpd_client_connection_count_limit as described here: Email are not recieving on 587 submission port: Connection concurrency limit exceeded
Comments
11 comments
My server just started getting this error.
Restarting the service seems to get it stable for 5 minutes or so.
The output matches the above. Adding the -s as stated in the turning off SRS thread has no impact on whether it occurs or not.
Any other suggestions?
@John
Does your server use CentOS? If so, you may also check SELinux (https://support.plesk.com/hc/en-us/articles/360001677533 or https://support.plesk.com/hc/en-us/articles/360003479954-Unable-to-send-and-email-in-Plesk-on-server-with-SELinux-enabled-Unable-to-open-file-var-lib-plesk-mail-srs-secret-Permission-denied)
Yes CentOS , SElinux is diabled
@John
Then it is needed to investigate the issue thoroughly. Please submit a support request using your available option.
I Have the same problem.
Configuration OK
grep postfix-srs /etc/postfix/master.cf
127.0.0.1:12346 inet n n n - - spawn user=popuser:popuser argv=/usr/lib64/plesk-9.0/postfix-srs
Selinux disabled
sestatus
SELinux status: disabled
Maillog
Nov 20 15:48:42 s3 postfix/smtpd[19836]: warning: read TCP map reply from 127.0.0.1:12346: unexpected EOF (Connection timed out)
Nov 20 15:48:42 s3 postfix/smtpd[19836]: warning: tcp:127.0.0.1:12346 lookup error for "mymail@mydomain.com"
Hi @Robert,
Are there any messages like the one below in the maillog as well?
warning: /usr/lib64/plesk-9.0/postfix-srs: process id #####: command time limit exceeded
Yes! a lot
Nov 26 03:56:53 s3 postfix/spawn[5033]: warning: /usr/lib64/plesk-9.0/postfix-srs: process id 5043: command time limit exceeded
Nov 26 03:56:53 s3 postfix/spawn[5052]: warning: /usr/lib64/plesk-9.0/postfix-srs: process id 5054: command time limit exceeded
Nov 26 03:56:53 s3 postfix/spawn[5056]: warning: /usr/lib64/plesk-9.0/postfix-srs: process id 5057: command time limit exceeded
Then let me provide you with some tips on how to troubleshoot this issue. Basically in can occur in case there is one IP that connects to the server and then does not close the connection for some reason.
1. Find all 'max connection count' in the maillog, it should looks like the example below:
# grep 'max connection count' /var/log/maillog | tail -n 2
postfix/anvil[28023]: statistics: max connection count 34 for (smtp:203.0.113.2 ) at Sep 3 11:47:48
postfix/anvil[51714]: statistics: max connection count 11 for (smtp:203.0.113.2 ) at Sep 3 12:25:57
In this example, there are 34 and 11 connections from 203.0.113.2 .
2. Then, check this IP with the netstat in order to understand whether or not there are some hanging connections:
# netstat -tulnap | grep 203.0.113.2 | wc -l
23
# netstat -tulnap | grep 203.0.113.2 | head -n 2
tcp 0 1 203.0.113.10:25 203.0.113.2:49701 FIN_WAIT1 -
tcp 0 1 203.0.113.10:25 203.0.113.2:49784 FIN_WAIT1 -
3. Now it's time to check Connect/Disconnect ratio from offending IP:
This is the normal output (connect matches disconnect):
# grep 'connect .*203.0.113.2' /var/log/maillog | awk '{print $6}' | sort | uniq -c
603 connect
603 disconnect
This is not correct output:
# grep 'connect .*203.0.113.2' /var/log/maillog | awk '{print $6}' | sort | uniq -c
86 connect
20 disconnect
4. In theory, the following script is intended to provide IPs addresses connection ratio statistics:
# for IP in `grep 'max connection count' /var/log/maillog | awk '{ print $12 }' | sort | uniq | sed 's/(smtp://g; s/)//g;'`; do echo -e "Checking IP.. $IP"; egrep "connect .*$IP" /var/log/maillog | awk '{print $6}' | sort | uniq -c ; done
Overall in case on step 3 you are getting connect/disconnect ratio mismatch, this means that host 203.0.113.2 does not close the connection and is causing the issue. Solution would be to either ban this IP address or contact them somehow letting them know that their mail client is malfunctioning.
As a workaround, it should be also possible to increase number of connection from one IP address at Tools & Settings > Mail Server Settings.
Ok thanks, I try to activate fail2ban
As per the following output I have adjusted the script to only get the IPs and remove the `(smtp:`, `(smtps:`, `(submission:` and `)` strings.
# for IP in `grep 'max connection count' /var/log/maillog | awk '{ print $12 }' | sort | uniq | grep -Eo '([0-9]\.?)+'`; do echo -e "Checking IP.. $IP"; egrep "connect .*$IP" /var/log/maillog | awk '{print $6}' | sort | uniq -c ; doneHello Reynaldo Abimael Aceves Chamosa
Thank you for sharing =)
Please sign in to leave a comment.