Apache is running but not responding (how to workaround DDoS attack)

Created:

2016-11-16 13:03:39 UTC

Modified:

2017-04-24 11:30:28 UTC

0

Was this article helpful?


Have more questions?

Submit a request

Apache is running but not responding (how to workaround DDoS attack)

Applicable to:

  • Plesk 12.5 for Linux
  • Plesk 11.0 for Linux
  • Plesk 11.5 for Linux
  • Plesk 12.0 for Linux
  • Plesk 10.1 for Linux

Symptoms

Heavy load on the server. All domains scripts are fine - no enormous load running 'top' command.

Cause

Can be a chance of DDoS attack.

In the /var/log/apache2/error.log :

[Mon Sep 23 14:33:34 2013] [error] [client 1.2.3.4] request failed: error reading the headers

Resolution

We can confirm it by checking the result of netstat command:

 netstat -an|awk '/tcp/ {print $6}'|sort|uniq -c

This will show the states and number of connections at that time. The different states that are visible mostly in servers are:

  1. ESTABLISHED - This will be legitimate connections established to the server
  2. SYN_SENT - The client will be actively attempting to establish a connection.
  3. SYN_RECV - A connection request has been received from the network.
  4. FIN_WAIT - The socket is closed, and the connection is shutting down.
  5. TIME_WAIT - The socket is waiting after close to handle packets still in the network.
  6. LISTEN - The socket is listening for incoming connections.
  7. LAST_ACK - The remote end has shut down, and the socket is closed. Waiting for acknowledgement.

If the number of connections in SYN_SENT , SYN_RECV , TIME_WAIT , FIN_WAIT are very large in the rate of 1000s then the server is surely under attack.

As a first step we can tweak the values set for SYN_SENT , SYN_RECV , TIME_WAIT , FIN_WAIT in the file /etc/sysctl.conf . Reduce the value of net.ipv4.tcp_fin_timeout to 3 or 5. Normally it set to 120, by default. Make the following changes in /etc/sysctl.conf

 # Enable TCP SYN cookie protection
net.ipv4.tcp_syncookies = 1

# Decrease the time default value for tcp_fin_timeout connection
net.ipv4.tcp_fin_timeout = 3

# Turn off the tcp_window_scaling
net.ipv4.tcp_window_scaling = 0

# Turn off the tcp_sack
net.ipv4.tcp_sack = 0

Then execute the command:

 sysctl -p

Afterwards, we will have to find out how the attack is being performed, is it from particular IP or from large number of IP addresses to the server. If it is from any particular IP to the server, then we can fix it by blocking the IP in the firewall. If it is from a large number of IP with one or 2 connections then we will have to find more details to stop it. But will will not be able to completely stop the DDOS attack, we will have to tweak some settings in the server so that the number of connections can be reduced.

Once you understand the port you need to figure out is the attack done on a particular domain or IP. Suppose the attack is done on port 80, then we can tweak the apache settings as follows:

  1. Increase the MaxClients so that we can prevent the condition of apache reaching its limit, since apache could not serve new requests. MaxClients can be set to a max value of the limit set in ServerLimit
  2. Set KeepAlive on to set the KeepAliveTimeout
  3. KeepAliveTimeout value to be reduced to 3 or 5

So the settings will be as follows:

 MaxClients 500
KeepAlive On
KeepAliveTimeout 3

 /etc/init.d/httpd restart

In order to narrow down the issue, we need to find out if the attack is on any particular IP in the server. This can be found using the following command:

netstat -lpan | grep SYN_RECV | awk '{print $4}' | cut -d: -f1 | sort | uniq -c | sort -nk 1

After confirming the attack to the IP, we need to find out if the attack is made to a particular domain in that IP or to the IP as a whole. For that, you can check the apache error logs or top command.

Also you may limit number of the connections by the following command:

iptables -I INPUT -i eth0 -p tcp --syn --dport 80 -m connlimit --connlimit-above 16 --connlimit-mask 24 -j REJECT
Have more questions? Submit a request
Please sign in to leave a comment.