Articles in this section

See more

High CPU usage for "sa-learn" and "spamtrain" processes in a server with Plesk

Avatar
Nelson Leiva
Updated
Follow
Plesk for Linux kb: technical ABT: Group A

Applicable to:

  • Plesk for Linux

Symptoms

High CPU usage for sa-learn and spamtrain processes in the Plesk server.

Cause

Since sa-learn script is used by spamassassin to learn the specific properties of spam from a folder which supposedly contains only spam emails.
CPU usage of the sa-learn process depends on the amount of the emails in mailboxes and their size. In turn, if the number of emails is huge - CPU over usage is expected.

Resolution

To work around the issue, use one of the following solutions:

  1. Connect to the server using SSH.

  2. Use nice or ionice utilities to decrease the priority of sa-update from the cron tasks.

  3. Create a wrapper file which will execute sa-learn with the lowest CPU priority:

    # cp -a /usr/bin/sa-learn{,.orig}
    # echo "/bin/nice -19 /usr/bin/perl -T -w /usr/bin/sa-learn.orig \$@" > /usr/bin/sa-learn
    # chmod +x /usr/bin/sa-learn

    Note: In Debian based OS the correct path is /usr/bin/nice

  4. Set the lowest priority to the currently running sa-learn processes execute the following:

    # for i in $(ps axuwf | grep sa-learn | awk '{print $2}'); do renice 19 $i; done

    Note: Process priority values range from -20 to 19. A process with the nice value of -20 is considered to be on top of the priority. And a process with nice value of 19 is considered to be low on the priority list.

  5. Disable the sa-update execution by moving 60sa-update and spamassassin files from /etc/cron.daily to a different location:

    # mv /etc/cron.daily/60sa-update /root

Comments

10 comments

Sort by Date Votes
  • Avatar
    Alexander Bien

    Are you sure that we need to modify 60sa-update in order to change high cpu from sa-learn?

    This pstreelog output (C7, P17.8.x) would suggest otherwise:

    |-anacron---run-parts-+-50plesk-daily---sw-engine---sw-engine---spamtrain---su---sa-learn

    I couldnt find any mentioning of sa-learn in sa-update either. Please advise.

    0
    Comment actions Permalink
  • Avatar
    Robert Asilbekov

    @Alexander Bien

    Thank you for the feedback. The article has been updated.

    0
    Comment actions Permalink
  • Avatar
    Peter Debik

    On CentOS

    echo "/bin/nice -19 /usr/bin/perl -T -w /usr/bin/sa-learn.orig $@" > /usr/bin/sa-learn

    must be executed as

    echo "/bin/nice -19 /usr/bin/perl -T -w /usr/bin/sa-learn.orig \$@" > /usr/bin/sa-learn

    because else, the "$@" will not be exported into /usr/bin/sa-learn.

    0
    Comment actions Permalink
  • Avatar
    Anton Maslov

    Peter, article updated! Thank you for report!

    0
    Comment actions Permalink
  • Avatar
    Bbennett

    Trying to disable and get the error: 

    mv /etc/cron.daily/60sa-update /root
    mv: cannot stat ‘/etc/cron.daily/60sa-update’: No such file or directory

    0
    Comment actions Permalink
  • Avatar
    Ivan Postnikov

    Hello Bbennett

    Did you execute this command as a root user?

    The 60sa-update exists on my test server and the issue wasn't reproduced:

    # stat /etc/cron.daily/60sa-update
    File: '/etc/cron.daily/60sa-update'
    Size: 448 Blocks: 8 IO Block: 4096 regular file
    Device: 40f0b681h/1089517185d Inode: 409504 Links: 1
    Access: (0755/-rwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root)
    Access: 2020-01-23 13:44:39.000000000 +0700
    Modify: 2020-01-23 13:44:39.000000000 +0700
    Change: 2020-01-27 20:52:40.452451535 +0700
    Birth: -

     

    # plesk -v
    Product version: Plesk Obsidian 18.0.23.1
    OS version: CentOS 7.5.1804 x86_64

     

    0
    Comment actions Permalink
  • Avatar
    Jair Cueva Junior

    In Ubuntu 18.04 this wrapper scripts was failing due to a slightly difference in the path of "nice" command, it should be: /usr/bin/nice

    Thanks for exposing this approach, it has opened my mind for possibilities!

    0
    Comment actions Permalink
  • Avatar
    Nelson Leiva

    Hi Jair Cueva Junior,

    The article has been updated. Thank you for noticing that!

    0
    Comment actions Permalink
  • Avatar
    Michael Fryd

    When importing large email accounts for new clients, my server was bogged down, even with sa-learn running with low priority. 

    We don't see single instances of sa-learn using a lot of resources, we see many instances of sa-learn running sequentially.



    My solution was to write a script that checks the system load average.  If the load is getting high, it pauses before running sa-learn.  This helps keep the load from getting too high.  If the load doesn't come down, or the load is too high, it doesn't bother running sa-learn.  This seems to be OK. I suspect the message that were skipped will be examined on a subsequent run.

     

    Here's my replacement for /usr/bin/sa-learn:

     

    #!/bin/bash
    #
    # replacement for /usr/bin/sa-learn
    # hack to keep sa-learn process from using too many resources
    # created 6/26/2020
    #
    # PLESK periodically calls /usr/bin/sa-learn to train spam assassin on recent mail.
    # if large amounts of new mail are present (perhaps a large external email
    # account was recently imported) the i/o burden can bog down the server. We have
    # seen load averages above 300.
    #
    # our solution is multi-pronged.
    # first, we simply don't run sa-learn if the load avergage is too high.
    # secondly, if the load average is starting to get high, we pause a little bit before
    # starting sa-learn
    # finally, if we do start sa-learn, we use nice to run it at a low priority.
    #
    # We have moved the original sa-learn script to /usr/bin/sa-learn.orig
    #
    # Note: we truncate the load average and look at only the integer portion.
    #
    # -Michael Fryd (michael@fryd.com) 6/26/2020
    #


    PUNT_LOAD_LIMIT=4 # If the load is higher than this we exit and don't run sa-learn.orig
    WAIT_LOAD_LIMIT=3 # if the load is higher than this, but below PUNT_LOAD_LIMIT, we delay before running
    SLEEP_TIME=0.4    # how long to delay (in seconds)
    MAX_ATTEMPTS=2    # max number of times to wait for load to come down before punting


    count=0
    while [ "$count" -lt "$MAX_ATTEMPTS" ]; do 
        # get the integer portion of the current 1 minute load average
        load=$(uptime | awk -F: '{print $5}' | awk -F"," '{print $1}' | tr -d '[:space:]' | cut -c1) # Poll Load

        # if the load is greater than the punt limit, give up. Don't wait, and no
        # additional passes
        if [[ "$load" -gt "$PUNT_LOAD_LIMIT" ]]; then
            exit 0; # give up, don't run sa-learn.orig at this time
        fi

        # if the load is low, then run sa-learn.orig at low priority, otherwise pause, then loop again
        if [[ "$load" -lt "$WAIT_LOAD_LIMIT" ]]; then
            /bin/nice -n 19 /usr/bin/perl -T -w /usr/bin/sa-learn.orig $@
            exit 0; # If successful
        else
            sleep $SLEEP_TIME;
        fi;
        (( count++ ))
    done;

    0
    Comment actions Permalink
  • Avatar
    Francisco Garcia

    Hi Michael Fryd,

    Thanks for such useful script whenever a mail importing is happening, it may be useful for other users.

    0
    Comment actions Permalink

Please sign in to leave a comment.

Have more questions? Submit a request

Related articles