Articles in this section

How to configure Apache Solr search for Dovecot?

Aleksandar Stoev
Updated
kb: how-to

Question

How to configure Apache Solr search for Dovecot?

Answer

Note: Apache Solr search for Dovecot is supported starting from Plesk Obsidian 18.0.74.

  1. Install Java runtime:

    # apt install default-jre

  2. Download Apache SOLR binary from Solr Downloads:

    # curl -L -o solr-9.10.0.tgz "https://www.apache.org/dyn/closer.lua/solr/solr/9.10.0/solr-9.10.0.tgz?action=download"

    Note: At the time of writing this article, the latest version is Solr 9.10.0.

  3. Unpack the archive:

    # tar -xzf solr-9.10.0.tgz; cd solr-9.10.0

  4. Enable ICU Tokenizer Plugin for SOLR by executing the following commands:

    # cp <path_to_solr>/modules/analysis-extras/lib/icu4j-74.2.jar <path_to_solr>/lib/icu4j-74.2.jar

    # cp <path_to_solr>/modules/analysis-extras/lib/lucene-analysis-icu-9.12.3.jar <path_to_solr>/lib/lucene-analysis-icu-9.12.3.jar

  5. Start SOLR server (listen on all interfaces):

    # bin/solr start -p 8983 -Dsolr.jetty.host=0.0.0.0 --force

  6. Create Dovecot core:

    # bin/solr create -c dovecot

  7. Remove default core configuration:

    # rm -f <path_to_solr>/server/solr/dovecot/conf/schema.xml

    # rm -f <path_to_solr>/server/solr/dovecot/conf/managed-schema.xml

    # rm -f <path_to_solr>/server/solr/dovecot/conf/solrconfig.xml

  8. Apply Dovecot-provided configuration:

    # wget https://raw.githubusercontent.com/dovecot/core/main/doc/solr-config-9.xml

    # wget https://raw.githubusercontent.com/dovecot/core/main/doc/solr-schema-9.xml

    # mv solr-config-9.xml <path_to_solr>/server/solr/dovecot/conf/solrconfig.xml

    # mv solr-schema-9.xml <path_to_solr>/server/solr/dovecot/conf/schema.xml

  9. Restart SOLR to regenerate managed-schema:

    # bin/solr stop ; bin/solr start -p 8983 -Dsolr.jetty.host=0.0.0.0 --force

  10. You should now see Dovecot core listed in the SOLR admin interface.

  11. Now switch to Dovecot configuration on Plesk Instance. Create /etc/dovecot/conf.d/16-fts.conf with the following content:

    CONFIG_TEXT: mail_plugins {
     fts = yes
     fts_solr = yes
    }
    fts_autoindex = yes
    fts_driver = solr 

    language en {
     default = yes
    }

    fts solr {
     url = http://<ip>:<port>/solr/dovecot # put ip of your solr host and port. And '/dovecot' is the name of core we created before
     batch_size = 1000
    }

    # userdb_sql_iterate_query obtains information from userdb sqlite database
    # The database itself could be found at /var/lib/plesk/mail/userdb/userdb.db
    # It is maintained by plesk itself
    userdb_sql_iterate_query = SELECT mailbox_name AS username, domain_name AS domain, mailbox_name || '@' || domain_name AS user FROM mailbox_quota;

    Note: Make sure to replace <ip>:<port> with the actual IP address and port.

  12. Restart Dovecot to load the new configuration:

    # systemctl restart dovecot

  13. Trigger full re-indexing:

    # doveadm fts rescan -A

  14. (Optional) Use an IMAP client or telnet session to verify search functionality. Telnet example commands:

    # a login user@example.com secretpassword

    # b select Inbox

    # c SEARCH TEXT "hello"

     

Was this article helpful?

Comments

5 comments
Date Votes
  • Avatar
    Host Europe
    (Edited )

    Two questions:

    * Is binding this to all interfaces needed/advisable from a security standpoint?
    * This does not automatically start the solr service yet?

    0
  • Avatar
    Aleksandar Stoev

    Hello Host Europe 

    Is binding this to all interfaces needed/advisable from a security standpoint?

    This depends on whether a remote or a local Solr instance is being used. If a remote one is being used, then binding on all interfaces would be necessary to ensure proper connectivity. However, if Solr is hosted on the same server as Plesk (a local Solr instance), then binding to localhost is generally the preferred approach:
    # bin/solr start -p 8983 -Dsolr.jetty.host=127.0.0.1 --force

    Thank you for pointing this out! The article will be updated to include this information.

    As for your second question - the service won't start automatically indeed. We're currently exploring a systemd service-based approach, so the service will be automatically started. Once we've completed the necessary tests, the article will be updated accordingly.

    1
  • Avatar
    Cmdr
    (Edited )

    Here are a few things I noticed while setting this up:

    You’re loading the ICU tokenizer plugin, but you’re not explicitly allowing Solr to load external plugins. With the configuration shown above, additional plugins will not be used. You can either allow plugins selectively or enable general library execution by adding the following to your solr.in.sh:

    SOLR_OPTS="$SOLR_OPTS -Dsolr.jetty.host=127.0.0.1 -Dsolr.config.lib.enabled=true"

    As Host Europe already pointed out, binding Solr to localhost is generally a safer default than binding to 0.0.0.0.


    Many non-English Plesk users probably also want to change the default language to their own and add English as a second language in the 16-fts.conf config file, among language specific filters and langauge tokenizers.

    Regarding the initial indexing of documents in Solr: I found that doveadm fts rescan -A doesn’t actually perform a full index on its own. It only updates parts of the index when a mailbox is actively searched or when new email arrives.

    If you want to push all existing emails into Solr immediately after running doveadm fts rescan -A, use:


    doveadm index -u "user@domain.tld" "name_of_imap_folder"

    I initially hoped to use:

    doveadm index -u user@domain.tld "*"

    but for some reason the wildcard doesn’t work - nothing happens. To work around this, I wrote a very simple script that lists all folders of a user’s mailbox and runs doveadm index for each one. This caused Solr’s document count to increase immediately.

    You can run the script per mailbox, for example:

    ./index.sh user1@domain.tld


    #!/bin/bash
    USER_EMAIL="$1"
    if [ -z "$USER_EMAIL" ]; then
       echo "Usage: $0 user@domain.tld"
       exit 1
    fi

    echo "Starting full FTS indexing for $USER_EMAIL"
    doveadm mailbox list -u "$USER_EMAIL" | while IFS= read -r FOLDER; do
       echo "Indexing folder: $FOLDER"
       doveadm index -u "$USER_EMAIL" "$FOLDER"
    done
    echo "All folders indexed for user: $USER_EMAIL"


    If you have a lot of mailboxes, you might want to extend it with an additional loop to run it for all mailboxes in one go.

    If anyone knows why * doesn’t work with doveadm index (or other doveadm commands), please let me know.

     Startup script to run Solr as a service:


    # /etc/systemd/system/solr.service
    [Unit]
    Description=Apache Solr for Dovecot FTS
    After=network.target

    [Service]
    Type=forking
    User=solr
    Group=solr
    ExecStart=/opt/solr/bin/solr start
    ExecStop=/opt/solr/bin/solr stop
    ExecReload=/opt/solr/bin/solr restart
    Restart=on-failure

    [Install]
    WantedBy=multi-user.target
     


     




     

    0
  • Avatar
    Talistech
    (Edited )

    Thanks for the detailed explanation, I have used your guide to deploy this on my servers.

    3 changes I have made on top of this tutorial;

    • I have created a solr user and made this user owner of my solr folder, instead of running this with root.
    • I have created a systemd service so I can manage this via systemd. Also easier to start on boot instead of planning a cron job.
    • I have made an override for the dovecot service to wait for solr to launch before launching itself.

    I was going to publish my version to the plesk forum but Cloudflare wouldnt let me.

    0
  • Avatar
    Charlie Jone
    (Edited )

    Is it possible to run multiple Solr cores for different domains or mailboxes in Dovecot—for example, to isolate indexing between large tenants and if so, how would the Dovecot fts_solr configuration be structured to support multiple cores? Slither io

    0

Please sign in to leave a comment.