Articles in this section

Grafana is flooding with emails on Plesk: Firing Value: [no value] Labels: - alertname = DatasourceError

Renan Poss Moreira
Updated
kb: technical

Applicable to:

  • Plesk for Linux

Symptoms

  • The following is shown by Grafana:

    Firing Value: [no value] Labels: - alertname = DatasourceError - rule_uid = XxXxXxXx - rulename = Partition "/var" utilization - sourceKey = hdd_util_var Annotations: - message = The disk space is almost full! Source: /modules/grafana/service/alerting/grafana/XxXxXxXx/view Silence: /modules/grafana/service/alerting/silence/new?alertmanager=grafana&matcher=alertname%3DDatasourceError&matcher=rule_uid%XxXxXxXx&matcher=rulename%3DPartition+%22%2Fvar%22+utilization&matcher=sourceKey%3Dhdd_utilvar Dashboard: /modules/grafana/service/d/monitoringhdd Panel: /modules/grafana/service/d/monitoring_hdd?viewPanel=4

    *Firing* Value: [no value] Labels: - alertname = DatasourceError - rule_uid = XxXxXxXx - rulename = nginx memory usage - sourceKey = memory_usage__webproxy Annotations: - message = The memory usage status is critical! Source: /modules/grafana/service/alerting/grafana/XxXxXxXx/view Silence: /modules/grafana/service/alerting/silence/new?alertmanager=grafana&matcher=alertname%3DDatasourceError&matcher=rule_uid%XxXxXxXx&matcher=rulename%3Dnginx+memory+usage&matcher=sourceKey%3Dmemory_usage__webproxy Dashboard: /modules/grafana/service/d/monitoring__services Panel: /modules/grafana/service/d/monitoring__services?viewPanel=7
    The current value is .

  • Plesk notifications show messages about exceeded thresholds which were not reached

  • The following message can be found in /var/log/grafana/grafana.log:

    logger=plugin.manager t=2022-06-22T13:46:52.610714606+01:00 level=error msg="Could not start plugin" pluginId=grafana-simple-json-backend-datasource err="Incompatible API version with plugin. Plugin version: 1, Client versions: [2]"

    logger=ngalert uid=XxXxXxX org=1 t=2022-06-22T04:39:50.041885056+01:00 level=error msg="failed to fetch alert rule" err="database is locked"

Cause

Unsupported Grafana version is installed.

Resolution

For Plesk on RHEL based Linux (AlmaLinux/RockyLinux/CentOS)

  1. Connect to the server via SSH

  2. Downgrade version of the the grafana package to the current one:

    # yumdb search from_repo plesk-ext-grafana
    Loaded plugins: fastestmirror
    grafana-8.3.6-1.x86_64
    from_repo = plesk-ext-grafana

    # rpm -e --nodeps grafana.x86_64 9.0.0-1

    # yum install grafana-8.3.6-1.x86_64

  3. Restart grafana:

    # service grafana-server restart

For Plesk on Debian based Linux (Debian/Ubuntu)

  1. Connect to the server via SSH

  2. Find the file containing non Plesk Grafana repository with this command:

    # grep -r graf /etc/apt/
    /etc/apt/sources.list:deb https://packages.grafana.com/oss/deb stable main
    /etc/apt/sources.list.d/plesk-ext-grafana.list:deb https://autoinstall.plesk.com/grafana/deb stable main

  3. Remove the line containing "packages.grafana.com" from the file found in step 2. (/etc/apt/sources.list)

  4. Install supported Grafana version:

    # apt-get install grafana=8.3.6

  5. Restart Grafana:

    # service grafana-server restart

Was this article helpful?

Comments

1 comment
Date Votes
  • Avatar
    Salvador Salas Prieto

    Hello,

    I started to receive these mails in one of my servers with Debian 8 but I don´t have the line you told in step 2:

     

    root@xxx:/# grep -r graf /etc/apt/

    /etc/apt/sources.list.d/plesk-ext-grafana.list:deb https://autoinstall.plesk.com/grafana/deb stable main

    Coincidencia en el fichero binario /etc/apt/trusted.gpg

    Also in /var/log/grafana/grafana.log I can see:

    logger=ngalert.scheduler rule_uid=avOeDO37k org_id=1 version=1 attempt=0 now=2023-06-23T00:03:00+02:00 t=2023-06-23T00:03:00.000941868+02:00 level=error msg="Failed to evaluate rule" error="[plugin.notRegistered] plugin not registered" duration=0s

    0

Please sign in to leave a comment.