How to Silence Netdata Alarms

Last Updated on April 2, 2024 EDT by Jordan

Understanding Netdata Alarms

Before silencing an alarm, it’s crucial to understand how Netdata alarms work. Netdata provides a set of predefined alarms and allows users to customize or create new ones. Each alarm is defined in a .conf file located in the health.d directory of your Netdata installation, typically found at /etc/netdata/health.d/.

There are a number of Netdata alerts that will occur that might not actually be applicable for your environment.

Common Netdata Alarms to Silence for WordPress

There are a number of common netdata alarms that you might see which you can silence that might not be applicable for your server hosting your WordPress sites.

• web_log_1m_unmatched
• web_log_1m_redirects
• web_log_1m_bad_requests
• web_log_1m_successful

Silencing Alarms Permanently

To permanently silence an alarm, for instance web_log_1m_redirects which triggers when you have a high ratio of HTTP status code 3xx requests in your logs versus others HTTP status codes.

Step 1 – SSH into your Server

Connect to your server where Netdata is installed.

Step 2 – Edit the alarm configuration

You now need to edit the alarm configuration, for web_log_1m_redirects it’s using the web health config.

cd /etc/netdata
./edit-config health.d/web_log.conf

Step 3 – Silence the alarm

Locate the web_log_1m_redirects template and change the to: webmaster to to: silent.

Here’s the original section

 template: web_log_1m_redirects
       on: web_log.type_requests
    class: Workload
     type: Web Server
component: Web log
   lookup: sum -1m unaligned of redirect
     calc: $this * 100 / $web_log_1m_requests
    units: %
    every: 10s
     warn: ($web_log_1m_requests > 120) ? ($this > (($status >= $WARNING ) ? (  1 ) : ( 20 )) ) : ( 0 )
    delay: up 2m down 15m multiplier 1.5 max 1h
  summary: Web log redirects
     info: Ratio of redirection HTTP requests over the last minute (3xx except 304)
       to: webmaster

Here’s the modified section

 template: web_log_1m_redirects
       on: web_log.type_requests
    class: Workload
     type: Web Server
component: Web log
   lookup: sum -1m unaligned of redirect
     calc: $this * 100 / $web_log_1m_requests
    units: %
    every: 10s
     warn: ($web_log_1m_requests > 120) ? ($this > (($status >= $WARNING ) ? (  1 ) : ( 20 )) ) : ( 0 )
    delay: up 2m down 15m multiplier 1.5 max 1h
  summary: Web log redirects
     info: Ratio of redirection HTTP requests over the last minute (3xx except 304)
       to: silent

Step 4 – Restart Netdata

You will need to restart Netdata for the changes to take affect.

sudo netdatacli reload-health