How to Silence Netdata Alarms

Understanding Netdata Alarms

Before silencing an alarm, it’s crucial to understand how Netdata alarms work. Netdata provides a set of predefined alarms and allows users to customize or create new ones. Each alarm is defined in a .conf file located in the health.d directory of your Netdata installation, typically found at /etc/netdata/health.d/.

There are a number of Netdata alerts that will occur that might not actually be applicable for your environment.

Common Netdata Alarms to Silence for WordPress

There are a number of common netdata alarms that you might see which you can silence that might not be applicable for your server hosting your WordPress sites.

• web_log_1m_unmatched
• web_log_1m_redirects
• web_log_1m_bad_requests
• web_log_1m_successful

Silencing Alarms Permanently

To permanently silence an alarm, for instance web_log_1m_redirects which triggers when you have a high ratio of HTTP status code 3xx requests in your logs versus others HTTP status codes.

Step 1 – SSH into your Server

Connect to your server where Netdata is installed.

Step 2 – Edit the alarm configuration

You now need to edit the alarm configuration, for web_log_1m_redirects it’s using the web health config.

cd /etc/netdata
./edit-config health.d/web_log.conf

Step 3 – Silence the alarm

Locate the web_log_1m_redirects template and change the to: webmaster to to: silent.

Here’s the original section

 template: web_log_1m_redirects
       on: web_log.type_requests
    class: Workload
     type: Web Server
component: Web log
   lookup: sum -1m unaligned of redirect
     calc: $this * 100 / $web_log_1m_requests
    units: %
    every: 10s
     warn: ($web_log_1m_requests > 120) ? ($this > (($status >= $WARNING ) ? (  1 ) : ( 20 )) ) : ( 0 )
    delay: up 2m down 15m multiplier 1.5 max 1h
  summary: Web log redirects
     info: Ratio of redirection HTTP requests over the last minute (3xx except 304)
       to: webmaster

Here’s the modified section

 template: web_log_1m_redirects
       on: web_log.type_requests
    class: Workload
     type: Web Server
component: Web log
   lookup: sum -1m unaligned of redirect
     calc: $this * 100 / $web_log_1m_requests
    units: %
    every: 10s
     warn: ($web_log_1m_requests > 120) ? ($this > (($status >= $WARNING ) ? (  1 ) : ( 20 )) ) : ( 0 )
    delay: up 2m down 15m multiplier 1.5 max 1h
  summary: Web log redirects
     info: Ratio of redirection HTTP requests over the last minute (3xx except 304)
       to: silent

Step 4 – Restart Netdata

You will need to restart Netdata for the changes to take affect.

sudo netdatacli reload-health