ESXi disk health in Zabbix: SMART for SATA, health log for NVMe

ESXi disk health in Zabbix: SMART for SATA, health log for NVMe

Table of Contents

The standard Zabbix templates for VMware (VMware and VMware Hypervisor, using the vSphere API) are great for CPU, memory, datastores and VM status. What they don’t tell you is how the physical disks in a standalone ESXi host are doing. You learn that a disk failed when a datastore goes offline — not a week earlier, when the drive started reallocating sectors.

For local storage — homelabs, branch-office hosts, backup servers, small clusters with local NVMe — we use two small templates that read disk health directly from the host over SSH.

Templates: esxi-smart-health and esxi-nvme-health — Zabbix 7.2+, tested with ESXi 7.0.

SATA disks: ESXi SMART Health

ESXi doesn’t ship smartctl, so this template needs it installed first — we package it as a community VIB: smartctl-esxi-vib (install with esxcli software vib install, no reboot needed).

How it works:

  1. Once per hour Zabbix runs ls -1 /dev/disks | grep t10.ATA | grep -v ':' over SSH — the list of SATA disks (without partitions).
  2. Low-level discovery creates one entry per disk, named by its serial number.
  3. For each disk, /opt/smartmontools/smartctl -s on -d sat,auto -A /dev/disks/<disk> is run hourly, and a JavaScript preprocessing step parses the attribute table into JSON.

Items per disk:

Item Why it matters
Reallocated Sector Count Sectors already moved to spare area
Current Pending Sector Unreadable sectors waiting to be reallocated
Offline Uncorrectable Sectors that failed during offline scans
Power On Hours Age of the drive — useful for planning replacements

The template ships with one trigger: Current Pending Sector > 0 (Average) — the clearest early sign of a dying disk. We recommend adding two more on the template, using the same style:

  • change(/ESXI SMART Health/smart.status[{#DISK_ID},Reallocated_Sector_Ct])>0 — the reallocated count grew;
  • last(/ESXI SMART Health/smart.status[{#DISK_ID},Offline_Uncorrectable])>0.

NVMe drives: ESXi NVMe Health

For NVMe, ESXi already has everything built in — nothing needs to be installed on the host. The template uses localcli:

  1. /bin/localcli nvme device list (hourly) discovers NVMe adapters (vmhba…).
  2. /bin/localcli nvme device log smart get --adapter <vmhba> (every 5 minutes) reads the NVMe SMART/health log.

Items and triggers per drive:

Item Triggers
Media Errors > 0 → High — data could not be recovered
Composite Temperature > 60 °C Warning, > 70 °C High
Available Spare < 20 % Warning, < 10 % High
Percentage Used (rated endurance) > 80 % Warning, > 95 % High
Power On Hours, Power Cycles, Controller Busy Time informational

Percentage Used is the one to watch for capacity planning: it’s the drive’s own estimate of how much of its rated write endurance is consumed. A consumer NVMe under a busy VM datastore can go from 10 % to 80 % surprisingly fast.

Setup (both templates)

Both templates use Zabbix SSH agent items: the Zabbix server (or proxy) logs in to ESXi as root with a key and runs the commands. No agent on ESXi.

1. Enable SSH on the ESXi host (Host → Manage → Services → TSM-SSH, set the policy to start with the host).

2. Create a key pair for Zabbix and put it where the Zabbix server looks for keys:

sudo -u zabbix mkdir -p /var/lib/zabbix/ssh_keys
sudo -u zabbix ssh-keygen -t rsa -b 4096 -m PEM -N "" -f /var/lib/zabbix/ssh_keys/esxi

and make sure zabbix_server.conf (or zabbix_proxy.conf) contains SSHKeyLocation=/var/lib/zabbix/ssh_keys.

3. Install the public key on ESXi by appending esxi.pub to /etc/ssh/keys-root/authorized_keys. On ESXi 7 you can also upload it through the host’s API:

curl -k -X PUT --data "@/var/lib/zabbix/ssh_keys/esxi.pub" \
  -u root:<password> https://<esxi-ip>/host/ssh_root_authorized_keys

On ESXi 8.0 U3 this endpoint returns 404 — edit the file directly instead.

4. Import the templates, create a host for the ESXi server with an interface pointing to its IP (the agent interface type is fine, it is only used for the address), link ESXI SMART Health and/or ESXI NVME Health, and set {$ESXI.SSH.KEY} to the key file name (esxi by default).

Tip

If SSH items fail with an authentication error although ssh -i esxi root@host works, regenerate the key in PEM format (-m PEM, as above). Depending on how Zabbix was built, it may not read the newer OpenSSH key format.

A note on security

These checks log in as root. Keep the private key readable only by the zabbix user, restrict SSH on ESXi to the Zabbix server’s address (Firewall → SSH Server → allowed IP addresses), and don’t reuse the key anywhere else. Also note that the smartctl VIB requires Secure Boot to be disabled; the NVMe template has no such requirement.

Summary

  • SATA: smartctl VIB + ESXI SMART Health — reallocated, pending and uncorrectable sectors.
  • NVMe: built-in localcli + ESXI NVME Health — media errors, spare, endurance, temperature.
  • Agentless, over SSH with a dedicated key; ESXi 7.0 and Zabbix 7.2+.

Running VMware or migrating to Proxmox and want the hardware properly monitored? Get in touch.

comments powered by Disqus

Related Posts

Are your network config backups actually working? Oxidized + Zabbix

Are your network config backups actually working? Oxidized + Zabbix

Oxidized is the de facto open-source tool for backing up network device configurations. It logs into switches, routers and firewalls — Cisco, Juniper, MikroTik, Fortinet, HPE/Aruba and dozens of others — downloads the running config and stores every version in Git. When a device dies or someone breaks the config, you have the last known good version and a diff of what changed.

Read More
Testing firewall rules continuously with Zabbix: what should — and shouldn't — be reachable

Testing firewall rules continuously with Zabbix: what should — and shouldn't — be reachable

Network segmentation looks great on the diagram: the DMZ can’t reach the database VLAN, office workstations can’t reach server management interfaces, backups are reachable only from the backup server. Then a year passes. Someone adds a temporary “any-any” rule to debug an issue. A firewall migration converts policies slightly differently. A new VLAN is created by copying an old one.

Read More
How to monitor your home internet connection — and prove it to your ISP

How to monitor your home internet connection — and prove it to your ISP

The video call freezes, the game lags, the trading terminal reconnects. A speed test five minutes later shows 500 Mbit/s and everything looks fine. You call your provider, they “see no issues on the line”, and suggest restarting the router.

Read More