Guide - How to check BTRFS volume integrity using Monit

[ben53252642]

A Monit rule and script that does a read only BTRFS file system volume scrub (in this case the root volume / ) at a set time each day and generates a Monit alert if errors were detected.

Rule for /etc/monit/monitrc:

Code: Select all

# BTRFS File System Integrity
check program btrfsintegritycheck with path "/etc/monit/scripts/btrfsintegritycheck.sh"
 with timeout 1800 seconds
 # Run at 4am each day
 every "0 4 * * *"
 if status != 0 then alert

Script that goes in:
/etc/monit/scripts/btrfsintegritycheck.sh

Code: Select all

#!/bin/bash
btrfs scrub start -Br / | grep -e 'no errors found' -e 'with 0 errors'

This is an extremely useful tool for detecting for example corruption of an SD card.

[ben53252642]

Wow! Monit / Mmonit just picked up a server with a corrupt BTRFS volume!

I investigated and sure enough, it was corrupt with unfixable errors, now restoring from a backup.

monit_btrfs.JPG (342.39 KiB) Viewed 884 times

Checking the server manually (output below):

btrfs2.JPG (111.72 KiB) Viewed 882 times

[ben53252642]

I really, really like BTRFS!

Using the below command (will probably vary a bit for different systems), I was able to get a list of files that had become corrupt on the BTRFS file system, delete them (which fixed the file system corruption), then restore just those files that had become corrupted from a backup.

journalctl --output cat | grep 'BTRFS' | grep '(path:' | sed -n -e 's/^.*\(path:\)/\1/p' | cut -f2- -d: | tr -d ")"

Note that I ran this command before the one above: btrfs scrub start -Bd /dev/sda2

Upon deleting the list of corrupt files....

btrfs5.JPG (57.81 KiB) Viewed 864 times