Ultimate bash script to backup domoticz.db every 5 min. to NAS

All kinds of 'OS' scripts

Moderator: leecollings

Post Reply
Kedi
Posts: 564
Joined: Monday 20 March 2023 14:41
Target OS: Raspberry Pi / ODroid
Domoticz version:
Location: Somewhere in NL
Contact:

Ultimate bash script to backup domoticz.db every 5 min. to NAS

Post by Kedi »

Below you will find my script to backup the Domoticz database every 5 minutes to a location on a NAS or any other mountable medium.
Linux programs bc and curl have to be installed. Also your NAS has to support external mounting.
In the script nfs is used, but you could also use samba, or even a SD-drive in one of the USB-ports.
Before everything you have to check if the command in variable MOUNTEXEC is working, you might have to tweak it.
You have create a Dummy device type Custom with Kb as Y-axis. In that device the size of the domoticz database is monitored.
The success or error is logged in a log file.
I have a NAS with Gigabit LAN, my Raspberry Pi 4B takes less then 2 seconds for a 33 Mb database and a Raspberry 3B+ takes less then 1 sec. for 8 Mb

A whole lot is at the beginning of the script configurable. Default it saves every 5 minutes for 3 days and every day (at 00:00) for 1 year.
My master domoticz backup takes 42 Gb space and my slave domoticz backup is 10 Gb big.

Care has been taken to make sure the script is working, but there is no guarantee.
B.t.w. I place my Domoticz cron scripts in: /home/pi/domoticz/scripts/cron/

Code: Select all

#!/usr/bin/env /bin/bash

SCRIPTNAME=`basename "$0"`
SCRIPTNAME=${SCRIPTNAME%.*}
LOGFILE=/var/log/$SCRIPTNAME.log
ERRORLOG=/var/log/$SCRIPTNAME-ERROR.log
HOSTNAME=$(hostname)
DOMO_IP="127.0.0.1"				# Domoticz IP
DOMO_PORT="8080"				# Domoticz port
DOMO_IDX=1503					# Domoticz IDX for Filesize monitor (Dummy/Custom with Y-axis Kb)
REMOTEHOSTIP="192.168.x.xxx"	# Remote host where the backup database is saved
REMOTEPATH="/volume1/RaspberryPi/Domoticz_DB/$HOSTNAME"
SOURCEFILE="domoticz.db"
SOURCEPATH="/home/pi/domoticz/"
MOUNTPOINT="/mnt/$SCRIPTNAME"
MOUNTEXEC="sudo mount -t nfs $REMOTEHOSTIP:$REMOTEPATH $MOUNTPOINT" # command to execute for mounting the remote host
LTB="LongtermBackup"
TMP_PATH="/tmp" 				# This dir should have empty space for 2 times the database size
# Delete backups older than 3 days = 864 (288 per day is saved every 5 minutes)
# 360 equals 30 hours of backup.
# Make daily backup of just 1 daily file at 00:00. in the LogtermBackup folder
MAX_NUM_BACKUPS=864
# Delete LongtermBackups older than 366 days
MAX_NUM_LTB_BACKUPS=366
MINUTES=$(date +"%M")

# Cron entry should be made
# */5 *   *   *   *   root    <path of the script>/<name of the script> >/dev/null 2>/dev/null
#

### === END OF USER CONFIGURABLE PARAMETERS === ###

if ! [ $(id -u) = 0 ]; then
    echo "Please run as root"
    exit 1
fi
TMP=`which bc`
if [[ -z "$TMP" ]]; then
    echo "Programm 'bc' is not installed! Install with sudo apt-get install bc"
    exit 2
fi

TMP=`which curl`
if [[ -z "$TMP" ]]; then
    echo "Programm 'curl' is not installed! Install with sudo apt-get install curl"
    exit 3
fi

# Remove this check if nfs mouting is NOT used.
if [[ $MOUNTEXEC == *"-t nfs"* ]]; then
    TMP=`which mountstats`
    if [[ -z "$TMP" ]]; then
        echo "Programm 'nfs-common' is not installed! Install with sudo apt-get install nfs-common"
        exit 4
    fi
fi

if ! [ -x "$(command -v sqlite3)" ]; then		# check if sqlite3 exists
	echo "sqlite3 does not exists, install with sudo apt-get install sqlite3"
	exit 5
fi

sleep 10		# to prevent running at exactly 5 minutes boundery where all other dzVents scripts might run.

# Start Timestamp for logging purpose.
STARTTIME=$(date +"%s.%N")

# Place in the log file information concerning the execution of this script
echo -n "Domoticz database backup start at " >> $LOGFILE
echo $(date) >> $LOGFILE

if [ ! -d $MOUNTPOINT ]; then
    mkdir -p $MOUNTPOINT
fi

if ! mountpoint -q $MOUNTPOINT; then
	eval $MOUNTEXEC
fi
if ! /bin/mountpoint -q $MOUNTPOINT; then
    touch $ERRORLOG
    chmod 0666 $ERRORLOG
    echo "Mount $MOUNTPOINT ERROR!!!" >> $ERRORLOG
    exit 99
fi

cd $MOUNTPOINT
if [ ! -d $MOUNTPOINT/$LTB ]; then
	mkdir $LTB
fi
ls -tr | head -n -$MAX_NUM_BACKUPS | xargs --no-run-if-empty rm
cd $MOUNTPOINT/$LTB/
ls -tr | head -n -$MAX_NUM_LTB_BACKUPS | xargs --no-run-if-empty rm

# Make a copy of the backup created at midnight to the 'LongTermBackup' directory
MINUTES_ZERO=`date +%M`
HOURS_ZERO=`date +%H`
if [[ "$HOURS_ZERO" == "00" && "$MINUTES_ZERO" == "00" ]]; then
	file2copy=`find $MOUNTPOINT -maxdepth 1 -type f -printf "\n%AT %p" | grep "0000" | head -n 1 | awk '{print $2}'`
	part1=$(dirname "$file2copy")
	part2=$(basename "$file2copy")
    cp $file2copy  $part1/$LTB/$part2
fi

### Create backup
TIMESTAMP=`/bin/date +%Y%m%d%H%M%S`
DB_VERSION=`sqlite3 $SOURCEPATH$SOURCEFILE "Select nValue from Preferences WHERE Key = 'DB_Version'"`
BUILD=`/home/pi/domoticz/domoticz --help | grep Giz | cut -d "(" -f2 | cut -d ")" -f1 | awk '{print $2}'`
# backups will be named: domoticz_YYYYMMDDHHMMSS_XXX_ZZZZZ.db
# where YYYYMMDDHHMMSS is current date and time, XXX is DB Version and ZZZZZ is build version.
BACKUPFILE=$HOSTNAME"_"$TIMESTAMP"_"$DB_VERSION"_"$BUILD".db"
sqlite3 $SOURCEPATH$SOURCEFILE ".backup $MOUNTPOINT/$BACKUPFILE"

# Get database filesize and report to Domoticz device.
FILESIZE=`stat -c%s "$SOURCEPATH$SOURCEFILE"`
FILESIZE=$((`expr $FILESIZE / 1000`))
curl "http://$DOMO_IP:$DOMO_PORT/json.htm?type=command&param=udevice&idx=$DOMO_IDX&nvalue=0&svalue=$FILESIZE"

sync
/bin/umount -l $MOUNTPOINT

# End timestamp
ENDTIME=$(date +"%s.%N")

# Convert nanoseconds to milliseconds
# crudely by taking first 3 decimal places
TIMEDIFF=`echo "$ENDTIME - $STARTTIME" | bc | awk -F"." '{print $1"."substr($2,1,3)}'`
SEC=`echo "$TIMEDIFF % 60" | bc`

echo "Total of $SEC seconds elapsed. Size: $FILESIZE Kb" >> $LOGFILE
echo "++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++" >> $LOGFILE

### Done!
Edit: check for nfs-common is added.
Logic will get you from A to B. Imagination will take you everywhere.
Post Reply

Who is online

Users browsing this forum: No registered users and 0 guests