Backup script Domoticz gives me headaches please help...

[NvBgm55]

Hi, I am using the backup script below which I found on domoticz wiki site. The backup part works great but I can't get the last part where folders older than xx days have to be deleted.

Code: Select all

#!/bin/bash
# Sript to automatic backup some stuff from Domoticz
# Run script every day from crontab.
# Example:	sudo crontab -e
#			Place this line below in your crontab document:
#			0 1 * * * sudo ~/domoticz/scripts/domoticz_backup.sh
#			Backup wil start every night at 1:00 hour past midnight
#
# LOCAL/FTP/SCP/MAIL/FILE PARAMETERS
SERVER="191.168.2.11"				# IP of NAS, used for ftp
USERNAME="xxxxx"         				# FTP username of Network disk used for ftp
PASSWORD="xxxxxxxxx"    		    		# FTP password of Network disk used for ftp
DESTDIR="/tmp"		            		# used for temorarily storage
DESTDIRNAS="Backups/Domoticz" 		# Path to your NAS backup folder
DOMO_IP="192.168.2.26"   			# Domoticz IP 
DOMO_PORT="8086"         				# Domoticz port
DOMO_PATH="/home/pi/domoticz" 		# Path to your Domoticz folder on raspberry 

TIMESTAMP=`/bin/date +%d-%m-%Y`

BACKUPFILE="domoticz_$TIMESTAMP.db"  # backups will be named "domoticz_YYYYMMDDHHMMSS.db.gz"
BACKUPFILEGZ="$BACKUPFILE".gz
### END OF USER CONFIGURABLE PARAMETERS

### First remove old backups
sudo /bin/rm $DOMO_PATH/backups/*
### Create backup and ZIP it
sudo /usr/bin/curl -s http://$DOMO_IP:$DOMO_PORT/backupdatabase.php > $DESTDIR/$BACKUPFILE
sudo gzip -1 $DESTDIR/$BACKUPFILE
sudo tar -zcvf $DESTDIR/domoticz_$TIMESTAMP.tar.gz $DOMO_PATH
	
### Send to Network disk through FTP
sudo curl -u "$USERNAME:$PASSWORD" --ftp-create-dirs "ftp://192.168.2.11/$DESTDIRNAS/$TIMESTAMP/" 
sudo curl -s --disable-epsv -v -T "$DESTDIR/$BACKUPFILEGZ" -u "$USERNAME:$PASSWORD" "ftp://192.168.2.11/$DESTDIRNAS/$TIMESTAMP/"
sudo curl -s --disable-epsv -v -T "$DESTDIR/domoticz_$TIMESTAMP.tar.gz" -u "$USERNAME:$PASSWORD" "ftp://192.168.2.11/$DESTDIRNAS/$TIMESTAMP/"

### Remove temp backup file
sudo /bin/rm $DESTDIR/$BACKUPFILEGZ
sudo /bin/rm $DESTDIR/domoticz_$TIMESTAMP.tar.gz

## --------------------------------------------------------------------------------------------------- ##

# get a list of files and dates from ftp and remove files older than ndays
ftpsite="sftp -b-  -oPort=22  $USERNAME@$SERVER"
putdir=$DESTDIRNAS

ndays=14

# work out our cutoff date
MM=`date --date="$ndays days ago" +%b`
DD=`date --date="$ndays days ago" +%d`
TT=`date --date="$ndays days ago" +%s`

echo removing files older than $MM $DD

# get directory listing from remote source
echo "
cd $putdir
ls -l
"|$ftpsite >dirlist

# skip first three and last line, ftp command echo
listing="`tail -n+4 dirlist|head -n-1`"

lista=( $listing )

# loop over our files
for ((FNO=0; FNO<${#lista[@]}; FNO+=9));do
  # month (element 5), day (element 6) and filename (element 8)
  # echo Date ${lista[`expr $FNO+5`]} ${lista[`expr $FNO+6`]}          File: ${lista[`expr $FNO+8`]}

  fdate="${lista[`expr $FNO+5`]} ${lista[`expr $FNO+6`]} ${lista[`expr $FNO+7`]}"
  sdate=`date --date="$fdate" +%s`
  # check the date stamp
  if [ $sdate -lt $TT ]
  then
      # Remove this file
      echo "$MM $DD: Removing  ${lista[`expr $FNO+5`]} /  ${lista[`expr $FNO+6`]} / ${lista[`expr $FNO+8`]}"
      $ftpsite <<EOMYF2
      cd $putdir
      rm ${lista[`expr $FNO+8`]}
      quit
EOMYF2

  fi
done

The script gets stuck on the next part.

Code: Select all

# get directory listing from remote source
echo"
cd $putdir
ls -l
"|$ftpsite > directory

When I run the script on the raspberry it gets stuck as shown below. No error message.

Code: Select all

* We are completely uploaded and fine
* Remembering we are in dir "Backups/Domoticz/07-12-2022/"
< 226 Transfer complete
* Connection #0 to host 192.168.2.11 left intact
removing files older than nov 23

Any help is very welcome.
Perhaps there is an easier way to delete older content folders than I highly recommend.

[waltervl]

Code: Select all

# get directory listing from remote source
echo "
cd $putdir
ls -l
"|$ftpsite >dirlist

This piece of code makes a file called 'dirlist' in the $putdir ( = Backups/Domoticz). Do you see that file? Does it has contents?

[NvBgm55]

No i do not have a file like that in my backup directory.
But i find the variable $ftpsite nowhere in the scrip.
Maby thats the problem ?

[waltervl]

In the original script it says:

Code: Select all

ftpsite="sftp -b-  -oPort=22  $USERNAME@$SERVER"

[willemd]

Why not use the standard automatic backup of domoticz via the settings? Or do you want more than just the database?
The standard method creates hourly, daily and monhtly backups and recycles them automatically.
If you need to keep those files longer, then you can use a simple script to copy those to another folder or system.

[NvBgm55]

That might be a solution. My main goal is to have the data available for a longer period of time, preferably in an sql database. I hear you say Influxdb, but that has a number of limitations for me. I would prefer to transfer the data from different sensors and meters every day from the domoticz database to another database on my NAS.

[waltervl]

You can also use HttpLink for this. HttpLink uses the a http stream to push data from Domoticz devices to a server accepting http requests (GET, POST, PUT). That can be any data source (including influxdb). See wiki
https://www.domoticz.com/wiki/HttpLink

[waltervl]

Just for your information: Domoticz uses SQLite as it's database. All data stays in the database except the detailed sensor data per 5 minutes or hour is at midnight merged to basic daily average data and placed in a different table (xxx_calendar). From these tables no data is removed. So everything can be reported from the past.

For the report function see https://www.domoticz.com/wiki/Managing_ ... ice_Report