Page 1 of 2
Script to backup to a Synology NAS
Posted: Tuesday 27 October 2020 20:56
by Johan1974
Hello I used the script
Code: Select all
#!/bin/bash
#
# 20201028 Modified to replace CR/LF (windows) to LF (Linux)
#
# LOCAL/FTP/SCP/MAIL PARAMETERS
#
SERVER="192.168.1.11" # IP of Synology NAS, used for ftp
USERNAME="Domoticz" # FTP username of Network disk used for ftp
PASSWORD="12345" # FTP password of Network disk used for ftp
DESTDIR=" /tmp/" # used for temorarily storage
DESTDIRNAS="/homes/domoticz/backup/" # Path to your Synology NAS backup folder
DOMO_IP="192.168.1.12" # Domoticz IP
DOMO_PORT="8080" # Domoticz port
### END OF USER CONFIGURABLE PARAMETERS
TIMESTAMP=`/bin/date +%Y%m%d%H%M%S`
BACKUPFILE="domoticz_$TIMESTAMP.db" # backups will be named "domoticz_YYYYMMDDHHMMSS.db.gz"
BACKUPFILEGZ="$BACKUPFILE".gz
#
### Create backup and ZIP it
#
/usr/bin/curl -s http://$DOMO_IP:$DOMO_PORT/backupdatabase.php > /tmp/$BACKUPFILE
gzip -9 /tmp/$BACKUPFILE
tar -zcvf /tmp/domoticz_scripts_$TIMESTAMP.tar.gz /home/pi/domoticz/scripts/
tar -zcvf /tmp/telegram_scripts_$TIMESTAMP.tar.gz /home/pi/tg/scripts/
#
### Send to Network disk through FTP
#
curl -s --disable-epsv -v -T"/tmp/$BACKUPFILEGZ" -u"$USERNAME:$PASSWORD" "ftp://$SERVER/$DESTDIRNAS"
curl -s --disable-epsv -v -T"/tmp/domoticz_scripts_$TIMESTAMP.tar.gz" -u"$USERNAME:$PASSWORD" "ftp://$SERVER/$DESTDIRNAS"
curl -s --disable-epsv -v -T"/tmp/telegram_scripts_$TIMESTAMP.tar.gz" -u"$USERNAME:$PASSWORD" "ftp://$SERVER/$DESTDIRNAS"
#
### Remove temp backup file
#
/bin/rm /tmp/$BACKUPFILEGZ
/bin/rm /tmp/domoticz_scripts_$TIMESTAMP.tar.gz
/bin/rm /tmp/telegram_scripts_$TIMESTAMP.tar.gz
#eoj
## Done!
I save this script at:
/home/pi/domoticz/scripts whit the name
domoticz_backup.sh
When I test the script with:
sudo /home/pi/domoticz/scripts/domoticz_backup.sh
I see the next error:
sudo: unable to execute /home/pi/domoticz/scripts/domoticz_backup.sh: No such file or directory
What is going wrong.??
Re: Script to backup to a Synology NAS
Posted: Tuesday 27 October 2020 21:23
by cyberclwn
Hi,
does "/bin/bash" exist?
And is
domoticz_backup.sh executable?
Code: Select all
ls -ls /home/pi/domoticz/scripts/domoticz_backup.sh
Check if there is a x in the line ...
rwx
if not, change it with
Code: Select all
chmod +x /home/pi/domoticz/scripts/domoticz_backup.sh
Re: Script to backup to a Synology NAS
Posted: Tuesday 27 October 2020 22:10
by Johan1974
Yes,
Code: Select all
844 -rwxr-xr-x 1 root root 863400 Jan 15 2017 /bin/bash
Code: Select all
4 -rwxr-xr-x 1 pi pi 1604 Oct 27 21:57 /home/pi/domoticz/scripts/domoticz_backup.sh
Re: Script to backup to a Synology NAS
Posted: Wednesday 28 October 2020 0:32
by waaren
Johan1974 wrote: ↑Tuesday 27 October 2020 22:10
Yes,
@Johan1974
How should we read your "Yes" ?
Is everything OK now or does it mean the settings were already as suggested but the command still does not work?
Re: Script to backup to a Synology NAS
Posted: Wednesday 28 October 2020 10:08
by Johan1974
Sorry, I'm not clear.
I have CMOD de file domoticz_backup to 777
When I use:
Code: Select all
ls -ls /home/pi/domoticz/scripts/domoticz_backup.sh
the answer is
Code: Select all
4 -rwxrwxrwx 1 pi pi 1609 Oct 28 09:50 /home/pi/domoticz/scripts/domoticz_backup.sh
When I use
Code: Select all
sudo sh /home/pi/domoticz/scripts/domoticz_backup.sh
the answer is
Code: Select all
tar: Removing leading `/' from member names
tar: /home/pi/domoticz/scripts/\r: Cannot stat: No such file or directory
tar: Exiting with failure status due to previous errors
tar: Removing leading `/' from member names
tar: /home/pi/tg/scripts/\r: Cannot stat: No such file or directory
tar: Exiting with failure status due to previous errors
'!b.gzCan't open '/tmp/domoticz_20201028100108
curl: try 'curl --help' or 'curl --manual' for more information
* Illegal characters found in URL
* Closing connection -1
* Illegal characters found in URL
* Closing connection -1
/bin/rm: cannot remove ‘/tmp/domoticz_20201028100108\r.db.gz\r\r’: No such file or directory
/bin/rm: cannot remove ‘/tmp/domoticz_scripts_20201028100108\r.tar.gz\r’: No such file or directory
/bin/rm: cannot remove ‘/tmp/telegram_scripts_20201028100108\r.tar.gz\r’: No such file or directory
The files
domoticz_20201028100108.db.gz and
telegram_scripts_20201028100108.tar.gz are made in the tmp directory on the raspberry. But not upload to the NAS
Re: Script to backup to a Synology NAS
Posted: Wednesday 28 October 2020 10:26
by waaren
Johan1974 wrote: ↑Wednesday 28 October 2020 10:08
tar: /home/pi/domoticz/scripts/\r: Cannot stat: No such file or directory
Did you edit the file with a windows editor ? (asking this because of the \r in your log)
Google for "windows editor unix line breaks"
Re: Script to backup to a Synology NAS
Posted: Wednesday 28 October 2020 10:58
by Johan1974
That's right, and I can remember making this mistake more than once ..

I started over from the beginning, with a new domoticz_backup.sh file. I make script executable with
Code: Select all
sudo chmod +x /home/pi/domoticz/scripts/domoticz_backup.sh
Then I run the script with:
Code: Select all
sudo /home/pi/domoticz/scripts/domoticz_backup.sh
the answer is:
Code: Select all
sudo: unable to execute /home/pi/domoticz/scripts/domoticz_backup.sh: No such file or directory
when I use
Code: Select all
sudo sh /home/pi/domoticz/scripts/domoticz_backup.sh
the answer is:
Code: Select all
tar: Removing leading `/' from member names
tar: /home/pi/domoticz/scripts/\r: Cannot stat: No such file or directory
tar: Exiting with failure status due to previous errors
tar: Removing leading `/' from member names
tar: /home/pi/tg/scripts/\r: Cannot stat: No such file or directory
tar: Exiting with failure status due to previous errors
'!b.gzCan't open '/tmp/domoticz_20201028104943
curl: try 'curl --help' or 'curl --manual' for more information
* Illegal characters found in URL
* Closing connection -1
* Illegal characters found in URL
* Closing connection -1
/bin/rm: cannot remove ‘/tmp/domoticz_20201028104943\r.db.gz\r\r’: No such file or directory
/bin/rm: cannot remove ‘/tmp/domoticz_scripts_20201028104943\r.tar.gz\r’: No such file or directory
/bin/rm: cannot remove ‘/tmp/telegram_scripts_20201028104943\r.tar.gz\r’: No such file or directory
I do something wrong:(
what is the difference between
sudo and
sudo sh
I read the manual on
https://www.domoticz.com/wiki/Daily_b ... ernal_hdd and follow the instructions
Re: Script to backup to a Synology NAS
Posted: Wednesday 28 October 2020 11:40
by waaren
sudo does use your default shell (probably bash ?) and sudo sh force the use of the sh shell
Re: Script to backup to a Synology NAS
Posted: Wednesday 28 October 2020 11:51
by Egregius
Why not use a script on your nas with rsync of the complete domoticz folder? Much easier.
If /tmp is on your SD card it'll ruin it.
If /tmp is in tmpfs you can hit memory limits.
Also, using rsync for the complete domoticz folder gives you the advantage of making versions possible and you'll have a backup of your scripts etc.
Re: Script to backup to a Synology NAS
Posted: Wednesday 28 October 2020 11:53
by Johan1974
Okay, so I use to run the script:
Code: Select all
sudo /home/pi/domoticz/scripts/domoticz_backup.sh
that is also what I read in the manual on the website.
but then nothing happens, only the following message:
Code: Select all
sudo: unable to execute /home/pi/domoticz/scripts/domoticz_backup.sh: No such file or directory
When asked by @cyberclwn whether bin / bash exists I get the answer :
Code: Select all
844 -rwxr-xr-x 1 root root 863400 Jan 15 2017 /bin/bash
Re: Script to backup to a Synology NAS
Posted: Wednesday 28 October 2020 12:34
by Johan1974
Egregius wrote: ↑Wednesday 28 October 2020 11:51
Why not use a script on your nas with rsync of the complete domoticz folder? Much easier.
If /tmp is on your SD card it'll ruin it.
If /tmp is in tmpfs you can hit memory limits.
Also, using rsync for the complete domoticz folder gives you the advantage of making versions possible and you'll have a backup of your scripts etc.
There are more roads that lead to Rome.

I think it is important to have a back up because I already have a lot of data in domoticz.
Can I find a manual of this script.?
I still hope to find a solution to my first problem
Re: Script to backup to a Synology NAS
Posted: Wednesday 28 October 2020 13:05
by waaren
Johan1974 wrote: ↑Wednesday 28 October 2020 12:34
I still hope to find a solution to my first problem
@Johan1974,
I modified your file in the opening post of this topic. It still contained CR/LF and with that the shell will not recognize the
#!/bin/bash (shebang) and does not know what to do with it.
Please copy and save it again and make sure no windows editor is touching it again.
And I fully agree with @Egregius. rsync is much better suited for the job.
Re: Script to backup to a Synology NAS
Posted: Wednesday 28 October 2020 14:20
by Johan1974
I started all over again and didn't use a windows editor this time.
And yes! It works now! It's very easy to use an editor, but I understand I have to do everything with Putty and ssh..
Thank you very much.
I am still interested in the script @Egregius uses. A total backup is always better.
Re: Script to backup to a Synology NAS
Posted: Wednesday 28 October 2020 14:30
by waaren
Johan1974 wrote: ↑Wednesday 28 October 2020 14:20
I started all over again and didn't use a windows editor this time.
And yes! It works now! It's very easy to use an editor, but I understand I have to do everything with Putty and ssh..
II use Notepad++ on my Windows desktop and -laptop and that editor is perfectly OK with Unix files and it is opensource.
Also not unimportant; it is completely free

Re: Script to backup to a Synology NAS
Posted: Wednesday 28 October 2020 16:08
by EddyG
I use rsnapshot to take locally snapshots of a few dirs of my Raspberry to a local mounted USB-stick (Sandisk 32GB)
The snapshot is taken every 4 Hours for the duration of 4 weeks, kind of son father grandfather backup.
Every 5 minutes I backup only the Domoticz database to my Synology NAS and keep 864 versions eg. 3 days. You could extend that.
The database is saved with date and time and database version in the name. Logging is done in /var/log
Below my script. sqlite3 and bc should be installed and Synology should support nfs.
Run it as root from cron every 5 minutes. On a Pi 3B+ is takes about 6-10 seconds.
All my Raspberry's are backuped this way hence the $HOSTNAME in the path.
Code: Select all
#!/usr/bin/env /bin/bash
SCRIPTNAME=`basename "$0"`
SCRIPTNAME=${SCRIPTNAME%.*}
LOGFILE=/var/log/$SCRIPTNAME.log
ERRORLOG=/var/log/$SCRIPTNAME_ERROR.log
HOSTNAME=$(hostname)
DOMO_IP="127.0.0.1" # Domoticz IP
DOMO_PORT="8080" # Domoticz port
REMOTEHOST="192.168.2.xxx"
REMOTEPATH="/volume1/RaspberryPi/Domoticz_DB/$HOSTNAME"
SOURCEFILE="domoticz.db"
SOURCEPATH="/home/pi/domoticz/"
MOUNTPOINT="/mnt/$SCRIPTNAME"
### END OF USER CONFIGURABLE PARAMETERS
# Start Timestamp
STARTTIME=$(date +"%s.%N")
# Place in the log file information concerning the execution of this script
echo -n "Domoticz database backup start at " >> $LOGFILE
echo $(date) >> $LOGFILE
if [ ! -d $MOUNTPOINT ]; then
sudo mkdir $MOUNTPOINT
fi
if ! mountpoint -q $MOUNTPOINT; then
sudo mount -t nfs $REMOTEHOST:$REMOTEPATH $MOUNTPOINT
fi
if ! /bin/mountpoint -q $MOUNTPOINT; then
touch $ERRORLOG
chmod 0666 $ERRORLOG
echo "Mount $MOUNTPOINT ERROR!!!" >> $ERRORLOG
exit 99
fi
#Delete backups older than 3 days = 864 (288 per day is saved every 5 minutes)
cd $MOUNTPOINT
ls -tr | head -n -864 | xargs --no-run-if-empty rm
### Create backup
TIMESTAMP=`/bin/date +%Y%m%d%H%M%S`
DB_VERSION=`sqlite3 $SOURCEPATH$SOURCEFILE "Select nValue from Preferences WHERE Key = 'DB_Version'"`
BACKUPFILE=$HOSTNAME"_"$TIMESTAMP"_"$DB_VERSION".db" # backups will be named "domoticz_YYYYMMDDHHMMSS_ZZZ.db"
/usr/bin/curl -s http://$DOMO_IP:$DOMO_PORT/backupdatabase.php > $MOUNTPOINT/$BACKUPFILE
sync
sudo /bin/umount -l $MOUNTPOINT
# End timestamp
ENDTIME=$(date +"%s.%N")
# Convert nanoseconds to milliseconds
# crudely by taking first 3 decimal places
TIMEDIFF=`echo "$ENDTIME - $STARTTIME" | bc | awk -F"." '{print $1"."substr($2,1,3)}'`
SEC=`echo "$TIMEDIFF % 60" | bc`
echo "Total of $SEC seconds elapsed." >> $LOGFILE
echo "++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++" >> $LOGFILE
### Done!
Re: Script to backup to a Synology NAS
Posted: Wednesday 28 October 2020 17:57
by Egregius
Johan1974 wrote: ↑Wednesday 28 October 2020 14:20
I am still interested in the script @Egregius uses. A total backup is always better.
This is executed at my nas (only part of the script copied because it backups multiple servers)
It'll create a versioned backup for every day it has run. Because of the --link-dest option only changed files occupy space. Files that didn't change are actually symlinks. Don't worry about that, linux takes care of that perfectly.
The great advantage of this is that you can go back in time to any version you want. And that you don't stress your raspberry. The NAS can do perfectmy what it does best: keeping backups.
Code: Select all
#!/bin/sh
NOW=$(date +"%Y-%m-%d")
BPATH=/volume1/homes/guy/backup
RUNLOG=/volume1/homes/guy/log/$NOW.txt
NAME="Domoticz"
echo ------------------- START $NAME -- $(date) | tee -a $RUNLOG
SOURCE="[email protected]:/domoticz/"
DESTINATION="$BPATH/$NAME/$NOW"
mkdir -p "$DESTINATION"
rsync -aP -e "ssh -i /root/.ssh/home" --stats --delete-after --links --ignore-errors --link-dest="../__prev/" "$SOURCE" "$DESTINATION" | tee -a $RUNLOG
rm -f "$BPATH/$NAME/__prev"
ln -s "$BPATH/$NAME/$NOW" "$BPATH/$NAME/__prev"
echo ------------------- END $NAME -- $(date) | tee -a $RUNLOG
Re: Script to backup to a Synology NAS
Posted: Wednesday 28 October 2020 18:47
by EddyG
Nice script working from the other side.
Just copying or rsyncing is a little bit dangerous on an open database.
A more save way is to use "backupdatabase.php" it has some provisions in the domoticz program for looking for busy and locks, and it Optimizes and Vacuums the database before a copy is made. The copy is made by the sqlite3 library.
Re: Script to backup to a Synology NAS
Posted: Thursday 29 October 2020 7:08
by Egregius
Never had an issue with it, restored multiple times. Works flawless.
Re: Script to backup to a Synology NAS
Posted: Thursday 29 October 2020 7:50
by waaren
Egregius wrote: ↑Thursday 29 October 2020 7:08
Never had an issue with it, restored multiple times. Works flawless.
Lucky you

And to be honest I do the same...
BUT
The SQLite developers say this:
How To Corrupt An SQLite Database File
1.2. Backup or restore while a transaction is active
Systems that run automatic backups in the background might try to make a backup copy of an SQLite database file while it is in the middle of a transaction. The backup copy then might contain some old and some new content, and thus be corrupt.
The best approach to make reliable backup copies of an SQLite database is to make use of the backup API that is part of the SQLite library. Failing that, it is safe to make a copy of an SQLite database file as long as there are no transactions in progress by any process. If the previous transaction failed, then it is important that any rollback journal (the *-journal file) or write-ahead log (the *-wal file) be copied together with the database file itself.
_____________________________________
So why take any chances? It is safer, quite easy and very quick to do something like
Code: Select all
sudo sqlite3 /opt/domoticz/domoticz/db '.backup myBackup.db'
and copy the myBackup to your target location.
Re: Script to backup to a Synology NAS
Posted: Thursday 12 November 2020 8:51
by EddyG
My script has to be changed. See here:
viewtopic.php?f=6&t=34253&p=260598#p260598
I my case (busy system) sometimes the methode of "backupdatabase.php" leads to a system freeze or halt.
My problem is solved by change that line in
Code: Select all
/usr/bin/sqlite3 $SOURCEPATH$SOURCEFILE ".backup $MOUNTPOINT/$BACKUPFILE"