Different Backups from Crontab to network

All kinds of 'OS' scripts

Moderator: leecollings

User avatar
jvdz
Posts: 2330
Joined: Tuesday 30 December 2014 19:25
Target OS: Raspberry Pi / ODroid
Domoticz version: 4.107
Location: Netherlands
Contact:

Re: Different Backups from Crontab to network

Post by jvdz »

Shouldn't that be:

Code: Select all

sudo sh domoticz_backup.sh
Jos
New Garbage collection scripts: https://github.com/jvanderzande/GarbageCalendar
nigels0
Posts: 221
Joined: Thursday 23 January 2014 12:43
Target OS: Raspberry Pi / ODroid
Domoticz version: 3.8153
Contact:

Re: Different Backups from Crontab to network

Post by nigels0 »

or...

sudo ./domoticz_backup.sh
Derik
Posts: 1602
Joined: Friday 18 October 2013 23:33
Target OS: Raspberry Pi / ODroid
Domoticz version: BETA
Location: Arnhem/Nijmegen Nederland
Contact:

Re: Different Backups from Crontab to network

Post by Derik »

is this standing in the upper leftcorner: #!/bin/bash
Is the document pi and not root?
Is the document 0777 ?
Xu4: Beta Extreme antenna RFXcomE,WU Fi Ping ip P1 Gen5 PVOutput Harmony HUE SolarmanPv OTG Winddelen Alive ESP Buienradar MySensors WOL Winddelen counting RPi: Beta SMAspot RFlinkTest Domoticz ...Different backups
assenzuid
Posts: 135
Joined: Friday 13 November 2015 9:11
Target OS: Raspberry Pi / ODroid
Domoticz version: Beta
Location: The Netherlands, Emmen Area
Contact:

Re: Different Backups from Crontab to network

Post by assenzuid »

I have running a daily backup script, but I'm looking/ searching for how to implement a notification trigger after a successfully backup went complete by email.

Someone some ideas or already implemented?

Thanks.
woody4165
Posts: 476
Joined: Monday 14 March 2016 13:55
Target OS: Linux
Domoticz version: beta
Location: Rome, Italy
Contact:

Re: Different Backups from Crontab to network

Post by woody4165 »

Very useful! Thanks.
I got it work.
What I would like to do is to remove previous backed files, let's say removing files older that a week before.
I haven't found a way to do it on ftp (my Nas).
I tried mdelete and other command found on internet, but none worked.
Any suggestion on this?


What would be nice is also to execute it via a switch.
Is there a way to execute a shell script from a lua script?

Thanks
Cubietruck - Linux cubietruck 4.13.16 (Debian GNU/Linux 8 (jessie)) + Domoticz + RFLink, Xiaomi Gateway, Owl USB, Yeelight Color and B/W, ESP8266, Broadlink RM2, Netatmo Thermostat
sebitop
Posts: 69
Joined: Sunday 08 June 2014 20:31
Target OS: Raspberry Pi / ODroid
Domoticz version:
Location: Toulouse France
Contact:

Re: Different Backups from Crontab to network

Post by sebitop »

Hi guys

I think I missed something... maybe someone can help me :

if I do a Save Database via domoticz just clicking the button I have a domoticz.db file created localy on my computer, this file is 2 MB

if I use the script (without zipping it as I have space on my server) I have a domoticz_[date].db but this file is 91kb

if i try to restore the database using one of these files generated the log says : Seems this is not our database, or it is corrupted!

So I think i missed something could someone help me ??

thank you for your help !!

cheers
domoticz on Raspberry Pi + RFXcom LAN moded + wifi
Blyss / DI.O / homemade / ESP / wemos
Derik
Posts: 1602
Joined: Friday 18 October 2013 23:33
Target OS: Raspberry Pi / ODroid
Domoticz version: BETA
Location: Arnhem/Nijmegen Nederland
Contact:

Re: Different Backups from Crontab to network

Post by Derik »

@ Sebitop..
What is the problem to make a backup from the whole domoticz map... you got all your settings saved
And when you have a very small file there is something wrong.
Xu4: Beta Extreme antenna RFXcomE,WU Fi Ping ip P1 Gen5 PVOutput Harmony HUE SolarmanPv OTG Winddelen Alive ESP Buienradar MySensors WOL Winddelen counting RPi: Beta SMAspot RFlinkTest Domoticz ...Different backups
woody4165
Posts: 476
Joined: Monday 14 March 2016 13:55
Target OS: Linux
Domoticz version: beta
Location: Rome, Italy
Contact:

Re: Different Backups from Crontab to network

Post by woody4165 »

@sebitop
I have just tried to restore the backup that was done automatically this night from the script
My domoticz.db is around 630Kb and trying to do it also from the domoticz web the dimension is the same.
But I suggest to backup, like it's done in the script, also the whole domoticz folder tree since if you have modified any files in that structure will not be saved in the domoticz.db backup. I know that it takes something around 20MB, but it's worthing.

What I'm trying to find is a way to delete previous backup saved on my ftp server (Nas), but I haven't found a way yet.
Cubietruck - Linux cubietruck 4.13.16 (Debian GNU/Linux 8 (jessie)) + Domoticz + RFLink, Xiaomi Gateway, Owl USB, Yeelight Color and B/W, ESP8266, Broadlink RM2, Netatmo Thermostat
woody4165
Posts: 476
Joined: Monday 14 March 2016 13:55
Target OS: Linux
Domoticz version: beta
Location: Rome, Italy
Contact:

Re: Different Backups from Crontab to network

Post by woody4165 »

I just solved my own request.

To remove older backup files using the script already published, I added those lines.

In the PARAMETERS section just add:

Code: Select all

    
REMOVETIMESTAMP=`/bin/date --date='2 day ago' +%Y%m%d`
REMOVEFILE="domoticz_$REMOVETIMESTAMP"
choosing how many days back you want the backup files to be deleted.

At the end of the script add:

Code: Select all

    
    ## Now remove old backup
    for i in `curl -s -l ftp://"$USERNAME":"$PASSWORD"@$SERVER/$DESTDIR/ | grep $REMOVEFILE` ;
    do
       curl ftp://${SERVER}/${DESTDIR}/${i} -u "${USERNAME}:${PASSWORD}" -O --quote "DELE ${DESTDIR}${i}"
    done
 
And that's all!

I'm not a "coder", but this is what I found going around and around.
Hope can help someone, like helped myself...
Cubietruck - Linux cubietruck 4.13.16 (Debian GNU/Linux 8 (jessie)) + Domoticz + RFLink, Xiaomi Gateway, Owl USB, Yeelight Color and B/W, ESP8266, Broadlink RM2, Netatmo Thermostat
sebitop
Posts: 69
Joined: Sunday 08 June 2014 20:31
Target OS: Raspberry Pi / ODroid
Domoticz version:
Location: Toulouse France
Contact:

Re: Different Backups from Crontab to network

Post by sebitop »

Hi Woody

I understand what you mean but it dosen't solve my problem

I don't understand why localy my .db is 2mb when using the script the .db is 91kb and impossible to restore ...
domoticz on Raspberry Pi + RFXcom LAN moded + wifi
Blyss / DI.O / homemade / ESP / wemos
woody4165
Posts: 476
Joined: Monday 14 March 2016 13:55
Target OS: Linux
Domoticz version: beta
Location: Rome, Italy
Contact:

Re: Different Backups from Crontab to network

Post by woody4165 »

Hi Sebitop

how big is the file if you call it directly from the browser like http://domoticz_ip:domoticz_port/backupdatabase.php
Cubietruck - Linux cubietruck 4.13.16 (Debian GNU/Linux 8 (jessie)) + Domoticz + RFLink, Xiaomi Gateway, Owl USB, Yeelight Color and B/W, ESP8266, Broadlink RM2, Netatmo Thermostat
sebitop
Posts: 69
Joined: Sunday 08 June 2014 20:31
Target OS: Raspberry Pi / ODroid
Domoticz version:
Location: Toulouse France
Contact:

Re: Different Backups from Crontab to network

Post by sebitop »

2 MB this is the point !

if I directly ave the db from the browser, the file is 2 MB and I can restore it.

if I simply use : /usr/bin/curl http://192.168.0.30:8080/backupdatabase.php > /tmp/test.db

test.db is 91kb .... and impossible to restore it
domoticz on Raspberry Pi + RFXcom LAN moded + wifi
Blyss / DI.O / homemade / ESP / wemos
woody4165
Posts: 476
Joined: Monday 14 March 2016 13:55
Target OS: Linux
Domoticz version: beta
Location: Rome, Italy
Contact:

Re: Different Backups from Crontab to network

Post by woody4165 »

this happens in the script as well if you launch the curl command from terminal?

what I would try is to write in another folder and another filename...

That's weird...
Cubietruck - Linux cubietruck 4.13.16 (Debian GNU/Linux 8 (jessie)) + Domoticz + RFLink, Xiaomi Gateway, Owl USB, Yeelight Color and B/W, ESP8266, Broadlink RM2, Netatmo Thermostat
sebitop
Posts: 69
Joined: Sunday 08 June 2014 20:31
Target OS: Raspberry Pi / ODroid
Domoticz version:
Location: Toulouse France
Contact:

Re: Different Backups from Crontab to network

Post by sebitop »

this is exactly what i type in a terminal : /usr/bin/curl http://192.168.0.30:8080/backupdatabase.php > /tmp/test.db

when i go into the tmp folder, the file is 91kb instead of 2 MB
domoticz on Raspberry Pi + RFXcom LAN moded + wifi
Blyss / DI.O / homemade / ESP / wemos
vikasjee
Posts: 1
Joined: Wednesday 10 June 2015 19:20
Target OS: Raspberry Pi / ODroid
Domoticz version:
Contact:

Re: Different Backups from Crontab to network

Post by vikasjee »

While taking a backup on windows 10 i am getting a backupdatabase.php instead of the .db file. Is anybody else getting same error? Can somebody help?
User avatar
Egregius
Posts: 2592
Joined: Thursday 09 April 2015 12:19
Target OS: Linux
Domoticz version: v2024.7
Location: Beitem, BE
Contact:

Re: Different Backups from Crontab to network

Post by Egregius »

That's probably because your Windows machine doesn't has PHP installed.
svanzon
Posts: 1
Joined: Sunday 22 January 2017 17:54
Target OS: Raspberry Pi / ODroid
Domoticz version:
Contact:

Re: Different Backups from Crontab to network

Post by svanzon »

I have the same problem as sebitop.

When I goto http://192.168.1.100:8090/backupdatabase.php (that are my Domoticz settings) I can safe a file of 450KB but when i use the automated script I get an empty DB file.

Any tips, used the script for the DB only and also tried the Synology one (get the DB and folders but DB is empty).
User avatar
K3rryBlue
Posts: 77
Joined: Tuesday 07 February 2017 22:13
Target OS: Raspberry Pi / ODroid
Domoticz version: V4.10xx
Location: Netherlands
Contact:

Re: Different Backups from Crontab to network

Post by K3rryBlue »

I backup 9 different folder and with the standard script it became a lot of seperate files dumped to my NAS.
So I decided to use the $timestamp to create a folder, to drop the 9 archives, by using the command:

Code: Select all

curl -u "$USERNAME:$PASSWORD" --ftp-create-dirs "ftp://$SERVER/$DESTDIRNAS/$TIMESTAMP/"

#Modified the drop location for the archives:
curl -s --disable-epsv -v -T"/tmp/kwhmeter.tar.gz" -u"$USERNAME:$PASSWORD" "ftp://$SERVER/$DESTDIRNAS/$TIMESTAMP/"
The daily backup is stored in seperate folders.
Domoticz V4.10717, RPi2 -> Linux raspberrypi 4.4.47-v7+ , RFXCOM - RFXtrx433 USB 433.92MHz ,RAZBERRY2 and OpenZWave USB Version: 1.4-2363-g1f10253-dirty
sschouten
Posts: 9
Joined: Thursday 23 April 2015 13:54
Target OS: Raspberry Pi / ODroid
Domoticz version:
Contact:

Re: Different Backups from Crontab to network

Post by sschouten »

Hello,

iám using this script to backup to my nas,
but i still get the bellow error. what sis going wrong ?

Code: Select all

#!/bin/bash
# LOCAL/FTP/SCP/MAIL PARAMETERS
SERVER="10.0.0.207" # IP of Synology NAS, used for ftp
USERNAME="domoticz-bk" # FTP username of Network disk used for ftp
PASSWORD="***********" # FTP password of Network disk used for ftp
DESTDIR="/opt/backup" # used for temorarily storage
DESTDIRNAS="/backups_lokaal/Domoticz-hoofddorp/" # Path to your Synology NAS backup folder
DOMO_IP="10.0.0.95" # Domoticz IP 
DOMO_PORT="8080" # Domoticz port 
### END OF USER CONFIGURABLE PARAMETERS
TIMESTAMP=`/bin/date +%Y%m%d%H%M%S`
BACKUPFILE="domoticz_$TIMESTAMP.db" # backups will be named "domoticz_YYYYMMDDHHMMSS.db.gz"
BACKUPFILEGZ="$BACKUPFILE".gz
### Create backup and ZIP it
/usr/bin/curl -s http://$DOMO_IP:$DOMO_PORT/backupdatabase.php > /tmp/$BACKUPFILE
gzip -9 /tmp/$BACKUPFILE
tar -zcvf /tmp/domoticz_scripts_$TIMESTAMP.tar.gz /home/pi/domoticz/scripts/
#tar -zcvf /tmp/telegram_scripts_$TIMESTAMP.tar.gz /home/pi/tg/scripts/
### Send to Network disk through FTP
curl -s --disable-epsv -v -T"/tmp/$BACKUPFILEGZ" -u"$USERNAME:$PASSWORD" "ftp://$SERVER/$DESTDIRNAS"
curl -s --disable-epsv -v -T"/tmp/domoticz_scripts_$TIMESTAMP.tar.gz" -u"$USERNAME:$PASSWORD" "ftp://$SERVER/$DESTDIRNAS"
#curl -s --disable-epsv -v -T"/tmp/telegram_scripts_$TIMESTAMP.tar.gz" -u"$USERNAME:$PASSWORD" "ftp://$SERVER/$DESTDIRNAS"
### Remove temp backup file
/bin/rm /tmp/$BACKUPFILEGZ
/bin/rm /tmp/domoticz_scripts_$TIMESTAMP.tar.gz
#/bin/rm /tmp/telegram_scripts_$TIMESTAMP.tar.gz
### Done!

Code: Select all

pi@Domoticz:~/domoticz/scripts $ sh domoticz_backup.sh
tar: Removing leading `/' from member names
tar: /home/pi/domoticz/scripts/\r: Cannot stat: No such file or directory
tar: Exiting with failure status due to previous errors
'!b.gzCan't open '/tmp/domoticz_20170619222329
curl: try 'curl --help' or 'curl --manual' for more information
* Illegal characters found in URL
* Closing connection -1
/bin/rm: cannot remove ‘/tmp/domoticz_20170619222329\r.db.gz\r\r’: No such file or directory
/bin/rm: cannot remove ‘/tmp/domoticz_scripts_20170619222329\r.tar.gz\r’: No such file or directory
Derik
Posts: 1602
Joined: Friday 18 October 2013 23:33
Target OS: Raspberry Pi / ODroid
Domoticz version: BETA
Location: Arnhem/Nijmegen Nederland
Contact:

Re: Different Backups from Crontab to network

Post by Derik »

mm

I do have a other strange problem..:
https://www.domoticz.com/forum/viewtopi ... =4&t=18015

So if there is someone that understand my strange error..
Hope he can help me..

Thanks
Xu4: Beta Extreme antenna RFXcomE,WU Fi Ping ip P1 Gen5 PVOutput Harmony HUE SolarmanPv OTG Winddelen Alive ESP Buienradar MySensors WOL Winddelen counting RPi: Beta SMAspot RFlinkTest Domoticz ...Different backups
Post Reply

Who is online

Users browsing this forum: Google [Bot] and 1 guest