Database backup with backupdatabase.php gives 10 s limit errors

On various Hardware and OS systems: pi / windows / routers / nas, etc

Moderator: leecollings

Post Reply
mgugu
Posts: 220
Joined: Friday 04 November 2016 12:33
Target OS: Raspberry Pi / ODroid
Domoticz version: Beta
Location: France
Contact:

Database backup with backupdatabase.php gives 10 s limit errors

Post by mgugu »

Hi All,
For years I have been using a CRON bash script to backup my database on a NAS on a daily basis. This script uses the built-in "backupdatabase.php".
But now I get errors about reaching the 10 s limit, probably due to the database size which has increased (> 10M).
Instead of using backupdatabase.php it would be possible to make a direct copy of the database or even rsyncing the whole domoticz folder.
But my question is: Is it safe to make such copies while domoticz is in use ?
User avatar
waltervl
Posts: 5904
Joined: Monday 28 January 2019 18:48
Target OS: Linux
Domoticz version: 2024.7
Location: NL
Contact:

Re: Database backup with backupdatabase.php gives 10 s limit errors

Post by waltervl »

I believe 10MB database is not that big, seems to be normal, mine is 16 MB. So probably look what is going on else, network, slow CPU, memory?
Is the automatic backup also working? Because then you could copy the domoticz/backups/daily folder daily. When you do that 10 min after the whole hour you are not conflicting with the automatic backup schedule that runs on the whole hour exactly.
Domoticz running on Udoo X86 (on Ubuntu)
Devices/plugins: ZigbeeforDomoticz (with Xiaomi, Ikea, Tuya devices), Nefit Easy, Midea Airco, Omnik Solar, Goodwe Solar
mgugu
Posts: 220
Joined: Friday 04 November 2016 12:33
Target OS: Raspberry Pi / ODroid
Domoticz version: Beta
Location: France
Contact:

Re: Database backup with backupdatabase.php gives 10 s limit errors

Post by mgugu »

waltervl wrote: Saturday 02 July 2022 15:12 I believe 10MB database is not that big, seems to be normal, mine is 16 MB. So probably look what is going on else, network, slow CPU, memory?
Is the automatic backup also working? Because then you could copy the domoticz/backups/daily folder daily. When you do that 10 min after the whole hour you are not conflicting with the automatic backup schedule that runs on the whole hour exactly.
Same story with automatic backup so I really think this is due to baclupdatabase.php which takes too much time.
You are right, the slow CPU (Pi 2B that I use for domoticz since 2014) is probably the cause.
Changing to Pi 4 is impossible at the moment, that is why I am looking for a workaround.
User avatar
waltervl
Posts: 5904
Joined: Monday 28 January 2019 18:48
Target OS: Linux
Domoticz version: 2024.7
Location: NL
Contact:

Re: Database backup with backupdatabase.php gives 10 s limit errors

Post by waltervl »

Then check the CPU load of your Pi if it is doing something else besides Domoticz. Also check your SD card, if writing is slow it could fail too.
Domoticz running on Udoo X86 (on Ubuntu)
Devices/plugins: ZigbeeforDomoticz (with Xiaomi, Ikea, Tuya devices), Nefit Easy, Midea Airco, Omnik Solar, Goodwe Solar
mgugu
Posts: 220
Joined: Friday 04 November 2016 12:33
Target OS: Raspberry Pi / ODroid
Domoticz version: Beta
Location: France
Contact:

Re: Database backup with backupdatabase.php gives 10 s limit errors

Post by mgugu »

waltervl wrote: Saturday 02 July 2022 21:26 Then check the CPU load of your Pi if it is doing something else besides Domoticz. Also check your SD card, if writing is slow it could fail too.
Everything is OK.
Solved that issue by reducing the database size.
I discovered in *_calendar tables, a huge amount of expired values (date up to 2015).
After deleting values < 2021-01-01 in *_calendar tables and vacuuming the database, the size decreased from 13 M to 5 M.
Post Reply

Who is online

Users browsing this forum: No registered users and 1 guest