Page 2 of 2
Re: 2024.7 stops working every few days
Posted: Friday 23 August 2024 18:02
by BartSr
domoticz:
2024.7 built 16178
Raspberry Pi 3b+ bullseye
crash at night several times this week
ping device is oke
only solution : restart service
then next error:
2024-08-23 17:04:11.683 Error: Raspberry hardware (12) thread seems to have ended unexpectedly
has this todo with crash at night?
how to?
Re: 2024.7 stops working every few days
Posted: Saturday 24 August 2024 6:14
by Kedi
Hiu80 wrote: ↑Tuesday 20 August 2024 14:52
Updated yesterday to 16182, crashed again today. Automatic back-up is disabled. Attached the crash log, but i do not know if there is any useable information in there.
Your problem comes probably from the GoodWe plugin.
Disable the plugin for some days and see if that solves the problem.
If so contact the plugin creator.
Re: 2024.7 stops working every few days
Posted: Sunday 25 August 2024 11:28
by BartSr
Code: Select all
Domoticz Version: 2024.7 (build 16187)
Build Hash: 6b9cbb21c
Compile Date: 2024-08-22 08:08:34
dzVents Version: 3.1.8
Python Version: 3.9.2 (default, Feb 28 2021, 17:03:44) [GCC 10.2.1 20210110]
Raspberry Pi 3b+ SDcard 32Gb
Further to regular stops last week:
I found that somewhere files are growing.
Yesterday I noticed SD card 32Gb 100% full causing failures.
Then restored an image. 32Gb SD card now 65%.
Today already 76% Database is steady around 60Mb.
installed on SD card:
Bullseye OS
docker compose for zwave2mqtt and zigbee2mqtt
native (no docker) domoticz install
native (no docker) mosquitto
wireguard
Any suggestion where to look for growing files?
I used command
sudo du -sm /* 2>/dev/null | sort -V
from that I learnt that /VAR is growing
is there a linux command which lists all files into a textfilw sorted on size?
Re: 2024.7 stops working every few days
Posted: Sunday 25 August 2024 13:22
by Kedi
Try:
Code: Select all
sudo du -ah /var | sort -rh | head -10 > textfile.txt
Re: 2024.7 stops working every few days
Posted: Sunday 25 August 2024 18:46
by BartSr
@ Kedi
thanks a lot. by changing /var to
/var/lib
/var/lib/docker I can find the 'growing file'
It looks like this file (xxx/yyy-json.log) is the one growing fast (ca.11%/day) :
Code: Select all
5,1G /var/lib/docker
4,2G /var/lib/docker/containers
4,1G /var/lib/docker/containers/896db812b0572298a3a703e65079aaa647a407b657d152772feb6a66d01da6c7/896db812b0572298a3a703e65079aaa647a407b657d152772feb6a66d01da6c7-json.log
4,1G /var/lib/docker/containers/896db812b0572298a3a703e65079aaa647a407b657d152772feb6a66d01da6c7
is there a way to find out which logfile it is?
zwave2mqtt and zigbee2mqtt are the only ones running in docker.
Re: 2024.7 stops working every few days
Posted: Sunday 25 August 2024 19:09
by Kedi
If you do a
You will get an overview, and the first part 'CONTAINER ID' is the first part of your log filename, eg. 896db812b0......
Re: 2024.7 stops working every few days
Posted: Sunday 25 August 2024 22:48
by BartSr
thanks a lot.
I found it's the zwave container.
will it be save to remove the logfile?
meanwhile I found that in zwavejs debug was set to on and results to store in a file so I presume it's that huge file.
I will report the results.
Bart
Re: 2024.7 stops working every few days
Posted: Monday 26 August 2024 6:49
by Kedi
It is simple.
1. Do a 'sudo su'
2. Stop the container
3. Remove the log file(s)
4. Start the container.
5. Exit the 'su'
A good manual on docker log files can be found here:
https://signoz.io/guides/docker-clear-logs/
Re: 2024.7 stops working every few days
Posted: Monday 26 August 2024 14:29
by BartSr
@Kedi
Thanks for your help.
Once I stopped the containers and restarted them it appeared there was no longer a log file. (I set log in ZwaveUI to off).
HD space used 48%. Before stopping both containers HD space used 78% SD card 32Gb.
Lessons learnt: only set log to on if there's a specific reason otherwise this may cause crash of domoticz due to less HD space.
Before I used SD card 16Gb but changed into 32Gb because of too less space left on HD.
Maybe it's a good idea to have notification once HD space used > 80%
-Bart
Re: 2024.7 stops working every few days
Posted: Tuesday 27 August 2024 23:59
by FlyingDomotic
BartSr wrote: ↑Monday 26 August 2024 14:29
Maybe it's a good idea to have notification once HD space used > 80%
It's not a good idea, it's an excellent idea!
Go to "Hardware" tab, and add a "Motherboard sensor" device. You'll have new devices with CPU/Memory/Disk space. It may then be judicious to make some notifications on them;-)
Re: 2024.7 stops working every few days
Posted: Friday 30 August 2024 15:40
by Hiu80
Kedi wrote: ↑Saturday 24 August 2024 6:14
Hiu80 wrote: ↑Tuesday 20 August 2024 14:52
Updated yesterday to 16182, crashed again today. Automatic back-up is disabled. Attached the crash log, but i do not know if there is any useable information in there.
Your problem comes probably from the GoodWe plugin.
Disable the plugin for some days and see if that solves the problem.
If so contact the plugin creator.
Thanks for the heads up. Some time ago i updated the plugin, but somehow used an older version of it. It is now properly updated, and will check if is still crashes.
Re: 2024.7 stops working every few days
Posted: Friday 06 September 2024 20:58
by microkid
Any update on this issue? Domoticz is crashing almost every day at random times. Still nothing in the log files.
Re: 2024.7 stops working every few days
Posted: Friday 06 September 2024 23:39
by waltervl
microkid wrote: ↑Friday 06 September 2024 20:58
Any update on this issue? Domoticz is crashing almost every day at random times. Still nothing in the log files.
Did you update to latest beta? There were some changes to at least show more logging before crashing. Also reports of no crashing. But all of this is readable in this topic....
Re: 2024.7 stops working every few days
Posted: Wednesday 11 September 2024 22:23
by SESTH
BartSr wrote: ↑Sunday 25 August 2024 22:48
thanks a lot.
I found it's the zwave container.
will it be save to remove the logfile?
meanwhile I found that in zwavejs debug was set to on and results to store in a file so I presume it's that huge file.
I will report the results.
Bart
Log rotation should help.
Re: 2024.7 stops working every few days
Posted: Sunday 15 September 2024 22:02
by BartSr
@sesth
thanks for pointing me to this brilliant explanation re the logfiles.
Re: 2024.7 stops working every few days
Posted: Wednesday 18 September 2024 20:12
by Rik60
Hi all,
I have a problem with Domoticz that it is crashing one/ two times a week. The only thing changed was a upgrade to version 2024.7 (i updated from 2024.6 to 2024.7 when it was released). Domoticz is running at a pi3B. The raspberry is running normal. A Domoticz service restart is enough to have Domoticz online again. Attached is a Domoticz_crash_log file. I can't find why Domoticz is crashing. Thanks for your help.
Re: 2024.7 stops working every few days
Posted: Wednesday 18 September 2024 20:53
by FlyingDomotic
Gulty could be:
Code: Select all
Thread 33 (Thread 0x6e2ff200 (LWP 13719) "Zigbee"):
#0 0x768e2824 in __GI___wait4 (pid=23406, stat_loc=0x6e2fc578, options=0, usage=0x0) at ../sysdeps/unix/sysv/linux/wait4.c:27
#1 0x00689f72 in dumpstack_gdb(bool) ()
#2 0x0068a3c8 in signal_handler(int, siginfo_t*, void*) ()
#3 <signal handler called>
#4 raise (sig=11) at ../sysdeps/unix/sysv/linux/raise.c:50
#5 <signal handler called>
#6 0x759fe1b4 in ?? () from /lib/arm-linux-gnueabihf/libpython3.9.so
#7 0x759fe2d4 in ?? () from /lib/arm-linux-gnueabihf/libpython3.9.so
Backtrace stopped: previous frame identical to this frame (corrupt stack?)
Signal 11 (SIGSEGV, segmentation violation) means that the program accessed a memory location that was not assigned to it. This may be a bug...
Re: 2024.7 stops working every few days
Posted: Friday 20 September 2024 9:58
by Rik60
Scrolling back in the history, maybe the Reolink plugin is the problem. I installed this plugin in june, almost the sametime i upgrade to 2024.7. Sinds then Domoticz was reacting strange, sometimes blockly events were party working and errors in the logging. So i have removed the Reolink plugin and will see what happens.