Script to restart Domoticz if it crashes Topic is solved

All kinds of 'OS' scripts

Moderator: leecollings

Toulon7559
Posts: 848
Joined: Sunday 23 February 2014 17:56
Target OS: Raspberry Pi / ODroid
Domoticz version: mixed
Location: Hengelo(Ov)/NL
Contact:

Re: Script to restart Domoticz if it crashes

Post by Toulon7559 »

@Egregius

Because of rather frequent crashes of one Domoticz_at_Raspberry, with interest looked at your 'crash-check-script'.
Problem I have that most of the time 'my' crashes are not 100%, but only for certain applications (similar to problem of HFman): see this thread_at_other_forum.
In one of the previous messages you presented a nice php-script for that aspect, but I prefer running as few scripts of different types as possible:
would it be possible to add some lines for application_check in your 'crash-check-script''?

On the other hand, perhaps a periodic check on failing application(s) is 'luxury', and a brutely forced restart or reboot by a timer e.g. every 6 hours might be effective as well.
Only preferable precaution for such timed, forced restart or reboot seems that it should not simply happen at the full hour, but at some 'odd' time such as 11 minutes after the full hour, to prevent conflicts with timed scriptrunning for backups, uploads, downloads, etc.
Last edited by Toulon7559 on Wednesday 30 August 2017 23:03, edited 1 time in total.
Set1 = RPI-Zero+RFXCom433+S0PCM+Shield for BMP180/DS18B20/RS485+DDS238-1ZNs
Set2 = RPI-3A++RFLinkGTW+ESP8266s+PWS_WS7000
Common = KAKUs+3*PVLogger+PWS_TFA_Nexus
plus series of 'satellites' for dedicated interfacing, monitoring & control.
User avatar
Egregius
Posts: 2589
Joined: Thursday 09 April 2015 12:19
Target OS: Linux
Domoticz version: v2024.7
Location: Beitem, BE
Contact:

Re: Script to restart Domoticz if it crashes

Post by Egregius »

Well... Hmmm...
You could do a periodic reboot from cron, for example, this would reboot daily at 4:11AM (when you probably aren't bothered with it):

Code: Select all

11 4 * * * sudo reboot
I use PHP for all stuff, only bash for some very basic things. PHP is very easy and versatily to handly any sort of data. Coming from Mysql, sqlite, xml, json, you name it, PHP can handle it.
If only part of the systems hangs I think I would add the devices that stop to a floorplan. Query their status in PHP, loop thru those and remember the last update time.
If the last update time is more than x minutes ago do something.
Of course, this would only work reliable if the devices report frequently.
Toulon7559
Posts: 848
Joined: Sunday 23 February 2014 17:56
Target OS: Raspberry Pi / ODroid
Domoticz version: mixed
Location: Hengelo(Ov)/NL
Contact:

Re: Script to restart Domoticz if it crashes

Post by Toulon7559 »

@Egregius

;-) ;-) Perhaps a challenge ....
Further analyzing the 'misbehaviours' in my system, which can be solved by reboot, I detect that one specific lua-script running at Raspberry3 randomly stops somewhere after 8 to 14 hours of running.
[Unfortunately very visible because it handles an upload to WUnderground. And strange, because the system load is insignificant for an upload each 20 minutes, and a similar lua-script on a RaspberryB runs without problems]
;-) Is in your inventory of clever scripts somewhere a remedy to catch&restart such script?

Not a beauty-solution, but my own (first) idea is application of an additional semaphore set by the suspected script: outside the script in Domoticz visible as a Dummy Switch. If the status (= validity time) of the Dummy is not updated within x minutes, then time to take measures.
Any better idea is welcome .......
Set1 = RPI-Zero+RFXCom433+S0PCM+Shield for BMP180/DS18B20/RS485+DDS238-1ZNs
Set2 = RPI-3A++RFLinkGTW+ESP8266s+PWS_WS7000
Common = KAKUs+3*PVLogger+PWS_TFA_Nexus
plus series of 'satellites' for dedicated interfacing, monitoring & control.
User avatar
Egregius
Posts: 2589
Joined: Thursday 09 April 2015 12:19
Target OS: Linux
Domoticz version: v2024.7
Location: Beitem, BE
Contact:

Re: Script to restart Domoticz if it crashes

Post by Egregius »

I don't use lua, can't help you with that. And why does a script has to run for so long? Is it in a loop or something? Why not break that loop after 1 minute and trigger the script to run by cron. That way you're sure the script starts every minute.
I could create a php script that checks the lastupdate of one or several devices and act upon that, but you don't want to use php...
Toulon7559
Posts: 848
Joined: Sunday 23 February 2014 17:56
Target OS: Raspberry Pi / ODroid
Domoticz version: mixed
Location: Hengelo(Ov)/NL
Contact:

Re: Script to restart Domoticz if it crashes

Post by Toulon7559 »

@Egregius

PHP is for me still in the learning curve (=> my own 'inventions' are still rather basic), and therefore any example is useful (and better starting point) as basis for adaptation.
;-) Quite on purpose I put in the previous message "Is in your inventory of clever scripts"....? ", because I do not discriminate on type of available scripts, whether PHP, Python or lua, or ........

The upload of weather-data to WUnderground is on purpose running 24/7, because the need for continuous registration. Interval is also quite on purpose at 20 minutes, because otherwise you exceed the max. number of uploads allowed per 24 hours: weather is (usually) not changing that fast that higher rate is meaningful.
Obviously, then it is a pain-in-the-ass if the upload is unexpectedly failing.
Last edited by Toulon7559 on Sunday 19 November 2017 20:10, edited 1 time in total.
Set1 = RPI-Zero+RFXCom433+S0PCM+Shield for BMP180/DS18B20/RS485+DDS238-1ZNs
Set2 = RPI-3A++RFLinkGTW+ESP8266s+PWS_WS7000
Common = KAKUs+3*PVLogger+PWS_TFA_Nexus
plus series of 'satellites' for dedicated interfacing, monitoring & control.
User avatar
Egregius
Posts: 2589
Joined: Thursday 09 April 2015 12:19
Target OS: Linux
Domoticz version: v2024.7
Location: Beitem, BE
Contact:

Re: Script to restart Domoticz if it crashes

Post by Egregius »

I wouldn't run a script in loop if it only needs to do something every 20 minutes.
Set it by cron
0,20,40 * * * * /path/to/script

If you want to check lastupdate of devices, this script checks devices of roomplan 2 and remembers the last update time. If lastupdatetime is more than 3600 seconds ago do something:

Code: Select all

<?php
$lastupdate=0;
$domoticz=json_decode(file_get_contents('http://127.0.0.1:8080/json.htm?type=devices&filter=all&used=true&plan=2'),true);
if(isset($domoticz['result'])){
	foreach($domoticz['result'] as $dom){
		$name=$dom['Name'];
		$update=strtotime($dom['LastUpdate']);
		if($update>$lastupdate)$lastupdate=$update;
	}
}
if($lastupdate>0&&$lastupdate<$_SERVER['REQUEST_TIME']-3600){
	shell_exec('/home/pi/restartsomething.sh');
}
?>
Toulon7559
Posts: 848
Joined: Sunday 23 February 2014 17:56
Target OS: Raspberry Pi / ODroid
Domoticz version: mixed
Location: Hengelo(Ov)/NL
Contact:

Re: Script to restart Domoticz if it crashes

Post by Toulon7559 »

;-) Thanks for the contribution to my learning curve for PHP!

Looks like a good starter to check&remedy by PHP-script those 'failing' ESP-readouts mentioned here
In my setup WUnderground tellback-info appears as device in Domoticz. If upload fails, also the time validity from the read info stalls, and with such PHP-script I can detect the anomaly: useful script-example also for dealing with this 'problem'!

For the above applications developed PHP-scripts based on your example, with json triggered by rid and running under a cron like the one in this message.
Refinements come after the basic testing ........
Set1 = RPI-Zero+RFXCom433+S0PCM+Shield for BMP180/DS18B20/RS485+DDS238-1ZNs
Set2 = RPI-3A++RFLinkGTW+ESP8266s+PWS_WS7000
Common = KAKUs+3*PVLogger+PWS_TFA_Nexus
plus series of 'satellites' for dedicated interfacing, monitoring & control.
Toulon7559
Posts: 848
Joined: Sunday 23 February 2014 17:56
Target OS: Raspberry Pi / ODroid
Domoticz version: mixed
Location: Hengelo(Ov)/NL
Contact:

Re: Script to restart Domoticz if it crashes

Post by Toulon7559 »

;-) Puzzled, because experiments not giving the effects expected.

Trying to find a solution for the 'problems' described in my previous message, put in a cron (besides the first line calling an sh-file containing calls to a series of Python-scripts):
# 1 line calling the crash_checker described earlier in this thread, triggered twice each hour
# 1 'simple' line commanding a reboot, triggered twice a day
# 2 lines pointing to PHP-scripts for continuous check on status of 2 devices [derived from Egregius' example]:
- report from an ESP8266 by MTTQ to Domoticz
- tellback from WUnderground

Is an odd collection, but purely intended for temporary experiments.

Code: Select all

*/5 * * * * /home/pi/domoticz/scripts/PythonRuntime01.sh 2>&1
25/50 * * * * /home/pi/domoticz/scripts/domoticz_crash_checker.sh 2>&1
11 4/20 * * * sudo reboot
* * * * * /usr/bin/php /home/pi/PHP/php_ESP8266C_91_reboot.php
* * * * * /usr/bin/php /home/pi/PHP/php_WU101_46_reboot.php
The first line is always OK, but nothing appears from the other lines ......
No errors when calling from the CLI the scriptlines contained under the above cron, but no results visible, except for line 3 (= reboot).
No effect visible when the lines 2 till 5 are individually added to line 1.
Apparently it makes a difference whether you trigger a line from CLI, or from cron.

Any suggestion to get them running_with_effect?
Set1 = RPI-Zero+RFXCom433+S0PCM+Shield for BMP180/DS18B20/RS485+DDS238-1ZNs
Set2 = RPI-3A++RFLinkGTW+ESP8266s+PWS_WS7000
Common = KAKUs+3*PVLogger+PWS_TFA_Nexus
plus series of 'satellites' for dedicated interfacing, monitoring & control.
User avatar
Egregius
Posts: 2589
Joined: Thursday 09 April 2015 12:19
Target OS: Linux
Domoticz version: v2024.7
Location: Beitem, BE
Contact:

Re: Script to restart Domoticz if it crashes

Post by Egregius »

Does the script works if you execute "/usr/bin/php /home/pi/PHP/php_ESP8266C_91_reboot.php" ?
What's in those php files?
Toulon7559
Posts: 848
Joined: Sunday 23 February 2014 17:56
Target OS: Raspberry Pi / ODroid
Domoticz version: mixed
Location: Hengelo(Ov)/NL
Contact:

Re: Script to restart Domoticz if it crashes

Post by Toulon7559 »

@Egregius

This is the PHP-script php_ESP8266C_91_reboot.php

Code: Select all

<?php
$lastupdate=0;
$domoticz=json_decode(file_get_contents('http://127.0.0.1:8080/json.htm?type=devices&rid=91'),true);
if(isset($domoticz['result'])){
	foreach($domoticz['result'] as $dom){
		$name=$dom['Name'];
		$update=strtotime($dom['LastUpdate']);
		if($update>$lastupdate)$lastupdate=$update;
	}
}
if($lastupdate>0&&$lastupdate<$_SERVER['REQUEST_TIME']-3600){
	shell_exec('sudo service domoticz restart');
}
?>
The second php-script only differs by rid-number 46 instead of 91.

This is Putty_CLI for first PHP-script with result:

Code: Select all

xx@raspberrypi:~$ sudo /usr/bin/php /home/pi/PHP/php_ESP8266C_91_reboot01.php
xx@raspberrypi:~$
That is reason to state 'no result visible'
In hindsight, perhaps useful (like I do for all my lua- and Python-scripting) to add at least during testing somewhere one or more simple one-line print-instruction(s): then you get positive&visible reporting that and how far the script is executing.
Set1 = RPI-Zero+RFXCom433+S0PCM+Shield for BMP180/DS18B20/RS485+DDS238-1ZNs
Set2 = RPI-3A++RFLinkGTW+ESP8266s+PWS_WS7000
Common = KAKUs+3*PVLogger+PWS_TFA_Nexus
plus series of 'satellites' for dedicated interfacing, monitoring & control.
User avatar
Egregius
Posts: 2589
Joined: Thursday 09 April 2015 12:19
Target OS: Linux
Domoticz version: v2024.7
Location: Beitem, BE
Contact:

Re: Script to restart Domoticz if it crashes

Post by Egregius »

Why did you change the json url?
now you only get the status of one device. No point of looping thru the result then.
If you don't want to use a roomplan to combine the 2 devices in 1 json call you can combine them in 1 script.

prefered way: http://www.domoticz.com/forum/viewtopic ... 89#p149685
Devices separately (added some echo statements):

Code: Select all

<?php
$idx1=183;
$idx2=257;
$timeout=3600;

$domoticz=json_decode(file_get_contents('http://127.0.0.1:8080/json.htm?type=devices&rid='.$idx1),true);
if(isset($domoticz['result'])){
	$lastupdate1=$domoticz['result'][0]['LastUpdate'];
	echo 'Device '.$idx1.' last updated at '.$lastupdate1.PHP_EOL;
}else echo 'Error retrieving data from domoticz for idx '.$idx1.PHP_EOL;

$domoticz2=json_decode(file_get_contents('http://127.0.0.1:8080/json.htm?type=devices&rid='.$idx2),true);
if(isset($domoticz2['result'])){
	$lastupdate2=$domoticz2['result'][0]['LastUpdate'];
	echo 'Device '.$idx2.' last updated at '.$lastupdate2.PHP_EOL;
}else echo 'Error retrieving data from domoticz for idx '.$idx2.PHP_EOL;

$lastupdate1=strtotime($lastupdate1);
$lastupdate2=strtotime($lastupdate2);
if($lastupdate1>0&&$lastupdate2>0&&$lastupdate1<$_SERVER['REQUEST_TIME']-3600&&$lastupdate2<$_SERVER['REQUEST_TIME']-$timeout){
	echo 'Last updates both more than '.$timeout.' seconds ago, restarting!';
	shell_exec('/home/pi/restartsomething.sh');
}
Toulon7559
Posts: 848
Joined: Sunday 23 February 2014 17:56
Target OS: Raspberry Pi / ODroid
Domoticz version: mixed
Location: Hengelo(Ov)/NL
Contact:

Re: Script to restart Domoticz if it crashes

Post by Toulon7559 »

@Egregius

2 Simple reasons for my changes and dedicated, separate scripts:
1) I have no roomplan
2) not yet far in the learning curve for php-scripts, and therefore only try small tuning-changes
=> run first simple test with only the most critical devices, and separate the handling of those devices to see whether one or the other, or all run without errors and/or report something. If they react differently also a lesson.

Thanks for upgrade of the script, because then just 1 script to run for checking of multiple, different rids:
slightly contrary to 2), but the echo-lines should really help visualizing the operation. The end result counts .........
;-) Homework for tomorrow to tune your example, perhaps adding a few more devices, and 'upgrade' the cron.
Set1 = RPI-Zero+RFXCom433+S0PCM+Shield for BMP180/DS18B20/RS485+DDS238-1ZNs
Set2 = RPI-3A++RFLinkGTW+ESP8266s+PWS_WS7000
Common = KAKUs+3*PVLogger+PWS_TFA_Nexus
plus series of 'satellites' for dedicated interfacing, monitoring & control.
User avatar
Egregius
Posts: 2589
Joined: Thursday 09 April 2015 12:19
Target OS: Linux
Domoticz version: v2024.7
Location: Beitem, BE
Contact:

Re: Script to restart Domoticz if it crashes

Post by Egregius »

If you add more devices I would definitely recommend using a roomplan for this. They are easy to create and maintain.
Also if more devices are added later you just have to add them to the roomplan without need to change the script.
Toulon7559
Posts: 848
Joined: Sunday 23 February 2014 17:56
Target OS: Raspberry Pi / ODroid
Domoticz version: mixed
Location: Hengelo(Ov)/NL
Contact:

Re: Script to restart Domoticz if it crashes

Post by Toulon7559 »

Now: see that 'single' PHP-script for multiple devices is running with effects as expected
Status 15th September2017:
The combination in cron of 'checkers' plus 'reboot/12hours' seems to work.
Very rude&basic, not a 100% remedy, but better than before .......
For 'Todo'-list: Make Roomplan & change 'checker' to Roomplan-script
Set1 = RPI-Zero+RFXCom433+S0PCM+Shield for BMP180/DS18B20/RS485+DDS238-1ZNs
Set2 = RPI-3A++RFLinkGTW+ESP8266s+PWS_WS7000
Common = KAKUs+3*PVLogger+PWS_TFA_Nexus
plus series of 'satellites' for dedicated interfacing, monitoring & control.
Derik
Posts: 1601
Joined: Friday 18 October 2013 23:33
Target OS: Raspberry Pi / ODroid
Domoticz version: BETA
Location: Arnhem/Nijmegen Nederland
Contact:

Re: Script to restart Domoticz if it crashes

Post by Derik »

mmm is there perhaps some one with a script that can reboot the rpi ?
My board is loosing connecting with my router...
And then i need to reboot, for new connection.
I monitor the board with a slave. so from that board i hope i can give a reboot command...
Xu4: Beta Extreme antenna RFXcomE,WU Fi Ping ip P1 Gen5 PVOutput Harmony HUE SolarmanPv OTG Winddelen Alive ESP Buienradar MySensors WOL Winddelen counting RPi: Beta SMAspot RFlinkTest Domoticz ...Different backups
User avatar
Egregius
Posts: 2589
Joined: Thursday 09 April 2015 12:19
Target OS: Linux
Domoticz version: v2024.7
Location: Beitem, BE
Contact:

Re: Script to restart Domoticz if it crashes

Post by Egregius »

Is it connected on wifi? I have one pi that loses wifi connection from time to time.
You don't need to reboot, just disable/enable wifi.
This is the script I use for that:

Code: Select all

#!/bin/bash
sudo ping -c1 192.168.2.1 > /dev/null
if [ $? != 0 ]
then
  echo "No network connection, restarting wlan0"
  sudo /sbin/ifdown 'wlan0'
  sleep 2
  sudo /sbin/ifup --force 'wlan0'
fi
Execute it every minute by cron and you should be fine.

If you're not on wifi try the script but replace wlan0 with the name of your connection, useally eth0 for cable.
Derik
Posts: 1601
Joined: Friday 18 October 2013 23:33
Target OS: Raspberry Pi / ODroid
Domoticz version: BETA
Location: Arnhem/Nijmegen Nederland
Contact:

Re: Script to restart Domoticz if it crashes

Post by Derik »

Egregius wrote: Tuesday 03 October 2017 7:31 Is it connected on wifi? I have one pi that loses wifi connection from time to time.
You don't need to reboot, just disable/enable wifi.
This is the script I use for that:

Code: Select all

#!/bin/bash
sudo ping -c1 192.168.2.1 > /dev/null
if [ $? != 0 ]
then
  echo "No network connection, restarting wlan0"
  sudo /sbin/ifdown 'wlan0'
  sleep 2
  sudo /sbin/ifup --force 'wlan0'
fi
Execute it every minute by cron and you should be fine.

If you're not on wifi try the script but replace wlan0 with the name of your connection, useally eth0 for cable.
mmm i do have a very fast and great Odroid Xu4....
The only problem he have,,, when i have a unstable internetconnection, he lose the connection.
Can i use this script also for a utp conection?
Thanks..
Xu4: Beta Extreme antenna RFXcomE,WU Fi Ping ip P1 Gen5 PVOutput Harmony HUE SolarmanPv OTG Winddelen Alive ESP Buienradar MySensors WOL Winddelen counting RPi: Beta SMAspot RFlinkTest Domoticz ...Different backups
User avatar
Egregius
Posts: 2589
Joined: Thursday 09 April 2015 12:19
Target OS: Linux
Domoticz version: v2024.7
Location: Beitem, BE
Contact:

Re: Script to restart Domoticz if it crashes

Post by Egregius »

Yes, replace wlan0 with the name of your connection
Derik
Posts: 1601
Joined: Friday 18 October 2013 23:33
Target OS: Raspberry Pi / ODroid
Domoticz version: BETA
Location: Arnhem/Nijmegen Nederland
Contact:

Re: Script to restart Domoticz if it crashes

Post by Derik »

mmm the name of my homenetwork??
i do not have a static in device i set it static in my router..
Xu4: Beta Extreme antenna RFXcomE,WU Fi Ping ip P1 Gen5 PVOutput Harmony HUE SolarmanPv OTG Winddelen Alive ESP Buienradar MySensors WOL Winddelen counting RPi: Beta SMAspot RFlinkTest Domoticz ...Different backups
jannl
Posts: 639
Joined: Thursday 02 October 2014 6:36
Target OS: Raspberry Pi / ODroid
Domoticz version: 2022.2
Location: Geleen
Contact:

Re: Script to restart Domoticz if it crashes

Post by jannl »

He means the ssid of your wifi accesspoint

Verstuurd vanaf mijn SM-G930F met Tapatalk

Post Reply

Who is online

Users browsing this forum: No registered users and 0 guests