Script to restart Domoticz if it crashes Topic is solved

[fisics]

Hey guys,

Thought I share a (very basic) script that I use to check if Domoticz has crashed (offline) and restart it, I have this setup on a cron job running every ten minutes.

My script basically launches a Telnet session to Domoticz on port 8080, if it fails to connect it removes the 2 temporary DB files -shm and -wal and relaunches the service.

This code could probably be written in one line by one of you geniuses, but as I'm not a coder - it works for me!

Hope that helps someone else who has frequent offline crashes...

---

Code: Select all

#!/bin/bash

# This is a script to check if Domoticz is running or crashed, if crashed delete the 2 db files and relaunch - my script lives in a folder called domo-alive-test

(
echo open 127.0.0.1 8080
sleep 2
echo "exit"
) | telnet > /home/pi/domo-alive-test/info-output.txt


sleep 5

# Telnet connection check, lets open info-out and make sure it contains the failed telnet output otherwise stop.

grep -Po '\?Invalid command' /home/pi/domo-alive-test/info-output.txt > /home/pi/domo-alive-test/telnet-connection-test.txt

sed -i '1!d' /home/pi/domo-alive-test/telnet-connection-test.txt

info=$(cat /home/pi/domo-alive-test/telnet-connection-test.txt)

rm /home/pi/domo-alive-test/telnet-connection-test.txt

if [ "$info" = "?Invalid command" ]
then
	# Telnet connection refused
	rm /home/pi/domo-alive-test/info-output.txt
	rm /home/pi/domoticz/domoticz.db-wal
	rm /home/pi/domoticz/domoticz.db-shm
	sudo service domoticz.sh start
	exit
else
	exit
fi

[fisics]

Just had another thought on this, you could probably do it much easier using:

sudo service domoticz.sh status

To check if it returns

"[FAIL] domoticz is not running ... failed!"

[Derik]

looks great..
And better than the watchdog....

Thanks
For sharing

[simonrg]

A different approach is documented on the Wiki using the Watchdog service on the Raspberry pi - http://www.domoticz.com/wiki/Setting_up ... i_watchdog - this runs on the Raspberry Pi, so is self-contained and copes with the Raspberry Pi crashing itself rebooting the Raspberry Pi.

@Derik why Watchdog not good?

[SweetPants]

I'm using 'monit' on Raspberry and Ubuntu. You can also use it to monitor/restart other processes like nginx

/etc/monit/conf.d/domoticz.conf

check process domoticz with pidfile /var/run/domoticz.pid
start program "/etc/init.d/domoticz.sh start"
stop program "/etc/init.d/domoticz.sh stop"

/etc/monit/conf.d/nginx.conf

check process nginx with pidfile /var/run/nginx.pid
start program "/etc/init.d/nginx start"
stop program "/etc/init.d/nginx stop"
if failed host 127.0.0.1 port 80
protocol http then restart
if 5 restarts within 5 cycles then timeout

Regards, Harry

[nigels0]

A different approach is documented on the Wiki using the Watchdog service on the Raspberry pi - http://www.domoticz.com/wiki/Setting_up ... i_watchdog - this runs on the Raspberry Pi, so is self-contained and copes with the Raspberry Pi crashing itself rebooting the Raspberry Pi.

@Derik why Watchdog not good?

I looked for the domoticz.log file in /tmp, but it doesn't exist - so the watchdog doesn't start up.

[CopyCatz]

As stated in the manual, specify the log location in the command line parameters:

-log file_path

Then point the watchdog to the log file.

[nigels0]

I guess the danger of having the log being written to the SD or USB repeatedly is why the log is not switched on by default

[sincze]

I guess the danger of having the log being written to the SD or USB repeatedly is why the log is not switched on by default

I'm writing the log to RAM or NAS.

It seems that NGINX is crashing a lot. (I configured monit to send me an e-mail so now I know exactly how many times and it is a lot).
10+ times per day it will be restarted by Monit. Is this a known issue? Should I tweak the nginx configuration.
Unfortunately I did not look at the nginx log file to see if it is maybe the known php issue.

https://discussion.dreamhost.com/thread-128870.html

Also Domoticz interface sometimes stalls... several seconds and then moves on again. (Nginx related??)

Sándor

[ThinkPad]

Moved topic to 'Scripts' section.

[frave]

Hi, I was looking into a more simpler approach just like fisics opted a few posts back. The reason why is because I found out Domoticz ocassionally crashes. It does not happen a lot but it is frustrating when finding out the moment you would like to flip a switch. I have found a way to interpret the output of "sudo domoticz.sh service status" and it works flawlessly. So to help others with the same problem I thought I will make my first post here.

I just made a file called "domoticz_state_checker.sh" and inserted the following code:

Code: Select all

#!/bin/bash
while true
do

DomoticzState=`sudo service domoticz.sh status`

if [[ $DomoticzState == "domoticz is running." ]]
        then
                echo 'Domoticz is running. Nothing to do.'
elif [[ $DomoticzState == "domoticz is not running ... failed!" ]]
        then
                echo 'Domoticz is not running. Restarting Domoticz...'
                sudo service domoticz.sh restart
                echo 'Domoticz restarted.'
fi

sleep 15
done

I just start the script with "nohup ./domoticz_state_checker.sh &" and it runs in the background. As you can see in the code it checks the state of Domoticz every 15 seconds and when it finds out Domoticz stopped working it performs a simple restart.

[Derik]

@ Fisics....

How do you do this?
and where do you place the files??

[roblom]

There is also an example of a script in the monit wiki.

[Ingmar]

Hi, I was looking into a more simpler approach just like fisics opted a few posts back. The reason why is because I found out Domoticz ocassionally crashes. It does not happen a lot but it is frustrating when finding out the moment you would like to flip a switch. I have found a way to interpret the output of "sudo domoticz.sh service status" and it works flawlessly. So to help others with the same problem I thought I will make my first post here.

I just made a file called "domoticz_state_checker.sh" and inserted the following code:

Code: Select all

#!/bin/bash
while true
do

DomoticzState=`sudo service domoticz.sh status`

if [[ $DomoticzState == "domoticz is running." ]]
        then
                echo 'Domoticz is running. Nothing to do.'
elif [[ $DomoticzState == "domoticz is not running ... failed!" ]]
        then
                echo 'Domoticz is not running. Restarting Domoticz...'
                sudo service domoticz.sh restart
                echo 'Domoticz restarted.'
fi

sleep 15
done

I just start the script with "nohup ./domoticz_state_checker.sh &" and it runs in the background. As you can see in the code it checks the state of Domoticz every 15 seconds and when it finds out Domoticz stopped working it performs a simple restart.

Thanks, works perfectly!

I added the "nohup /home/pi/domoticz/scripts/domoticz_state_checker.sh &" command in the rc.local file (in /etc) so that it automatically starts when the Pi starts up.

[frave]

Thanks for your success story and of course your addition of the automatic startup. I think I have solved that differently. But your solution is plain and simple.

[frave]

I am writing this just to inform anyone that wants to use this script and is not running on a Raspberry Pi. I recently migrated Domoticz to new hardware with a standard Debian 8 (jessie) installation so I stopped using my Raspberry Pi for Domoticz. I then came to know the script I wrote above does not work anymore. I therefore edited it as followes.

Code: Select all

#!/bin/bash
while true
do

DomoticzState=`sudo service domoticz status`

if [[ $DomoticzState == *"active (running)"* ]]
        then
                echo 'Domoticz is running. Nothing to do.'
elif [[ $DomoticzState == *"inactive (dead)"* ]]
        then
                echo 'Domoticz is not running. Restarting Domoticz...'
                sudo service domoticz restart
                echo 'Domoticz restarted.'
         fi

sleep 15
done
### Done

Then start the script the same way, with the command "nohup ./domoticz_state_checker.sh &", to run in the background.

[HvdW]

A perfect script.
Thanks a lot!

This unexpected stall of Domoticz on the Rasperry affects everyone (I think)
The code resolves the problem in a very simple manner.

Code: Select all

  #!/bin/bash
    DomoticzState=`sudo service domoticz.sh status`

    if [[ $DomoticzState == "domoticz is running." ]]
            then
                    echo 'Domoticz is running. Nothing to do.'
    elif [[ $DomoticzState == "domoticz is not running ... failed!" ]]
            then
                    echo 'Domoticz is not running. Restarting Domoticz...'
                    sudo service domoticz.sh restart
                    echo 'Domoticz restarted.'
    fi

sudo chmod +x domoticz_state_checker.sh

crontab -e
*/5 * * * * /home/pi/scripts/domoticz_state_checker.sh

However I removed the loop and made a cronjob which makes the script run every 5 minutes

[Derik]

@ HDVW

Very simple script...
I give your srcipt a try, because it is very very simple..

Please Can you make a option when your script is restart Domotic that the board is sending a email?
I think that is a great option

[Derik]

strange...
Try the script on my RPi, works great..

Try it on my CB: not...
I have:

ScreenShot042.jpg (36.77 KiB) Viewed 56562 times

I have:

Code: Select all

sudo chmod +x restart_domoticz.sh

I have:

Code: Select all

*/3 * * * * sudo /home/linaro/scripts/restart_domoticz.sh

The same as on my RPi..
I changed the pad..

Some one please, where do i go wrong?

EDIT:
Found a solution..
Changed restart in start

[tjabas]

Im really new to this Linux and rasberry, but how do i install this script?

I have lot of problems with Domoticz freezing, so i guess this script would do the trick.