presence detection script LUA or System Alive Checker

[guantolento]

hi, i try to configure the System Alive Checker (Ping) integrate in hardware of domoticz. actually i test this system with 8 device some wired and some wireless. i take some test with a ping command in ssh from the raspberry console to the device in list. the ping is under 1 ms for the wired device and from 50/300 ms for the wireless device. after some day i look the log and sometimes the device come "off" or "on" in random. i look the log of the device (qnap) and the log of the access point (fritzbox) and the device are always "on".
i attach the snapshot of my setting in System Alive Checker (Ping).
With a LUA or BASH script i can resolve this problem ???
thanks a lot.

[jannl]

Should work better with a bash/python/perl script. Check the wiki for an example

[guantolento]

Should work better with a bash/python/perl script. Check the wiki for an example

Hi jannl, last night i look for wiki but i think its too late and i don't find this. Today i look for.

Inviato dal mio GT-I9301I utilizzando Tapatalk

[AndrevdBerg]

Hi, I experienced similar issues;
The problem is that sometimes, even when a device/phone is always on and connected, it might not be seen/detectable every now and then, not even pingable (ping) or scannable (fing) in a LUA/bash script.
The only way to solve it is to allow for a short grace period (say 4-5 min, as the script runs every minute this is 4-5 times):

In LUA it looks like below:

someoneactive = CheckActive(Mydevice) -- check using ping or fing command
If (someoneactive) then
SomeoneHome=STRUE
else
-- if it is more than 300 seconds ago since the particular device/phone went offline, consider it offline, otherwise it maybe offline or not
if (timediff(otherdevices_lastupdate[Mydevice])>300) then
SomeOneHome=SFALSE
else
SomeOneHome=SMAYBE
end
end
-- now update the domoticz variable if it has changed (affecting the above otherdevices_lastupdate[]!):
if (uservariables['Variable:SomeOneHome']~=SomeOneHome then
commandArray['Variable:SomeOneHome']=SomeOneHome
end


Does this help?

[guantolento]

this is the script example in wiki:

Code: Select all

commandArray = {}
 
 ping_success=os.execute('ping -c1 192.168.1.156')
 if ping_success then
   print("ping success")
   commandArray['Ping']='On'
 else
   print("ping fail")
   commandArray['Ping']='Off'	
 end
 
return commandArray

but if i need to set "off" only if device don't ping for 2 minutes consecutive it's possible ??? and how ????

[guantolento]

iam looking to this wiki :
https://www.domoticz.com/wiki/Presence_detection
someone use this script to check the presence of the ip il lan ??? work correctly???? bye bye.

[guantolento]

the problem with this wiki is that the crontab can start the script only 1 minute. i need to start the script every 20 second for example ???
can i use these issue???

* * * * * /home/pi/script/check_device_online.py
* * * * * sleep 20; /home/pi/script/check_device_online.py
* * * * * sleep 40; /home/pi/script/check_device_online.py

and i repeat this 3 line in crontab for all mi 8 device to check. it's possible ??????


edit: the script not work !!!!!!

[zedd]

Hi Guantolento,

I think you need to do some reading-up on crontab and check the headers of check_device_online.py! The script works fine.

You don't really use 'sleep' within crontab, you can just use crontab to start your scripts at the desired time.
Second, the check_device_online.py-script requires some input:

Code: Select all

check_device_online.py %Host %Switchid %Interval %Cooldownperiod

My cron looks like this:

Code: Select all

*/10 * * * * /home/pi/domoticz/scripts/check_device_online.py aaa.bbb.ccc.ddd 16 10 300 # MyPhone

This will start the script every ten minutes and will scan for device 'aaa.bbb.ccc.ddd' (replace with IP address). This IP corresponds with a switch in Domoticz with index 16. It will scan for this device every 10 seconds. A device will only be put 'Off' when it hasn't been seen in 300 seconds.

HTH,
Zedd

[guantolento]

Hi Guantolento,

I think you need to do some reading-up on crontab and check the headers of check_device_online.py! The script works fine.

You don't really use 'sleep' within crontab, you can just use crontab to start your scripts at the desired time.
Second, the check_device_online.py-script requires some input:

Code: Select all

check_device_online.py %Host %Switchid %Interval %Cooldownperiod

My cron looks like this:

Code: Select all

*/10 * * * * /home/pi/domoticz/scripts/check_device_online.py aaa.bbb.ccc.ddd 16 10 300 # MyPhone

This will start the script every ten minutes and will scan for device 'aaa.bbb.ccc.ddd' (replace with IP address). This IP corresponds with a switch in Domoticz with index 16. It will scan for this device every 10 seconds. A device will only be put 'Off' when it hasn't been seen in 300 seconds.

HTH,
Zedd

thank you Zedd, tomorrow i take some test about this. good afternoon.

[guantolento]

i setup all as the wiki, when i try to execute the script manually this is the report in tab:

Code: Select all

pi@raspberrypi:~/script$ python check_device_online.py 192.168.135.104 163 10 30
16:28:26- pid file exists
16:28:26- Seems to be an old file, ignoring.
16:28:26- script started.
Traceback (most recent call last):
  File "check_device_online.py", line 101, in <module>
    lastreported = domoticzstatus()
  File "check_device_online.py", line 79, in domoticzstatus
    json_object = json.loads(domoticzrequest(domoticzurl))
  File "check_device_online.py", line 96, in domoticzrequest
    response = urllib2.urlopen(request)
  File "/usr/lib/python2.7/urllib2.py", line 154, in urlopen
    return opener.open(url, data, timeout)
  File "/usr/lib/python2.7/urllib2.py", line 431, in open
    response = self._open(req, data)
  File "/usr/lib/python2.7/urllib2.py", line 449, in _open
    '_open', req)
  File "/usr/lib/python2.7/urllib2.py", line 409, in _call_chain
    result = func(*args)
  File "/usr/lib/python2.7/urllib2.py", line 1227, in http_open
    return self.do_open(httplib.HTTPConnection, req)
  File "/usr/lib/python2.7/urllib2.py", line 1200, in do_open
    r = h.getresponse(buffering=True)
  File "/usr/lib/python2.7/httplib.py", line 1111, in getresponse
    response.begin()
  File "/usr/lib/python2.7/httplib.py", line 444, in begin
    version, status, reason = self._read_status()
  File "/usr/lib/python2.7/httplib.py", line 408, in _read_status
    raise BadStatusLine(line)
httplib.BadStatusLine: ''

someone can help me ?

ps the only different is the folder where is the script. but i don't think this is a problem.

[philuxe]

same here

edit : fixed, I had forgotten to update some variables in the script

[guantolento]

I think the problem is the syntax of the script. In the week end i try some test with it.

Inviato dal mio GT-I9301I utilizzando Tapatalk

[zedd]

This is the script I use:

Code: Select all

#!/usr/bin/python
#   Title: check_device_online.py
#   Author: Chopper_Rob
#   Date: 25-02-2015
#   Info: Checks the presence of the given device on the network and reports back to domoticz
#   URL : https://www.chopperrob.nl/domoticz/5-report-devices-online-status-to-domoticz
#   Version : 1.6.2

import sys
import datetime
import time
import os
import subprocess
import urllib2
import json
import base64

# Settings for the domoticz server
domoticzserver="localhost:8080"
domoticzusername = ""
domoticzpassword = ""
domoticzpasscode = ""

# If enabled. The script will log to the file _.log
# Logging to file only happens after the check for other instances, before that it only prints to screen.
log_to_file = True

# The script supports two types to check if another instance of the script is running.
# One will use the ps command, but this does not work on all machine (Synology has problems)
# The other option is to create a pid file named _.pid. The script will update the timestamp
# every interval. If a new instance of the script spawns it will check the age of the pid file.
# If the file doesn't exist or it is older then 3 * Interval it will keep running, otherwise is stops.
# Please chose the option you want to use "ps" or "pid", if this option is kept empty it will not check and just run.
check_for_instances = "pid"



# DO NOT CHANGE BEYOND THIS LINE
if len(sys.argv) != 5 :
  print ("Not enough parameters. Needs %Host %Switchid %Interval %Cooldownperiod.")
  sys.exit(0)

device=sys.argv[1]
switchid=sys.argv[2]
interval=sys.argv[3]
cooldownperiod=sys.argv[4]
previousstate=-1
lastsuccess=datetime.datetime.now()
lastreported=-1
base64string = base64.encodestring('%s:%s' % (domoticzusername, domoticzpassword)).replace('\n', '')
domoticzurl = 'http://'+domoticzserver+'/json.htm?type=devices&filter=all&used=true&order=Name'

if check_for_instances.lower() == "pid":
  pidfile = sys.argv[0] + '_' + sys.argv[1] + '.pid'
  if os.path.isfile( pidfile ):
    print datetime.datetime.now().strftime("%H:%M:%S") + "- pid file exists"
    if (time.time() - os.path.getmtime(pidfile)) < (float(interval) * 3):
      print datetime.datetime.now().strftime("%H:%M:%S") + "- script seems to be still running, exiting"
      print datetime.datetime.now().strftime("%H:%M:%S") + "- If this is not correct, please delete file " + pidfile
      sys.exit(0)
    else:
      print datetime.datetime.now().strftime("%H:%M:%S") + "- Seems to be an old file, ignoring."
  else:
    open(pidfile, 'w').close()

if check_for_instances.lower() == "ps":
  if int(subprocess.check_output('ps x | grep \'' + sys.argv[0] + ' ' + sys.argv[1] + '\' | grep -cv grep', shell=True)) > 2 :
    print (datetime.datetime.now().strftime("%H:%M:%S") + "- script already running. exiting.")
    sys.exit(0)

def log(message):
  print message
  if log_to_file == True:
    logfile = open(sys.argv[0] + '_' + sys.argv[1] + '.log', "a")
    logfile.write(message + "\n")
    logfile.close()

def domoticzstatus ():
  json_object = json.loads(domoticzrequest(domoticzurl))
  status = 0
  switchfound = False
  if json_object["status"] == "OK":
    for i, v in enumerate(json_object["result"]):
      if json_object["result"][i]["idx"] == switchid:
        switchfound = True
        if json_object["result"][i]["Status"] == "On":
          status = 1
        if json_object["result"][i]["Status"] == "Off":
          status = 0
  if switchfound == False: print (datetime.datetime.now().strftime("%H:%M:%S") + "- Error. Could not find switch idx in Domoticz response. Defaulting to switch off.")
  return status

def domoticzrequest (url):
  request = urllib2.Request(url)
  request.add_header("Authorization", "Basic %s" % base64string)
  response = urllib2.urlopen(request)
  return response.read()

log (datetime.datetime.now().strftime("%H:%M:%S") + "- script started.")

lastreported = domoticzstatus()
if lastreported == 1 :
  log (datetime.datetime.now().strftime("%H:%M:%S") + "- according to domoticz, " + device + " is online")
if lastreported == 0 :
  log (datetime.datetime.now().strftime("%H:%M:%S") + "- according to domoticz, " + device + " is offline")

while 1==1:
  #currentstate = subprocess.call('ping -q -c1 -W 1 '+ device + ' > /dev/null', shell=True)
  currentstate = subprocess.call('sudo arping -q -c1 -W 1 '+ device + ' > /dev/null', shell=True)

  if currentstate == 0 : lastsuccess=datetime.datetime.now()
  if currentstate == 0 and currentstate != previousstate and lastreported == 1 :
    log (datetime.datetime.now().strftime("%H:%M:%S") + "- " + device + " online, no need to tell domoticz")
  if currentstate == 0 and currentstate != previousstate and lastreported != 1 :
    if domoticzstatus() == 0 :
      log (datetime.datetime.now().strftime("%H:%M:%S") + "- " + device + " online, tell domoticz it's back")
      domoticzrequest("http://" + domoticzserver + "/json.htm?type=command&param=switchlight&idx=" + switchid + "&switchcmd=On&level=0" + "&passcode=" + domoticzpasscode)
    else:
      log (datetime.datetime.now().strftime("%H:%M:%S") + "- " + device + " online, but domoticz already knew")
    lastreported=1

  if currentstate == 1 and currentstate != previousstate :
    log (datetime.datetime.now().strftime("%H:%M:%S") + "- " + device + " offline, waiting for it to come back")

  if currentstate == 1 and (datetime.datetime.now()-lastsuccess).total_seconds() > float(cooldownperiod) and lastreported != 0 :
    if domoticzstatus() == 1 :
      log (datetime.datetime.now().strftime("%H:%M:%S") + "- " + device + " offline, tell domoticz it's gone")
      domoticzrequest("http://" + domoticzserver + "/json.htm?type=command&param=switchlight&idx=" + switchid + "&switchcmd=Off&level=0" + "&passcode=" + domoticzpasscode)
    else:
      log (datetime.datetime.now().strftime("%H:%M:%S") + "- " + device + " offline, but domoticz already knew")
    lastreported=0

  time.sleep (float(interval))

  previousstate=currentstate
  if check_for_instances.lower() == "pid": open(pidfile, 'w').close()

DO check however if your Domoticz can be reached through localhost:8080 or adapt the variable in the script accordingly. If I read your error-message correctly, your Domoticz is not reachble through 8080?

[Siewert308SW]

Hi Guantolento,

Do as Zedd and wiki says and your fine.
Which port did you configure for Domoticz.
Fun thing about this script is when a device is online it detects it within 10 to 20sec.
But when gone it sets the virtual switch off according your given timers in the cron part.
Still use this script for my devices which don't have bluetooth.
But was searching for a more reliable version as my WiFi wasn't detected at some places in the house.
While BT is even detected attic while the Pi3 is located in the livingroom, thats almost 8mtr true 2 concrete floors.

At the moment some device on wifi still use this script.
But 2 laptops and 3 phones are detected by Bluetooth.
It's still the same script only it scans for BT devices like it does for WiFi.
As Bluetooth can only handle one search every 5 seconds i set them like at a minimum of 10 seconds apart...
You only need to know the MAC of your bluetooth device, it doesn't have to be paired with you Pi.

Be aware, don't set all the timers at the same interval.
You will get introuble as BT can't handle that.
And using BT has one side effect, on a Pi3 not noticeable but on the Pi2 your CPU usage will be higher then normal

Fun part about BT is that is reliable, stable, not sensitive for ip changes incase you don't have static ip's

Below you'll find a example of my cron using network and bluetooth devices.
Also the two python script created by Chopperrob but one of those i amended for BT.
Same script only pings for BT devices...

eth script: same as Zedd his version
https://bitbucket.org/Siewert308SW/domo ... ew-default

BT script: replaced ping with l2ping for pinging bluetooth devices
https://bitbucket.org/Siewert308SW/domo ... ew-default


My cron:

Code: Select all

########################
# Laptops Bluetooth

# Siewert Laptop BT
*/10 * * * * sudo python /home/pi/domoticz/scripts/python/bt_device_online.py 00:00:00:00:00:00 9 45 60 > /dev/null 2>&1

# Jerina Laptop BT
*/10 * * * * sudo python /home/pi/domoticz/scripts/python/bt_device_online.py 00:00:00:00:00:00 10 45 60 > /dev/null 2>&1

########################
# Telefoons Bluetooth

# Jerina GSM BT
*/10 * * * * sudo python /home/pi/domoticz/scripts/python/bt_device_online.py 00:00:00:00:00:00 13 60 90 > /dev/null 2>&1

# Siewert GSM BT
*/10 * * * * sudo python /home/pi/domoticz/scripts/python/bt_device_online.py 00:00:00:00:00:00 12 70 90 > /dev/null 2>&1

# Natalya GSM BT
*/10 * * * * sudo python /home/pi/domoticz/scripts/python/bt_device_online.py 00:00:00:00:00:00 14 80 90 > /dev/null 2>&1

########################
# Media Devices

# Televisie
*/10 * * * * sudo python /home/pi/domoticz/scripts/python/eth_device_online.py 192.168.178.11 15 160 180 > /dev/null 2>&1

# MediaBox
*/10 * * * * sudo python /home/pi/domoticz/scripts/python/eth_device_online.py 192.168.178.120 16 170 180 > /dev/null 2>&1

########################
# Overige Devices

# Raspberry Slave eth0
*/10 * * * * sudo python /home/pi/domoticz/scripts/python/eth_device_online.py 192.168.178.111 24 240 300 > /dev/null 2>&1

########################
# Network Devices

# Router
*/10 * * * * sudo python /home/pi/domoticz/scripts/python/eth_device_online.py 192.168.178.1 19 60 60 > /dev/null 2>&1

# Internet
*/10 * * * * sudo python /home/pi/domoticz/scripts/python/eth_device_online.py 8.8.8.8 20 60 60 > /dev/null 2>&1

[zedd]

Nice alteration, Siewert, I'll implement that one as well.
It will save me from looking up all IP's and making them static!

[Siewert308SW]

Nice alteration, Siewert, I'll implement that one as well.
It will save me from looking up all IP's and making them static!

Yep, sometimes it doesn't have to go the hard way.
It also saves me adding and maintaining IP for family.
When we are on holidays and my mother, mother/brother in law or neighbours are taking care of the house.
So they don't have to worry about the alarm or lights when entering/leaving the house

[guantolento]

Sorry for my delay .... in my python script there are some format errors, i take the code and paste in notepad++ with lenguage python and now work.
just one information. in crontab i setting this.

*/10 * * * * sudo python /home/pi/script/check_device_online.py 192.168.135.4 183 60 60

but after 34 seconds i power off the device i look that the virtual switch come off, i think it need come off after 60 seconds ..... why ????

i dont' understand what mean the first value after the idx. the second is the delay to put off i think.

[lost]

You only need to know the MAC of your bluetooth device, it doesn't have to be paired with you Pi.

The problem is you advertise some known BT MACs all the time, when checking for presence.

And that's easy to monitor and a clever thief checking all your radio devices from the outside (to know about your 868/433MHz detectors, as well as BT ones as some BT-LE ones now exists...) may understand very quickly why you are doing this!

After this, spoofing a BT MAC address is a one liner. He just have to get in unnoticed.

So you have 2 choice:
1)Pairing, but this may take longer for a presence detection & less range. But you'll benefit from the BT crypto/authenticate used for a full pairing. That's the best you may get from BT, even if it's security level have never been so good that's now not supposed to be the common thief range of knownledge.

2)At least checking a few more info the BT device gives in the response when BT-pinged, but are at least not continuously advertised every 10 seconds. On top of that, for devices found present you can relax on the check period: 10/15mn instead of 10s when absent for instance. Thus less chance to catch a full response at the right time when a phone is there... and understand why sparse pings occurs: The lucky one that gets one ping+response in a few mn check may believe this is a device that just got out of range and tries to pair again.

And you may also combine: (2) for quick response, able to just give an added delay for alarm trigger for instance... then (1), but if full pairing is not done withing 30s for instance: Ring all the bells!

On my side I made a python running forever, as a service, configured by a yml file with the BT devices to search & infos to check (plus some per device search timings, to handle tricky ones), triggering a virtual switch for global presence in Domoticz used to act on secure panel/alarm handling Lua scripts.

Still not perfect, depending on BT security limits, but at least not spoofable with some "public" info broadcasted every 10s when I'm not at home!