Working "Presence Detection" for Python 3 Script

Python and python framework

Moderator: leecollings

Post Reply
gschmidt
Posts: 200
Joined: Thursday 20 December 2018 11:03
Target OS: Raspberry Pi / ODroid
Domoticz version:
Contact:

Working "Presence Detection" for Python 3 Script

Post by gschmidt »

Hi,

I have a python 2.7 script which I want to change to 3.5.3 because domoticz is using 3.5.3 and reports errors due to version differences.
On my RPi, python 2.7 is default. I changed this with an ~/.bash_aliases to 3.5.3:

Code: Select all

pi@Domotica-Pi:~ $ python --version
Python 3.5.3
pi@Domotica-Pi:~ $ 
Now this script is using the urllib.request module: import urllib.request

But when I run the script I get the error:

Code: Select all

pi@Domotica-Pi:~ $ sudo python /home/pi/domoticz/scripts/python/check_device_online.py 192.168.1.70 19 10 120
Traceback (most recent call last):
  File "/home/pi/domoticz/scripts/python/check_device_online.py", line 15, in <module>
    import urllib.request
ImportError: No module named request
But when I test if the urllib.request module is present in the python 3.5 library....it is there:

Code: Select all

pi@Domotica-Pi:~ $ python -c "import urllib.request"
pi@Domotica-Pi:~ $ echo $?
0  # urllib.request module exists in system
pi@Domotica-Pi:~ $ 
Why does the Python script does not see the module?

Greetzzz,

Gerben
Last edited by gschmidt on Saturday 23 February 2019 12:46, edited 1 time in total.
gschmidt
Posts: 200
Joined: Thursday 20 December 2018 11:03
Target OS: Raspberry Pi / ODroid
Domoticz version:
Contact:

Re: Python 3.5 script error: import urllib.request module not found

Post by gschmidt »

I figured out that urllib.request is in the python 3.5.3 lib:

Code: Select all

pi@Domotica-Pi:~ $ python
Python 3.5.3 (default, Sep 27 2018, 17:25:39) 
[GCC 6.3.0 20170516] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import urllib.request
>>> urllib.request.Request
<class 'urllib.request.Request'>
>>> 
But when I run the code, it gives an error that the module not exist???
gschmidt
Posts: 200
Joined: Thursday 20 December 2018 11:03
Target OS: Raspberry Pi / ODroid
Domoticz version:
Contact:

Re: Python 3.5 script error: import urllib.request module not found

Post by gschmidt »

I have converted the script to python 3
The script is working (on a Rpi3B+)

I first had to switch the default Python version from 2.7.14 to 3.5.3
Since I only wanted to test the script in the 3.5.3 environment, I created a "~/.bash_aliases" file in the home/pi directory with:

Code: Select all

nano ~/.bash_aliases
in the editor I typed:

Code: Select all

alias python='/usr/bin/python3.5'
Then I converted the script online from 2.7 to 3.5
When running the script for testing I used "sudo python3" since "sudo python" only works when I change the .bash file in the root folder:

Code: Select all

sudo python3 /home/pi/domoticz/scripts/check_device_online.py 192.168.x.xx 19 10 120
After converting the script from 2to3 and running the script, it still had errors which managed to fix. The main changes were:

Code: Select all

base64string = base64.encodestring(('%s:%s' % (domoticzusername, domoticzpassword)).encode()).decode().replace('\n', '')
json_object = json.loads(domoticzrequest(domoticzurl).decode('utf-8'))
This is the converted script to python3.

Code: Select all

#!/usr/bin/python
#   Title: check_device_online.py
#   Author: Chopper_Rob
#   Date: 25-02-2015
#   Info: Checks the presence of the given device on the network and reports back to domoticz
#   URL : https://www.chopperrob.nl/domoticz/5-report-devices-online-status-to-domoticz
#
#   Mofied for Python 3
#   Author: Gerben Schmidt
#   Date: 23-02-2019
#   Version : 1.6.3
 
import sys
import datetime
import time
import os
import subprocess
import urllib.request, urllib.error, urllib.parse
import json
import base64
 
# Settings for the domoticz server
domoticzserver="192.168.2.1:8080"
domoticzusername = "admin"
domoticzpassword = "admin"
domoticzpasscode = "Light/Switch Protection"
 
# If enabled. The script will log to the file _.log
# Logging to file only happens after the check for other instances, before that it only prints to screen.
log_to_file = False
 
# The script supports two types to check if another instance of the script is running.
# One will use the ps command, but this does not work on all machine (Synology has problems)
# The other option is to create a pid file named _.pid. The script will update the timestamp
# every interval. If a new instance of the script spawns it will check the age of the pid file.
# If the file doesn't exist or it is older then 3 * Interval it will keep running, otherwise is stops.
# Please chose the option you want to use "ps" or "pid", if this option is kept empty it will not check and just run.
check_for_instances = "pid"

 
# DO NOT CHANGE BEYOND THIS LINE
if len(sys.argv) != 5 :
  print ("Not enough parameters. Needs %Host %Switchid %Interval %Cooldownperiod.")
  sys.exit(0)

device=sys.argv[1]
switchid=sys.argv[2]
interval=sys.argv[3]
cooldownperiod=sys.argv[4]
previousstate=-1
lastsuccess=datetime.datetime.now()
lastreported=-1
base64string = base64.encodestring(('%s:%s' % (domoticzusername, domoticzpassword)).encode()).decode().replace('\n', '')
domoticzurl = 'http://'+domoticzserver+'/json.htm?type=devices&filter=all&used=true&order=Name'
 
if check_for_instances.lower() == "pid":
  pidfile = sys.argv[0] + '_' + sys.argv[1] + '.pid'
  if os.path.isfile( pidfile ):
    print(datetime.datetime.now().strftime("%H:%M:%S") + "- pid file exists")
    if (time.time() - os.path.getmtime(pidfile)) < (float(interval) * 3):
      print(datetime.datetime.now().strftime("%H:%M:%S") + "- script seems to be still running, exiting")
      print(datetime.datetime.now().strftime("%H:%M:%S") + "- If this is not correct, please delete file " + pidfile)
      sys.exit(0)
    else:
      print(datetime.datetime.now().strftime("%H:%M:%S") + "- Seems to be an old file, ignoring.")
  else:
    open(pidfile, 'w').close() 
 
if check_for_instances.lower() == "ps":
  if int(subprocess.check_output('ps x | grep \'' + sys.argv[0] + ' ' + sys.argv[1] + '\' | grep -cv grep', shell=True)) > 2 :
    print((datetime.datetime.now().strftime("%H:%M:%S") + "- script already running. exiting."))
    sys.exit(0)
 
def log(message):
  print(message)
  if log_to_file == True:
    logfile = open(sys.argv[0] + '_' + sys.argv[1] + '.log', "a")
    logfile.write(message + "\n")
    logfile.close()

def domoticzstatus ():
  json_object = json.loads(domoticzrequest(domoticzurl).decode('utf-8'))
  status = 0
  switchfound = False
  if json_object["status"] == "OK":
    for i, v in enumerate(json_object["result"]):
      if json_object["result"][i]["idx"] == switchid:
        switchfound = True
        if json_object["result"][i]["Status"] == "On": 
          status = 1
        if json_object["result"][i]["Status"] == "Off": 
          status = 0
  if switchfound == False: print((datetime.datetime.now().strftime("%H:%M:%S") + "- Error. Could not find switch idx in Domoticz response. Defaulting to switch off."))
  return status

def domoticzrequest (url):
  request = urllib.request.Request(url)
  request.add_header("Authorization", "Basic %s" % base64string)
  response = urllib.request.urlopen(request)
  return response.read()

log (datetime.datetime.now().strftime("%H:%M:%S") + "- script started.")
 
lastreported = domoticzstatus()
if lastreported == 1 :
  log (datetime.datetime.now().strftime("%H:%M:%S") + "- according to domoticz, " + device + " is online")
if lastreported == 0 :
  log (datetime.datetime.now().strftime("%H:%M:%S") + "- according to domoticz, " + device + " is offline")
 
while 1==1:
  # currentstate = subprocess.call('ping -q -c1 -W 1 '+ device + ' > /dev/null', shell=True)
  currentstate = subprocess.call('sudo arping -q -c1 -W 1 '+ device + ' > /dev/null', shell=True)
 
  if currentstate == 0 : lastsuccess=datetime.datetime.now()
  if currentstate == 0 and currentstate != previousstate and lastreported == 1 : 
    log (datetime.datetime.now().strftime("%H:%M:%S") + "- " + device + " online, no need to tell domoticz")
  if currentstate == 0 and currentstate != previousstate and lastreported != 1 :
    if domoticzstatus() == 0 :
      log (datetime.datetime.now().strftime("%H:%M:%S") + "- " + device + " online, tell domoticz it's back")
      domoticzrequest("http://" + domoticzserver + "/json.htm?type=command&param=switchlight&idx=" + switchid + "&switchcmd=On&level=0" + "&passcode=" + domoticzpasscode)
    else:
      log (datetime.datetime.now().strftime("%H:%M:%S") + "- " + device + " online, but domoticz already knew")
    lastreported=1
 
  if currentstate == 1 and currentstate != previousstate :
    log (datetime.datetime.now().strftime("%H:%M:%S") + "- " + device + " offline, waiting for it to come back")
 
  if currentstate == 1 and (datetime.datetime.now()-lastsuccess).total_seconds() > float(cooldownperiod) and lastreported != 0 :
    if domoticzstatus() == 1 :
      log (datetime.datetime.now().strftime("%H:%M:%S") + "- " + device + " offline, tell domoticz it's gone")
      domoticzrequest("http://" + domoticzserver + "/json.htm?type=command&param=switchlight&idx=" + switchid + "&switchcmd=Off&level=0" + "&passcode=" + domoticzpasscode)
    else:
      log (datetime.datetime.now().strftime("%H:%M:%S") + "- " + device + " offline, but domoticz already knew")
    lastreported=0
 
  time.sleep (float(interval))
 
  previousstate=currentstate
  if check_for_instances.lower() == "pid": open(pidfile, 'w').close()
The virtual (phone) switches are responding properly, but the Domoticz Log is still reporting an error.

Code: Select all

2019-02-23 09:43:08.715 Error: EventSystem: Failed to execute python event script "/home/pi/domoticz/scripts/python/check_device_online.py"
2019-02-23 09:43:08.715 Error: EventSystem: Traceback (most recent call last):
2019-02-23 09:43:08.715 Error: EventSystem: File "/home/pi/domoticz/scripts/python/check_device_online.py", line 38, in <module>
2019-02-23 09:43:08.715 Error: EventSystem: if len(sys.argv) != 5 :
2019-02-23 09:43:08.715 Error: EventSystem: AttributeError: module 'sys' has no attribute 'argv' 
I can't figure out (yet) why this error occurs....but it appears that it has no effect on running the script.
Maybe somebody has an idea how to get rid of the log error?

Greetzzz,

Gerben
Last edited by gschmidt on Saturday 23 February 2019 15:02, edited 4 times in total.
gschmidt
Posts: 200
Joined: Thursday 20 December 2018 11:03
Target OS: Raspberry Pi / ODroid
Domoticz version:
Contact:

Re: Python 3.5 script error: import urllib.request module not found

Post by gschmidt »

Got it!

The location of the python script on my pi caused the log error!
I thought it should be in: "/home/pi/domoticz/scripts/python/check_device_online.py"
but it should be "/home/pi/domoticz/scripts/check_device_online.py"

Now the errors are gone! Working....
solarboy
Posts: 300
Joined: Thursday 01 November 2018 19:47
Target OS: Raspberry Pi / ODroid
Domoticz version: 2024.6
Location: Portugal
Contact:

Re: Working "Presence Detection" for Python 3 Script

Post by solarboy »

After upgrading to Bullseye I realised my presence detection wasn't working and that I had python2 and python 3 on my old Buster system.
I am a bit reluctant to install python 2 so I tried this solution but so far it is failing at line 53

Code: Select all

admin@domoticz2:~ $ sudo python3 /home/admin/domoticz/scripts/check_device_online.py 192.168.2.13 1110 10 700
Traceback (most recent call last):
  File "/home/admin/domoticz/scripts/check_device_online.py", line 53, in <module>
    base64string = base64.encodestring(('%s:%s' % (domoticzusername, domoticzpassword)).encode()).decode().replace('\n', '')
AttributeError: module 'base64' has no attribute 'encodestring'
I have changed my server IP, username and password in lines 23-25.

Does anyone have an idea what I am doing wrong, I don't "speak" python at all.
Intel NUC with Ubuntu Server VM (Proxmox),mosquitto(docker),RFXtrx433E,zwavejsUI (docker),Zigbee2mqtt(docker),SMA Hub (docker),Harmony Hub plugin, Kodi plugin,Homebridge(docker)+Google Home,APC UPS,SMA Modbus,Mitsubishi MQTT, Broadlink,Dombus
solarboy
Posts: 300
Joined: Thursday 01 November 2018 19:47
Target OS: Raspberry Pi / ODroid
Domoticz version: 2024.6
Location: Portugal
Contact:

Re: Working "Presence Detection" for Python 3 Script

Post by solarboy »

The line is :

Code: Select all

base64string = base64.encodestring(('%s:%s' % (domoticzusername, domoticzpassword)).encode()).decode().replace('\n', '')
Intel NUC with Ubuntu Server VM (Proxmox),mosquitto(docker),RFXtrx433E,zwavejsUI (docker),Zigbee2mqtt(docker),SMA Hub (docker),Harmony Hub plugin, Kodi plugin,Homebridge(docker)+Google Home,APC UPS,SMA Modbus,Mitsubishi MQTT, Broadlink,Dombus
solarboy
Posts: 300
Joined: Thursday 01 November 2018 19:47
Target OS: Raspberry Pi / ODroid
Domoticz version: 2024.6
Location: Portugal
Contact:

Re: Working "Presence Detection" for Python 3 Script

Post by solarboy »

Just as a heads up, at this point nothing on this page functions.

https://www.domoticz.com/wiki/Presence_detection

Including the original links and the updated links.
Intel NUC with Ubuntu Server VM (Proxmox),mosquitto(docker),RFXtrx433E,zwavejsUI (docker),Zigbee2mqtt(docker),SMA Hub (docker),Harmony Hub plugin, Kodi plugin,Homebridge(docker)+Google Home,APC UPS,SMA Modbus,Mitsubishi MQTT, Broadlink,Dombus
willemd
Posts: 621
Joined: Saturday 21 September 2019 17:55
Target OS: Raspberry Pi / ODroid
Domoticz version: 2024.1
Location: The Netherlands
Contact:

Re: Working "Presence Detection" for Python 3 Script

Post by willemd »

solarboy wrote: Saturday 20 January 2024 15:51 Just as a heads up, at this point nothing on this page functions.

https://www.domoticz.com/wiki/Presence_detection

Including the original links and the updated links.
I am using the plugin for presence detection called "system alive checker" without problems (setup menu, choose from hardware list)
Note you can only have one system alive checker running per system, but you can of course configure a list of IP addresses to be checked.
solarboy
Posts: 300
Joined: Thursday 01 November 2018 19:47
Target OS: Raspberry Pi / ODroid
Domoticz version: 2024.6
Location: Portugal
Contact:

Re: Working "Presence Detection" for Python 3 Script

Post by solarboy »

willemd wrote: Saturday 20 January 2024 16:59
solarboy wrote: Saturday 20 January 2024 15:51 Just as a heads up, at this point nothing on this page functions.

https://www.domoticz.com/wiki/Presence_detection

Including the original links and the updated links.
I am using the plugin for presence detection called "system alive checker" without problems (setup menu, choose from hardware list)
Note you can only have one system alive checker running per system, but you can of course configure a list of IP addresses to be checked.
Great suggestion , I am currently testing it, doesn't work so well with iphones (which my daughter has) but it's good enough.

Thanks for your input !
Intel NUC with Ubuntu Server VM (Proxmox),mosquitto(docker),RFXtrx433E,zwavejsUI (docker),Zigbee2mqtt(docker),SMA Hub (docker),Harmony Hub plugin, Kodi plugin,Homebridge(docker)+Google Home,APC UPS,SMA Modbus,Mitsubishi MQTT, Broadlink,Dombus
Kouseri
Posts: 57
Joined: Sunday 04 January 2015 21:24
Target OS: Raspberry Pi / ODroid
Domoticz version:
Location: Finland
Contact:

Re: Working "Presence Detection" for Python 3 Script

Post by Kouseri »

solarboy wrote: Saturday 20 January 2024 15:33 The line is :

Code: Select all

base64string = base64.encodestring(('%s:%s' % (domoticzusername, domoticzpassword)).encode()).decode().replace('\n', '')
I faced the same issue and it looks like the base64.decodestring() function has been deprecated since Python 3.1, and removed in Python 3.9

https://stackoverflow.com/questions/691 ... g-error-wh

Based on that information I managed to modify line 53 and the fix seems to be working for me at least. Please find the modified code from below:

Code: Select all

base64string = base64.encodebytes(('%s:%s' % (domoticzusername, domoticzpassword)).encode('utf8')).decode('utf8').replace('\n', '')
solarboy
Posts: 300
Joined: Thursday 01 November 2018 19:47
Target OS: Raspberry Pi / ODroid
Domoticz version: 2024.6
Location: Portugal
Contact:

Re: Working "Presence Detection" for Python 3 Script

Post by solarboy »

Brilliant. At some point I'll try Bullseye again and this will be helpful. Gotta love breaking changes.
Intel NUC with Ubuntu Server VM (Proxmox),mosquitto(docker),RFXtrx433E,zwavejsUI (docker),Zigbee2mqtt(docker),SMA Hub (docker),Harmony Hub plugin, Kodi plugin,Homebridge(docker)+Google Home,APC UPS,SMA Modbus,Mitsubishi MQTT, Broadlink,Dombus
Post Reply

Who is online

Users browsing this forum: No registered users and 1 guest