Presence detection for Fritzbox script v2.0: using TR064 protocol communication to your router

[funky]

can you find a Mac adress of the phone in http://fritz.box and then run the following command
sudo python fritzhosts.py -i 192.168.178.1 -pyourpassword -d mac adress iPhone

for example:
sudo python fritzhosts.py -i 192.168.178.1 -prar -d F0:18:98:50:2A:83


are you sure your phone gets its ip adres from your fritzbox router? wifi enabled? it should be in de Wifi table of fritz.box. If its there and not presented in the script something is wrong.

[gerbenvanasselt]

this is the output i get:

Code: Select all

FritzHosts:
version:            0.6.2
model:              FRITZ!Box 7490
ip:                 192.168.178.1

Details for host:      64:A2:F9:0F:9A:6F

NewActive              : 1
NewIPAddress           : 192.168.178.54
NewInterfaceType       : 802.11
NewHostName            : OnePlus-6
NewAddressSource       : DHCP
NewLeaseTimeRemaining  : 864000

[funky]

Yep thats good! it can find the phone.. strange when you sudo python fritzhosts.py -i 192.168.178.1 -pyourpassword its not showing up.. it should be ..

[funky]

The script is made for an extender. Tomorrow I will work on the script to make it also available to disable the extender. What I also will do is change the script that you can use the Mac adress directly to access it then it should work.

[gerbenvanasselt]

Thanks for your replies and efforts so far.

(Bedankt voor al je inspanningen tot nu toe, deze aanwezigheidsdetectie heeft veel potentie!)

[funky]

Thanks for your replies and efforts so far.

(Bedankt voor al je inspanningen tot nu toe, deze aanwezigheidsdetectie heeft veel potentie!)

No thanks its nice to help other people. At my home it works perfectly and it will also at yours!
(fijne avond nog

[AllesVanZelf]

I think I commented out en removed the lines containing the externder part. So then this is the code:

Code: Select all

#!/bin/bash

#presence_detection.sh

#script Presence detection for domoticz  & Fritzbox v1.03
#created by g.j fun - 2 feb 2019
#making use of  fritzconnection  url: https://pypi.org/project/fritzconnection/
#dependencies: lxml, jq and requests
#tested with: Fritzbox 7581 and Fritzbox Repeater 1750E should work with all routers who supporting TR-064 protocol.
#place fritzconnection.py and fritzhosts.py in same directory as this script. 
#dont forget to change parameters router inside fritzconnection.py and activate TR-064 on router. 
#standard debug information is on just set debug to  false to disable it. 
# if you have a repeater please specify ip below. 

#setup device(s) change values here
ip_device1=192.168.188.23
name_device1="Samsung S9"
ip_device2=192.168.178.100
name_device2="Iphone werk"
ip_device3=192.168.188.57
name_device3="Samsung S7"
ip_device4=192.168.188.30
name_device4="Samsung S5"
idxdevice1=433
idxdevice2=448
idxdevice3=445
idxdevice4=442

#setup  router (http://fritz.box), repeater (Fritzbox repeater http://fritz.repeater) and  domoticz change values
password_fritzbox=yourpassword
host_fritzbox=192.168.188.1
#host_repeater=192.168.178.21
host_domoticz=192.168.188.80:8070

#set debug to true or false to see logging information 
enable_debug=true

#current directory script
cwd=$(cd -P -- "$(dirname -- "$0")" && pwd -P)

#get current status of device(s) in domoticz 

if [ "$enable_debug" == "true" ]; then
status_domoticz_device1=$(curl 'http://'$host_domoticz'/json.htm?type=devices&rid='$idxdevice1 | jq -r [.result][][].Data)
status_domoticz_device2=$(curl 'http://'$host_domoticz'/json.htm?type=devices&rid='$idxdevice2 | jq -r [.result][][].Data)
status_domoticz_device3=$(curl 'http://'$host_domoticz'/json.htm?type=devices&rid='$idxdevice3 | jq -r [.result][][].Data)
status_domoticz_device4=$(curl 'http://'$host_domoticz'/json.htm?type=devices&rid='$idxdevice4 | jq -r [.result][][].Data)

echo "status domoticz device 1 = $status_domoticz_device1"  
echo "status domoticz device 2 = $status_domoticz_device2"
echo "status domoticz device 3 = $status_domoticz_device3"  
echo "status domoticz device 4 = $status_domoticz_device4"

else
status_domoticz_device1=$(curl -s 'http://'$host_domoticz'/json.htm?type=devices&rid='$idxdevice1 | jq -r [.result][][].Data)
status_domoticz_device2=$(curl -s 'http://'$host_domoticz'/json.htm?type=devices&rid='$idxdevice2 | jq -r [.result][][].Data)
status_domoticz_device3=$(curl -s 'http://'$host_domoticz'/json.htm?type=devices&rid='$idxdevice3 | jq -r [.result][][].Data)
status_domoticz_device4=$(curl -s 'http://'$host_domoticz'/json.htm?type=devices&rid='$idxdevice4 | jq -r [.result][][].Data)

fi

#get device status of devices in Route

output_router=$(python $cwd/fritzhosts.py -i $host_fritzbox -p$password_fritzbox)
echo "getting output router" ;
status_router_device1=$(grep "$ip_device1" <<<"$output_router" | grep "active") 
status_router_device2=$(grep "$ip_device2" <<<"$output_router" | grep "active") 
status_router_device3=$(grep "$ip_device3" <<<"$output_router" | grep "active") 
status_router_device4=$(grep "$ip_device4" <<<"$output_router" | grep "active") 

#sleep 2;
#output_extender==$(python $cwd/fritzhosts.py -i $host_repeater -p$password_fritzbox)
#status_extender_device1=$(grep "$ip_device1" <<<"$output_extender" | grep "active") 
#status_extender_device2=$(grep "$ip_device2" <<<"$output_extender" | grep "active") 


if [ "$enable_debug" == "true" ]; then
echo "output router devices = $output_router" 
#echo "output repeater devices = $output_extender" 
echo "status router device 1 = $status_router_device1"
echo "status router device 2 = $status_router_device2"
echo "status router device 3 = $status_router_device3"
echo "status router device 4 = $status_router_device4"

#echo "status repeater device 1 = $status_extender_device1"
#echo "status repeater device2 = $status_extender_device2"
fi

## Device 1
if [ -z "$status_router_device1" ] ; then
# device is not active in router so set it to off
status_router_device1="Off" 
else
#device is active but so we set variable to on 
status_router_device1="On"
fi

if [ "$status_router_device1" == "$status_domoticz_device1" ]; then
# both are simular so there is nothing to change. 
echo $(date -u) "status router and domoticz for $name_device1 are simular, we do nothing"  
else
#router status vs domoticz status are not equal we set domoticz status to router status. 
#we change the value in domoticz
echo $(date -u)"status router is not simular to status domoticz. We change status domoticz for $name_device1 to  $status_router_device1" 
wget -q --delete-after "http://$host_domoticz/json.htm?type=command&param=switchlight&idx=$idxdevice1&switchcmd=$status_router_device1" >/dev/null 2>&1

#we send logging information to domoticz
wget -q --delete-after "http://$host_domoticz/json.htm?type=command&param=addlogmessage&message=presence-detection-logging $name_device1 = $status_router_device1" >/dev/null 2>&1
 
fi

## Device 2
if [ -z "$status_router_device2" ] ; then
# device is not active in router so set it to off
status_router_device2="Off" 
else
#device is active but so we set variable to on 
status_router_device2="On"
fi

if [ "$status_router_device2" == "$status_domoticz_device2" ]; then
# both are simular so there is nothing to change.
echo $(date -u) "status router and domoticz for $name_device2 are simular, we do nothing" 
else
#router status vs domoticz status are not equal we set domoticz status to router status. 
echo $(date -u) "status router is not simular to status domoticz. We change status domoticz for $name_device2 to  $status_router_device2" 
#we change the value in domoticz
wget -q --delete-after "http://$host_domoticz/json.htm?type=command&param=switchlight&idx=$idxdevice2&switchcmd=$status_router_device2" >/dev/null 2>&1

#we send logging information to domoticz
wget -q --delete-after "http://$host_domoticz/json.htm?type=command&param=addlogmessage&message=presence-detection-logging $name_device2 = $status_router_device2" >/dev/null 2>&1
 
fi

## Device 3
if [ -z "$status_router_device3" ] ; then
# device is not active in router so set it to off
status_router_device3="Off" 
else
#device is active but so we set variable to on 
status_router_device3="On"
fi

if [ "$status_router_device3" == "$status_domoticz_device3" ]; then
# both are simular so there is nothing to change.
echo $(date -u) "status router and domoticz for $name_device3 are simular, we do nothing" 
else
#router status vs domoticz status are not equal we set domoticz status to router status. 
echo $(date -u) "status router is not simular to status domoticz. We change status domoticz for $name_device3 to  $status_router_device3" 
#we change the value in domoticz
wget -q --delete-after "http://$host_domoticz/json.htm?type=command&param=switchlight&idx=$idxdevice3&switchcmd=$status_router_device3" >/dev/null 2>&1

#we send logging information to domoticz
wget -q --delete-after "http://$host_domoticz/json.htm?type=command&param=addlogmessage&message=presence-detection-logging $name_device3 = $status_router_device3" >/dev/null 2>&1
 
fi


## Device 4
if [ -z "$status_router_device4" ] ; then
# device is not active in router so set it to off
status_router_device4="Off" 
else
#device is active but so we set variable to on 
status_router_device4="On"
fi

if [ "$status_router_device4" == "$status_domoticz_device4" ]; then
# both are simular so there is nothing to change.
echo $(date -u) "status router and domoticz for $name_device4 are simular, we do nothing" 
else
#router status vs domoticz status are not equal we set domoticz status to router status. 
echo $(date -u) "status router is not simular to status domoticz. We change status domoticz for $name_device4 to  $status_router_device4" 
#we change the value in domoticz
wget -q --delete-after "http://$host_domoticz/json.htm?type=command&param=switchlight&idx=$idxdevice4&switchcmd=$status_router_device4" >/dev/null 2>&1

#we send logging information to domoticz
wget -q --delete-after "http://$host_domoticz/json.htm?type=command&param=addlogmessage&message=presence-detection-logging $name_device4 = $status_router_device4" >/dev/null 2>&1
 
fi

The problem is that for example the S7 is active on the Fritzbox, and therefore active on domoticz, but this phone is for sure not at home. It takes some time to update. This is a Fritbox problem, I think. arping gives better results till now.
But thanks a lot for the effort you out into this!!

[funky]

Strange i do not have this problem. Update interval works every 30 seconds. And it can werk every 10 seconds

Problem with arping is the time-out time for iPhones. See documentation arping, They speak of time-outs of 45 minutes.

[funky]

Ok guys. I updated the script. It now works on Mac adresses instead of ip adresses. Its faster and more reliable. Mac adresses can be found in http://fritz.box
home network > network > devices pencil and then Device information.


@Alles van zelf (voor jou de repeater opgelost en als het goed is opgelost dat je devices niet geüpdatet worden) and @gerben (nu op basis van mac adressen wat het probleem oplost dat de tabel niet volledig werd ingeladen) will you check if it works?? git clone https://github.com/hydex80/presence_det ... z_fritzbox

Changed 1.04:
1.Devices are now targeted with Mac adresses instead of ipadresses > more reliable and faster.
2.If you don't have a repeater, leave the ip adress of the host_repeater empty. The script will ignore repeater settings.
3.There was a bug , sometimes devices was not updated, its now fixed.

Added:
1. Extra checks on availability of fritzhost and / or repeater
2. some nice colors for readability.

[gerbenvanasselt]

Thanks, Will try it tonight and let you know.

Verstuurd vanaf mijn ONEPLUS A6003 met Tapatalk

[gerbenvanasselt]

I get this error when running the python command

sudo python presence_detection.sh
File "presence_detection.sh", line 17
mac_device1=64:A2:F9:0F:9A:6F
^
SyntaxError: invalid syntax

this is the code i am using: (I blurred the password)

Code: Select all

#setup device(s) change values here
mac_device1=64:A2:F9:0F:9A:6F
name_device1="Phone Gerben"
mac_device2=20:04:F1:28:1B:2D:9E
name_device2="Phone Mirjam"
idxdevice1=42
idxdevice2=43

#setup  router (http://fritz.box), repeater (Fritzbox repeater http://fritz.repeater) and  domoticz change values
password_fritzbox=**********
host_fritzbox=192.168.178.1
#setup repeater, if you dont have any leave blank 
host_repeater=
#setup host domoticz
host_domoticz=192.168.178.58:8080

[funky]

Its not sudo python but sudo bash
So run the script with sudo bash presence_detection.sh

And are you Sure the Mac adress Of device2 is correct?
Is 7 digits and Mac adresses are 6 digits

[gerbenvanasselt]

Now it is working after bash command.

Do I have to add a crontab to get it working every minute? I saw you're example in this thread. I changed the path to the location of the script. But nothing happens.

Verstuurd vanaf mijn ONEPLUS A6003 met Tapatalk

[funky]

Now it is working after bash command.

Do I have to add a crontab to get it working every minute? I saw you're example in this thread. I changed the path to the location of the script. But nothing happens.

Verstuurd vanaf mijn ONEPLUS A6003 met Tapatalk

* * * * * /home/pi/domoticz/presence_detection.sh >> /home/pi/logs/presence2.log

The >>/home/pi/logs/presence2.log part you can see the progress if its running.

This is they way it works. are you sure you use the good crontab? There are 2 crontab.
sudo crontab -e
or
just
crontab -e
Im also confused wich I have to use so I changed them both. after that reboot and see in the /home/pi/logs/presence2.log if its running.

[funky]

oh there is another thing I forgot. You have to make the script executable with sudo chmod +x presence_detection.sh

[gerbenvanasselt]

I am very satisfied with the second version of the script that uses the Mac adress for presence detection. After some tips and tricks from Funky it runs very well.

I used arping in the past but an early conclusion is that it is much more stable than the chopper_rob script I used before.

@Funky thanks for the effort you've put in to creating this script.

[Prutsium]

Would it not make sense to join forces with: viewtopic.php?f=65&t=20467&start=460
The way presence is handled is much better (Phone online > Anyone present = On & Override options)

Just a suggestion

[capman]

When I run the presence_detection.sh file , I get some errors.
???

Code: Select all

pi@keuken:~ $ python3 --version                                                 Python 3.5.3
pi@keuken:~ $ python --version                                                  Python 2.7.13
pi@keuken:~ $ cd /home/domoticz/scripts/bash
pi@keuken:/home/domoticz/scripts/bash $ sudo bash presence_detection.sh
presence_detection.sh: regel 2: $'\r': opdracht niet gevonden
presence_detection.sh: regel 4: $'\r': opdracht niet gevonden
presence_detection.sh: regel 14: $'\r': opdracht niet gevonden
presence_detection.sh: regel 15: $'\r': opdracht niet gevonden
presence_detection.sh: regel 23: $'\r': opdracht niet gevonden
presence_detection.sh: regel 31: $'\r': opdracht niet gevonden
presence_detection.sh: regel 34: $'\r': opdracht niet gevonden
presence_detection.sh: regel 39: $'\r': opdracht niet gevonden
presence_detection.sh: regel 42: $'\r': opdracht niet gevonden
presence_detection.sh: regel 44: $'\r': opdracht niet gevonden
presence_detection.sh: regel 138: syntaxfout nabij onverwacht symbool 'fi'
'resence_detection.sh: regel 138: `        fi

Also a python error. Domoticz says python version 3.5.3

Code: Select all

pi@keuken:/home/domoticz/scripts/python $ sudo apt-get install jq Python
Pakketlijsten worden ingelezen... Klaar
Boom van vereisten wordt opgebouwd
De statusinformatie wordt gelezen... Klaar
E: Kan pakket Python niet vinden

But when installing, it says 'can not find python package' !?

[capman]

I got have jq installed with sudo pip install python jq. But still no luck

Code: Select all

pi@keuken:~ $ sudo apt-get install python jq python-lxml python-requests
Pakketlijsten worden ingelezen... Klaar
Boom van vereisten wordt opgebouwd
De statusinformatie wordt gelezen... Klaar
jq is reeds de nieuwste versie (1.5+dfsg-1.3).
python-lxml is reeds de nieuwste versie (3.7.1-1).
python-requests is reeds de nieuwste versie (2.12.4-1).
python is reeds de nieuwste versie (2.7.13-2).
De volgende pakketten zijn automatisch geïnstalleerd en zijn niet langer nodig:
  kodi-eventclients-common libssh-4
Gebruik 'sudo apt autoremove' om ze te verwijderen.
0 opgewaardeerd, 0 nieuw geïnstalleerd, 0 te verwijderen en 0 niet opgewaardeerd.
pi@keuken:~ $ sudo apt autoremove
Pakketlijsten worden ingelezen... Klaar
Boom van vereisten wordt opgebouwd
De statusinformatie wordt gelezen... Klaar
De volgende pakketten zullen VERWIJDERD worden:
  kodi-eventclients-common libssh-4
0 opgewaardeerd, 0 nieuw geïnstalleerd, 2 te verwijderen en 0 niet opgewaardeerd.
Na deze bewerking zal er 708 kB schijfruimte vrijkomen.
Wilt u doorgaan? [J/n]
(Database wordt ingelezen ... 89529 bestanden en mappen momenteel geïnstalleerd.)
kodi-eventclients-common (2:18.0-1~stretch) wordt verwijderd ...
libssh-4:armhf (0.7.3-2+deb9u1) wordt verwijderd ...
Bezig met afhandelen van triggers voor libc-bin (2.24-11+deb9u3) ...
pi@keuken:~ $ sudo apt-get update
Geraakt:1 http://archive.raspberrypi.org/debian stretch InRelease
Ophalen:2 http://raspbian.raspberrypi.org/raspbian stretch InRelease [15,0 kB]
Ophalen:3 http://raspbian.raspberrypi.org/raspbian stretch/main armhf Packages [11,7 MB]
Ophalen:4 http://raspbian.raspberrypi.org/raspbian stretch/contrib armhf Packages [56,9 kB]
Ophalen:5 http://raspbian.raspberrypi.org/raspbian stretch/non-free armhf Packages [95,5 kB]
Ophalen:6 http://raspbian.raspberrypi.org/raspbian stretch/rpi armhf Packages [1360 B]
11,8 MB opgehaald in 14s (830 kB/s)
Pakketlijsten worden ingelezen... Klaar
pi@keuken:~ $ sudo apt-get upgrade
Pakketlijsten worden ingelezen... Klaar
Boom van vereisten wordt opgebouwd
De statusinformatie wordt gelezen... Klaar
Opwaardering wordt doorgerekend... Klaar
0 opgewaardeerd, 0 nieuw geïnstalleerd, 0 te verwijderen en 0 niet opgewaardeerd.
pi@keuken:~ $ sudo bash presence_detection.sh
bash: presence_detection.sh: Bestand of map bestaat niet
pi@keuken:~ $ cd /home/domoticz/scripts/python/
pi@keuken:/home/domoticz/scripts/python $ sudo bash presence_detection.sh
presence_detection.sh: regel 2: $'\r': opdracht niet gevonden
presence_detection.sh: regel 4: $'\r': opdracht niet gevonden
presence_detection.sh: regel 14: $'\r': opdracht niet gevonden
presence_detection.sh: regel 15: $'\r': opdracht niet gevonden
presence_detection.sh: regel 23: $'\r': opdracht niet gevonden
presence_detection.sh: regel 31: $'\r': opdracht niet gevonden
presence_detection.sh: regel 34: $'\r': opdracht niet gevonden
presence_detection.sh: regel 39: $'\r': opdracht niet gevonden
presence_detection.sh: regel 42: $'\r': opdracht niet gevonden
presence_detection.sh: regel 44: $'\r': opdracht niet gevonden
presence_detection.sh: regel 138: syntaxfout nabij onverwacht symbool 'fi'
'resence_detection.sh: regel 138: `        fi
pi@keuken:/home/domoticz/scripts/python $

[funky]

Looks like you forgot a “ inside the presence detection script. Can you post the whole présence detection script here” i send you a PM