Page 1 of 7
Presence detection for Fritzbox script v2.0: using TR064 protocol communication to your router
Posted: Friday 18 January 2019 17:06
by funky
Hello there,
Since i was not happy with the available presence detection scripts which are worked not very well with iPhone. I made a presence detection script using the TR-064 protocol of my Fritzbox router and get direct information of the router itself.
This saves battery of your smartphone because there is no need of pinging all the time and its more accurate, because the router is made for having the correct information about network devices.
This script is tested with a fritzbox 7581 router but should be working with more routers who has a tr064 protocol implementation.
install script newest version 2.4:
Code: Select all
git clone https://github.com/hydex80/presence_detection_domoticz_fritzbox.git
more information:
https://github.com/hydex80/presence_det ... z_fritzbox
Old version 2.3 can be downloaded here:
Code: Select all
git clone -b version-2.3 https://github.com/hydex80/presence_detection_domoticz_fritzbox
wiki:
https://www.domoticz.com/wiki/Presence_ ... ox_router)
FAQ en readme:
https://github.com/hydex80/presence_det ... /README.md
Changed 2.4 18 dec 2020
- Now support for multiple repeaters (as many as you want
- Install script is updated. More checks and a little help by showing the default gateway.
- Config.txt is totally rewritten. Sorry if you update to this version you have to reinstall. Tip: Make a print screen of your old config.txt. You can leave the domoticz settings as is. In the script you can choose to set manually set the settings for the devices. If you want a fresh install? Remove the old devices from domoticz first before you start the script.
- delay time added. Can be set inside the script file. Change only if for some reason your FritzBox is replying slower then the default 2 seconds
Changed 2.3 26 juli 2019
1.Added lightning protection (thnx to allesvanzelf for supporting and added these lines)
2.Some text changes.
Changed 2.2 - 16 march 2019:
1.Check if dependencies are installed is now fixed.
2.Updated FAQ:
https://github.com/hydex80/presence_det ... /README.md
Changed 2.0 - 23 februari 2019:
1. Easy installer. Now there is no need to edit script files
2. Add a lot of checks to make the script more reliable.
3. Its now possible to add as many devices as you want.
Changed 1.04 - 4 februari 2019:
1.Devices are now targeted with Mac adresses instead of ipadresses > more reliable and faster.
2.If you don't have a repeater, leave the ip adress of the host_repeater empty. The script will ignore repeater settings.
3.There was a bug , sometimes devices where not updated, its now fixed.
Changed 1.0:
1. Extra checks on availability of fritzhost and / or repeater
2. some nice colors for readability.
Re: Presence detection for Fritzbox script v1.0: using TR064 protocol communication to your router
Posted: Wednesday 30 January 2019 23:43
by andreo
Hello Funky,
Thx for sharing your script, i have tested this on a fritzbox 7490.
When i run the fritzconnection.py i get the following output
Code: Select all
pi@raspberrypi:~/domoticz/scripts/python $ python fritzconnection.py
FritzConnection:
version: 0.6
model: FRITZ!Box 7490
()
After running the script presence_detection.sh i get the list of active devices on router and extender
Code: Select all
pi@raspberrypi:~/domoticz/scripts/python $ ./presence_detection.sh
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 1884 100 1884 0 0 429k 0 --:--:-- --:--:-- --:--:-- 613k
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 1889 100 1889 0 0 453k 0 --:--:-- --:--:-- --:--:-- 614k
status domoticz device 1 = On
status domoticz device 2 = Off
getting output router
output router devices =
FritzHosts:
version: 0.6.2
model: FRITZ!Box 7490
ip: 192.168.178.1
List of registered hosts:
n: ip name mac status
0: 192.168.178.201 Andre - -
1: 192.168.178.44 Chromecast 30:FD:38:AD:CA:7D active
2: 192.168.178.38 DESKTOP-33H19JO 00:DB:DF:72:1A:C3 -
3: 192.168.178.42 DESKTOP-VN90648 00:26:6C:62:BF:7A -
4: 192.168.178.21 DESKTOP-VN90648 B4:82:FE:61:A1:9F -
5: 192.168.178.30 DMP-BDT120 CC:7E:E7:E5:86:10 active
6: 192.168.178.20 DNR 00:04:63:10:9A:AE active
7: 192.168.178.29 Domoticz 4C:66:41:7F:37:28 -
8: 192.168.178.22 ESP-0729A1 B4:E6:2D:07:29:A1 active
9: 192.168.178.32 ESP-Easy-0 5C:CF:7F:02:08:3A -
10: 192.168.178.41 Galaxy-A3-2016 78:00:9E:EB:9F:C2 -
11: 192.168.178.26 Galaxy-A3-2017 EC:10:7B:30:88:87 -
12: 192.168.178.46 HUAWEI-Mate-9-98d17a4e7c4 8C:0D:76:CB:33:4E active
13: 192.168.178.39 NB0121 00:1E:65:00:FC:E0 -
14: 192.168.178.31 P1Gateway-33 5C:CF:7F:15:B6:6C active
15: 192.168.178.35 PC-192-168-178-35 00:60:34:06:FC:AA active
16: 192.168.178.48 PanasonicWT50 CC:7E:E7:80:F3:69 active
17: 192.168.178.36 QNAP 00:08:9B:C8:77:D3 -
18: 192.168.178.34 SYNC 50:65:83:2C:6D:6D -
19: 192.168.178.52 TV-Samsung-LED28 78:BD:BC:0F:E9:F1 -
20: 192.168.178.202 VPN - -
21: 192.168.178.45 VPNrouter B8:27:EB:7F:7D:F2 active
22: 192.168.178.24 android-dd162b810130b789 00:73:E0:77:6F:32 -
23: 192.168.178.25 dimmer-TV-links-3 A0:20:A6:21:BC:66 active
24: 192.168.178.23 dimmer-TV-rechts-4 A0:20:A6:1A:BB:F9 -
25: 192.168.178.47 dimmer-bank-rechts-6 5C:CF:7F:2F:60:3A -
26: 192.168.178.28 dimmer-gang-2 A0:20:A6:1A:BA:43 active
27: 192.168.178.1 fritz.box 34:31:C4:C9:96:9F active
28: 192.168.178.43 fritz.repeater 62:FF:4D:FA:A7:B0 active
29: 192.168.178.40 iPad-ac B8:FF:61:59:E5:AC active
30: 192.168.178.33 iPad-van-Andre 70:DE:E2:17:4A:02 active
31: 192.168.178.54 iPadvanrancoise 5C:97:F3:8D:8E:00 active
32: 192.168.178.29 raspberrypi B8:27:EB:A7:87:3B active
33: 192.168.178.27 test-unit-1-1 18:FE:34:FB:BB:37 -
output extender devices = =
FritzHosts:
version: 0.6.2
model: FRITZ!WLAN Repeater 1750E
ip: 192.168.178.43
List of registered hosts:
n: ip name mac status
0: 192.168.178.1 Access Point 34:31:C4:C9:96:A1 active
1: 192.168.178.44 Chromecast 30:FD:38:AD:CA:7D -
2: 192.168.178.38 DESKTOP-33H19JO 00:DB:DF:72:1A:C3 -
3: 192.168.178.21 DESKTOP-VN90648 B4:82:FE:61:A1:9F active
4: 192.168.178.32 ESP-Easy-0 5C:CF:7F:02:08:3A active
5: 192.168.178.26 Galaxy-A3-2017 EC:10:7B:30:88:87 active
6: 192.168.178.20 PC-192-168-178-20 00:04:63:10:9A:AE active
7: 192.168.178.22 PC-192-168-178-22 B4:E6:2D:07:29:A1 -
8: 192.168.178.24 PC-192-168-178-24 24:EC:99:17:E4:CA -
9: 192.168.178.25 PC-192-168-178-25 A0:20:A6:21:BC:66 active
10: 192.168.178.28 PC-192-168-178-28 A0:20:A6:1A:BA:43 active
11: 192.168.178.29 PC-192-168-178-29 B8:27:EB:A7:87:3B active
12: 192.168.178.30 PC-192-168-178-30 CC:7E:E7:E5:86:10 active
13: 192.168.178.31 PC-192-168-178-31 5C:CF:7F:15:B6:6C active
14: 192.168.178.34 PC-192-168-178-34 50:65:83:2C:6D:6D -
15: 192.168.178.35 PC-192-168-178-35 00:60:34:06:FC:AA -
16: 192.168.178.42 PC-192-168-178-42 00:26:6C:62:BF:7A -
17: 192.168.178.45 PC-192-168-178-45 B8:27:EB:7F:7D:F2 active
18: 192.168.178.46 PC-192-168-178-46 8C:0D:76:CB:33:4E -
19: 192.168.178.47 PC-192-168-178-47 5C:CF:7F:2F:60:3A -
20: 192.168.178.48 PC-192-168-178-48 CC:7E:E7:80:F3:69 active
21: 192.168.178.52 PC-192-168-178-52 78:BD:BC:0F:E9:F1 -
22: 192.168.178.29 Samsung-Galaxy-S7-edge 4C:66:41:7F:37:28 -
23: 192.168.178.24 android-dd162b810130b789 00:73:E0:77:6F:32 -
24: 192.168.178.23 dimmer-TV-rechts-4 A0:20:A6:1A:BB:F9 active
25: 192.168.178.43 fritz.repeater 7C:FF:4D:FA:A7:AE active
26: 192.168.178.40 iPad-ac B8:FF:61:59:E5:AC active
27: 192.168.178.33 iPad-van-Andre 70:DE:E2:17:4A:02 active
28: 192.168.178.54 iPadvanrancoise 5C:97:F3:8D:8E:00 -
29: 192.168.178.27 test-unit-1-1 18:FE:34:FB:BB:37 active
status router device 1 = 12: 192.168.178.46 HUAWEI-Mate-9-98d17a4e7c4 8C:0D:76:CB:33:4E active
status router device 2 =
status extender device 1 =
status extender device2 = 5: 192.168.178.26 Galaxy-A3-2017 EC:10:7B:30:88:87 active
Wed 30 Jan 22:35:14 UTC 2019 status router and domoticz for Mate9 are simular, we do nothing
Wed 30 Jan 22:35:14 UTC 2019 status router is not simular to status domoticz. We change status domoticz for GalaxyA3 to On
Also the devices in Domoticz are set to the correct status.
I put the 3 files in the ~/domoticz/scripts/python directory but how do i run the script automactily? do i have to run the sh script by cronjob?
Re: Presence detection for Fritzbox script v1.0: using TR064 protocol communication to your router
Posted: Thursday 31 January 2019 8:16
by funky
Hi Andreo,
I made a cronjob to run the script. I run it every 30 seconds.
* * * * * /home/pi/domoticz/presence_detection.sh >> /home/pi/logs/presence2.log
* * * * * ( sleep 30 ; /home/pi/domoticz/presence_detection.sh) >> /home/pi/logs/presence2.log
Re: Presence detection for Fritzbox script v1.0: using TR064 protocol communication to your router
Posted: Friday 01 February 2019 0:04
by andreo
it's working
after trying many solutions, this is finally the first presence detection that works accurately
Re: Presence detection for Fritzbox script v1.0: using TR064 protocol communication to your router
Posted: Friday 01 February 2019 7:21
by funky
Nice to hear!
Re: Presence detection for Fritzbox script v1.0: using TR064 protocol communication to your router
Posted: Friday 01 February 2019 8:02
by emme
I'm working on the same on NodeRed
the ARP table on fritz seems to be really accurate!!!
Re: Presence detection for Fritzbox script v1.0: using TR064 protocol communication to your router
Posted: Friday 01 February 2019 20:54
by gerbenvanasselt
I am very eager to install this presence detection, but cannot even complete the first step.
I am heaving trouble installing
pip install fritzconnection and the pip install lxml requests
Building wheels for collected packages: lxml
Running setup.py bdist_wheel for lxml ... \
It sort of crashes and nothing happens. Anyone having the same problem?
Hope to hear from you.
Re: Presence detection for Fritzbox script v1.0: using TR064 protocol communication to your router
Posted: Friday 01 February 2019 23:10
by funky
You can skip step 1 Because fritzconnection is already part of the bundel. You dont need pip , pip is not crashing you have to wait for a very long time, but forget Using pip
Just install the dependencies step 2 like this:
sudo apt-get install jq Python
sudo apt-get install python3.6 LXML
sudo apt-get install python-lxml REQUEST
Sudo apt-get install python-requests and go further with step 3 and so on
I have changed the readme on github. Removed step 1.
Good luck!
Re: Presence detection for Fritzbox script v1.0: using TR064 protocol communication to your router
Posted: Saturday 02 February 2019 17:18
by AllesVanZelf
I tried this presence detection with my Fritzbox 7390, but sometimes a samsung s7 isn't detected and other times an iPhone X isn't detected. I have better results with the arping method found
here
Re: Presence detection for Fritzbox script v1.0: using TR064 protocol communication to your router
Posted: Saturday 02 February 2019 17:31
by gerbenvanasselt
The script part i Managed, The script is talking to my fritzbox 7490.
But now i am struggling with the dependencies. I installed python 3.6.5, but the first dependency (JQ) is saying that it cannot find the package Python. The same with the lxml can't find package 3.6. I don't have a clue, what are the options?
Re: Presence detection for Fritzbox script v1.0: using TR064 protocol communication to your router
Posted: Saturday 02 February 2019 22:58
by funky
gerbenvanasselt wrote: ↑Saturday 02 February 2019 17:31
The script part i Managed, The script is talking to my fritzbox 7490.
But now i am struggling with the dependencies. I installed python 3.6.5, but the first dependency (JQ) is saying that it cannot find the package Python. The same with the lxml can't find package 3.6. I don't have a clue, what are the options?
The problem is that in my first suggestion I made a typo I tested it for you use the following code below:
Code: Select all
sudo apt-get install python jq python-lxml python-requests
and after that
Code: Select all
sudo apt-get update
sudo apt-get upgrade
then everything is updated and it should work now.
edit the script, change parameters and
Start the script with sudo bash presence_detection.sh
Re: Presence detection for Fritzbox script v1.0: using TR064 protocol communication to your router
Posted: Saturday 02 February 2019 23:11
by funky
AllesVanZelf wrote: ↑Saturday 02 February 2019 17:18
I tried this presence detection with my Fritzbox 7390, but sometimes a samsung s7 isn't detected and other times an iPhone X isn't detected. I have better results with the arping method found
here
Do you have a fritzbox extender / repeater? and did you add the extender to this script? When you don't do this and your iPhone is switched to the extender then the script can't detect it anymore because its looking to the fritzbox table. you can add the extender by searching for its ip adres and add it to the script. The script will look for your iPhone on the extender and on the fritzbox and then it should work.
Look for the line: host_extender= (or if you downloaded the newest version of my script look for: host_repeater=)
and put there the ip adress of the extender. For example: host_extender=192.168.178.33 || or host_repeater=192.168.178.33
Go to
http://fritz.repeater, login with same password as fritzbox
and there you see the ip adress of your repeater.
When you enable de debug mode in the script by setting debug=on
Then you see at the end the following lines. here is my example:
---
21: 192.168.178.21 fritz.repeater 7C:FF:4D:24:6C:C4 active
22: 192.168.178.46 iPadvanPatrick 78:FD:94:85:BA:63 -
23: 192.168.178.22 iPhone7GertJan AB:1F:74:41:2C:62 active
24: 192.168.178.27 iPhonevjkshoorn 24:EE:28:C1:EE:B0 active
status router device 1 =
status router device 2 =
status repeater device 1 = 23: 192.168.178.22 iPhone7GertJan AB:1F:74:41:2C:62 active
status repeater device2 = 24: 192.168.178.27 iPhonevjkshoorn 24:EE:28:C1:EE:B0 active
za feb 2 22:58:06 UTC 2019status router is not simular to status domoticz. We change status domoticz for Iphone GJ to On
za feb 2 22:58:06 UTC 2019 status router is not simular to status domoticz. We change status domoticz for Iphone Pat to On
In the above example you see that status router device 1 is empty and status repeater device 1 is filled. In this case both iPhones are found on the repeater and not on the router it self. The script will automatically take the state of the repeater and set it to domoticz
If you have more then 1 repeater, let me know I will change the script for you.
And your suggestion with arp.. I used it before and its the reason I created this script, because when a iPhone goes to screensaver mode arping is not accurate anymore. And this script is using the table of the router and is always accurate even when the iPhone is in screensaver mode
Re: Presence detection for Fritzbox script v1.0: using TR064 protocol communication to your router
Posted: Saturday 02 February 2019 23:32
by funky
Thnx guys for all the feedback. I realize my instructions where a bit messy. I updated the readme.me and the script lines! If you have any problems just post here I will help you (im from the Netherlands (ik zie dat veel van jullie dit ook zijn
.. so if you want to speak dutch you can send me a PM, please do not speak dutch in this forum).
Re: Presence detection for Fritzbox script v1.0: using TR064 protocol communication to your router
Posted: Sunday 03 February 2019 0:09
by AllesVanZelf
And what do I do with the script when I do not have an extender?
I tried to remove the parts of the extender. But maybe it went wrong there.
Re: Presence detection for Fritzbox script v1.0: using TR064 protocol communication to your router
Posted: Sunday 03 February 2019 0:22
by funky
AllesVanZelf wrote: ↑Sunday 03 February 2019 0:09
And what do I do with the script when I do not have an extender?
I tried to remove the parts of the extender. But maybe it went wrong there.
Good point. I will change the script (after this weekend) so that If you leave the repeater empty it will pass the repeater lines in the code.
I already found a problem in the script that may cause your problem. I will fix it and let you know.
Re: Presence detection for Fritzbox script v1.0: using TR064 protocol communication to your router
Posted: Sunday 03 February 2019 16:46
by gerbenvanasselt
I have installed python from you're instructions and all the python dependencies are installed. When i run the script (sudo bash presence_detection.sh) i get the following error. Looked in the scripts but could not find anything in line 113 position 11 that could cause the error. Any suggestions?
There is a list in the output, but the devices that I wan to use for presence detection are not in the list. The list is not complete. I am missing more devices. Is there an explaination for this. I looked into some things, but it is a mixture of 2,4ghz and 5 ghz devices. I miss my oneplus 6 and my wives nokia 6.
Code: Select all
sudo bash presence_detection.sh
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 1894 100 1894 0 0 125k 0 --:--:-- --:--:-- --:--:-- 132k
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 1894 100 1894 0 0 233k 0 --:--:-- --:--:-- --:--:-- 264k
status domoticz device 1 = Off
status domoticz device 2 = Off
Traceback (most recent call last):
File "/home/pi/domoticz/plugins/presence_detection/fritzhosts.py", line 196, in <module>
main()
File "/home/pi/domoticz/plugins/presence_detection/fritzhosts.py", line 193, in main
_print_status(_get_cli_arguments())
File "/home/pi/domoticz/plugins/presence_detection/fritzhosts.py", line 190, in _print_status
print_hosts(fh)
File "/home/pi/domoticz/plugins/presence_detection/fritzhosts.py", line 113, in print_hosts
status,
UnicodeEncodeError: 'ascii' codec can't encode character u'\xa0' in position 11: ordinal not in range(128)
getting output router
Traceback (most recent call last):
File "/home/pi/domoticz/plugins/presence_detection/fritzhosts.py", line 196, in <module>
main()
File "/home/pi/domoticz/plugins/presence_detection/fritzhosts.py", line 193, in main
_print_status(_get_cli_arguments())
File "/home/pi/domoticz/plugins/presence_detection/fritzhosts.py", line 190, in _print_status
print_hosts(fh)
File "/home/pi/domoticz/plugins/presence_detection/fritzhosts.py", line 103, in print_hosts
hosts = fh.get_hosts_info()
File "/home/pi/domoticz/plugins/presence_detection/fritzhosts.py", line 77, in get_hosts_info
while index < self.host_numbers:
File "/home/pi/domoticz/plugins/presence_detection/fritzhosts.py", line 59, in host_numbers
result = self.action('GetHostNumberOfEntries')
File "/home/pi/domoticz/plugins/presence_detection/fritzhosts.py", line 51, in action
return self.fc.call_action(SERVICE, actionname, **kwargs)
File "/home/pi/domoticz/plugins/presence_detection/fritzconnection.py", line 405, in call_action
action = self._get_action(service_name, action_name)
File "/home/pi/domoticz/plugins/presence_detection/fritzconnection.py", line 378, in _get_action
raise ServiceError('Unknown Service: ' + service_name)
fritzconnection.ServiceError: Unknown Service: Hosts:1
output router devices =
FritzHosts:
version: 0.6.2
model: FRITZ!Box 7490
ip: 192.168.178.1
Re: Presence detection for Fritzbox script v1.0: using TR064 protocol communication to your router
Posted: Sunday 03 February 2019 21:48
by funky
gerbenvanasselt wrote: ↑Sunday 03 February 2019 16:46
I have installed python from you're instructions and all the python dependencies are installed. When i run the script (sudo bash presence_detection.sh) i get the following error. Looked in the scripts but could not find anything in line 113 position 11 that could cause the error. Any suggestions?
- sudo bash presence_detection.sh
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 1894 100 1894 0 0 125k 0 --:--:-- --:--:-- --:--:-- 132k
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 1894 100 1894 0 0 233k 0 --:--:-- --:--:-- --:--:-- 264k
status domoticz device 1 = Off
status domoticz device 2 = Off
This is an authentication problem. Or the ip adres of the router is not correct or your password is not correct. Use the password of the fritzbox browser login. If you didn't change the password its located on the fritzbox itself.
you can quickly check to do the following:
sudo python fritzhosts.py -i <here your ip adress of your router> -p <here your password of your router>
for example sudo python fritzhosts.py -i 192.168.178.4 -pmypassword
If it works then change the settings inside presence detection.sh
Re: Presence detection for Fritzbox script v1.0: using TR064 protocol communication to your router
Posted: Sunday 03 February 2019 21:56
by gerbenvanasselt
[/quote]
This is an authentication problem. Or the ip adres of the router is not correct or your password is not correct. Use the password of the fritzbox browser login. If you didn't change the password its located on the fritzbox itself.
you can quickly check to do the following:
sudo python fritzhosts.py -I <here your ip adress of your router> -p <here your password of your router>
for example sudo python fritzhosts.py -i 192.168.178.4 -pmypassword
If it works then change the settings inside presence detection.sh
[/quote]
There is a list in the output, but the devices that I want to use for presence detection are not in the list. The list is not complete. I am missing more devices. Is there an explaination for this. I looked into some things, but it is a mixture of 2,4ghz and 5 ghz devices. I miss my oneplus 6 and my wives nokia 6.
Code: Select all
FritzHosts:
version: 0.6.2
model: FRITZ!Box 7490
ip: 192.168.178.1
List of registered hosts:
n: ip name mac status
0: 192.168.178.26 Chromecast :77:33:C0:83:CE active
1: 192.168.178.22 GENM-PC--GenM :1A:04:FF:C2:9F active
2: 192.168.178.59 GW-A0C9A0D63E89 A0:C9:A0:D6:3E:89 active
3: 192.168.178.62 Galaxy-Tab-A-2016 :B5:D0:54:51:0F active
4: 192.168.178.63 Galaxy-Tab-A-2016 :B1:0F:41:E6:88 active
5: 192.168.178.61 Google-Home-Mini :07:0B:50:C6:72 active
6: 192.168.178.41 HP9F655E :4A:3E:9F:65:5E active
7: 192.168.178.36 HTC-One-A9 :01:84:D4:22:41 -
8: 192.168.178.25 HTC-One-M7 :7A:88:34:79:7A -
9: 192.168.178.21 HenK-BDS280 :1C:50:22:C5:1B -
Re: Presence detection for Fritzbox script v1.0: using TR064 protocol communication to your router
Posted: Sunday 03 February 2019 21:58
by funky
do you have a repeater (extender)? goto
http://fritz.repeater
if you have a repeater look for its ip adress and run fritzhosts again with the repeaters ip adres.. Password is the same as the fritzbox.. Are your devices there?
Re: Presence detection for Fritzbox script v1.0: using TR064 protocol communication to your router
Posted: Sunday 03 February 2019 22:04
by gerbenvanasselt
No, I don't have a repeater just the fritz 7490 router
When i place my computer's IP in it it works, but the phones are not detected.