Presence detection for Fritzbox script v2.0: using TR064 protocol communication to your router

All kinds of 'OS' scripts

Moderator: leecollings

TeamKleijn
Posts: 13
Joined: Sunday 05 January 2020 13:03
Target OS: -
Domoticz version:
Contact:

Re: Presence detection for Fritzbox script v2.0: using TR064 protocol communication to your router

Post by TeamKleijn »

Hello All,

II've tried to install the presence_detection_domoticz_fritzbox.sh from funky, but i can't get it worked...
- As described in the procedure the install done at the Raspberry.
- Start the script, it asks the Fritzbox IP, the Repeater IP, password and the devices to monitor.
- The hardware is created in domoticz.
- In the devices screen the devices appear, wich i've moved to my Switches screen (Green arrow)
- After that change the CRON job file with crontab -e and paste the line witch given by ending the script.

If i test with sudo bash presence_detection.sh debug, i'll see the script is working, the Fritz router gives the information for the devices the Domoticz settings chaged to the actual Fritzbox status, so far it works but it does not update automatically...

so i'll checked the CRON job,
first ps -eaf | grep cron
Output > root 1369 1 0 00:36 ? 00:00:01 /usr/sbin/cron -f
pi 14901 14877 0 14:07 pts/0 00:00:00 grep --color=auto cron
Ok, (Or not ?)

After that tail /var/log/auth.log
Output >
Jan 5 14:07:23 raspberrypi sshd[14862]: pam_unix(sshd:session): session opened for user pi by (uid=0)
Jan 5 14:07:23 raspberrypi systemd-logind[298]: New session c6 of user pi.
Jan 5 14:08:01 raspberrypi CRON[14911]: pam_unix(cron:session): session opened for user pi by (uid=0)
Jan 5 14:08:01 raspberrypi CRON[14912]: pam_unix(cron:session): session opened for user root by (uid=0)
Jan 5 14:08:01 raspberrypi CRON[14912]: pam_unix(cron:session): session closed for user root
Jan 5 14:08:01 raspberrypi CRON[14911]: pam_unix(cron:session): session closed for user pi
Jan 5 14:09:01 raspberrypi CRON[14981]: pam_unix(cron:session): session opened for user pi by (uid=0)
Jan 5 14:09:01 raspberrypi CRON[14982]: pam_unix(cron:session): session opened for user root by (uid=0)
Jan 5 14:09:01 raspberrypi CRON[14981]: pam_unix(cron:session): session closed for user pi
Jan 5 14:09:01 raspberrypi CRON[14982]: pam_unix(cron:session): session closed for user root
Seems to be fine (or it's a little bit to fast ?)

and then tail /var/log/auth.log
Output >
Jan 5 14:07:23 raspberrypi sshd[14862]: pam_unix(sshd:session): session opened for user pi by (uid=0)
Jan 5 14:07:23 raspberrypi systemd-logind[298]: New session c6 of user pi.
Jan 5 14:08:01 raspberrypi CRON[14911]: pam_unix(cron:session): session opened for user pi by (uid=0)
Jan 5 14:08:01 raspberrypi CRON[14912]: pam_unix(cron:session): session opened for user root by (uid=0)
Jan 5 14:08:01 raspberrypi CRON[14912]: pam_unix(cron:session): session closed for user root
Jan 5 14:08:01 raspberrypi CRON[14911]: pam_unix(cron:session): session closed for user pi
Jan 5 14:09:01 raspberrypi CRON[14981]: pam_unix(cron:session): session opened for user pi by (uid=0)
Jan 5 14:09:01 raspberrypi CRON[14982]: pam_unix(cron:session): session opened for user root by (uid=0)
Jan 5 14:09:01 raspberrypi CRON[14981]: pam_unix(cron:session): session closed for user pi
Jan 5 14:09:01 raspberrypi CRON[14982]: pam_unix(cron:session): session closed for user root
pi@raspberrypi:~ $ tail /var/log/syslog
Jan 5 14:08:01 raspberrypi CRON[14912]: (CRON) info (No MTA installed, discarding output)
Jan 5 14:08:01 raspberrypi CRON[14911]: (CRON) info (No MTA installed, discarding output)
Jan 5 14:09:01 raspberrypi CRON[14990]: (pi) CMD (/home/pi/presence_detection_domoticz_fritzbox/presence_detection.sh >> /home/pilogs/presence2.log)
Jan 5 14:09:01 raspberrypi CRON[14981]: (CRON) info (No MTA installed, discarding output)
Jan 5 14:09:01 raspberrypi CRON[14991]: (root) CMD (/home/pi/presence_detection_domoticz_fritzbox/presence_detection.sh)
Jan 5 14:09:01 raspberrypi CRON[14982]: (CRON) info (No MTA installed, discarding output)
Jan 5 14:10:01 raspberrypi CRON[15044]: (root) CMD (/home/pi/presence_detection_domoticz_fritzbox/presence_detection.sh)
Jan 5 14:10:01 raspberrypi CRON[15045]: (pi) CMD (/home/pi/presence_detection_domoticz_fritzbox/presence_detection.sh >> /home/pilogs/presence2.log)
Jan 5 14:10:01 raspberrypi CRON[15036]: (CRON) info (No MTA installed, discarding output)
Jan 5 14:10:01 raspberrypi CRON[15037]: (CRON) info (No MTA installed, discarding output)
Ok, no MTA that's correct but the command is executed every minute ....

But there is also no log created at /home/pilogs/presence2.log :roll:
I think the CRONjob does nothing what i'm doing wrong ?

What to do now ?
Hopely someone can help me with the solving this problem

Thanks a lot in advantage

Rob (Team Kleijn)
percy159
Posts: 2
Joined: Thursday 17 October 2019 13:15
Target OS: Raspberry Pi / ODroid
Domoticz version:
Contact:

Re: Presence detection for Fritzbox script v2.0: using TR064 protocol communication to your router

Post by percy159 »

TeamKleijn wrote: Sunday 05 January 2020 14:15
- After that change the CRON job file with crontab -e and paste the line witch given by ending the script.

If i test with sudo bash presence_detection.sh debug, i'll see the script is working, the Fritz router gives the information for the devices the Domoticz settings chaged to the actual Fritzbox status, so far it works but it does not update automatically...

Rob (Team Kleijn)
Hey Rob,

i think i had the same issue.
You need to run 'cronetab -e' without 'sudo'.
Otherwise you write the task for the user "root" and not for the 'normal' user.

Kind regards,
percy
funky
Posts: 74
Joined: Saturday 06 May 2017 22:38
Target OS: Raspberry Pi / ODroid
Domoticz version: newest
Location: Zeist, Netherlands
Contact:

Re: Presence detection for Fritzbox script v2.0: using TR064 protocol communication to your router

Post by funky »

Yep use crontab -e
funky
Posts: 74
Joined: Saturday 06 May 2017 22:38
Target OS: Raspberry Pi / ODroid
Domoticz version: newest
Location: Zeist, Netherlands
Contact:

Re: Presence detection for Fritzbox script v2.0: using TR064 protocol communication to your router

Post by funky »

When you want to protect your domoticz with username and password fill in at local network 127.0.0.1
Then only your raspberry pi has access to domoticz and the rest is behind username / password.
The script will work with these settings
TeamKleijn
Posts: 13
Joined: Sunday 05 January 2020 13:03
Target OS: -
Domoticz version:
Contact:

Re: Presence detection for Fritzbox script v2.0: using TR064 protocol communication to your router

Post by TeamKleijn »

percy159 wrote: Monday 06 January 2020 12:22 Hey Rob,

i think i had the same issue.
You need to run 'cronetab -e' without 'sudo'.
Otherwise you write the task for the user "root" and not for the 'normal' user.

Kind regards,
percy
Ok Percy thanks a lot, at the moment i've all de-installed but it's installed and tested quickly
I'll inform here the results of my testing experience (and if good i'll put it in production)

"Team Kleijn"
Rob
Cootjec
Posts: 1
Joined: Sunday 02 February 2020 23:04
Target OS: Raspberry Pi / ODroid
Domoticz version: 4.10717
Location: NL
Contact:

Re: Presence detection for Fritzbox script v2.0: using TR064 protocol communication to your router

Post by Cootjec »

Thank you all, and special to Funky for this script. It all works for me, but I have a question, I am running the script and I have 1 fritz.box and 2 repeaters. I gave both the 2 IP adresses of the repeaters in the confix.txt file of the repeater, Is this enough to get it working with both repeaters and the fritz.box router ?
Hielkep
Posts: 11
Joined: Monday 28 January 2019 9:29
Target OS: Raspberry Pi / ODroid
Domoticz version: BETA
Contact:

Re: Presence detection for Fritzbox script v2.0: using TR064 protocol communication to your router

Post by Hielkep »

I installed the script and i got it working good.
i had 1 problem, where presence detection was not working correctly.
i have a fritzbox 7590, fritzrepeater 3000 and fritzpowerline 1660e in wifi mesh, with lates firmware (7.13 on box 7.12 on rest)
initially i choose to add the ip of the repeater 3000, but detection was hit and miss.
in debug i saw that absent clients where present in the repeater 3000 but not in fritzbox. also not on powerline 1660.
website of the fritsbox showed client absent.
when i deleted the ip address of the repeaters, everything is fine.

so my guess is when you have wifi mesh, only ip of fritzbox is needed
TeamKleijn
Posts: 13
Joined: Sunday 05 January 2020 13:03
Target OS: -
Domoticz version:
Contact:

Re: Presence detection for Fritzbox script v2.0: using TR064 protocol communication to your router

Post by TeamKleijn »

percy159 wrote: Monday 06 January 2020 12:22
Ok Percy thanks a lot, at the moment i've all de-installed but it's installed and tested quickly
I'll inform here the results of my testing experience (and if good i'll put it in production)

"Team Kleijn"
Rob
I'm stopped. At now i've the FritzBox 7581 running with 2 repeaters in mesh. I'm back to the standard presence detection in Domoticz.
During test some devices are presented as their real sitauation, others does not.

"Team Kleijn"
Rob
Hielkep
Posts: 11
Joined: Monday 28 January 2019 9:29
Target OS: Raspberry Pi / ODroid
Domoticz version: BETA
Contact:

Re: Presence detection for Fritzbox script v2.0: using TR064 protocol communication to your router

Post by Hielkep »

TeamKleijn wrote: Saturday 06 June 2020 16:57
percy159 wrote: Monday 06 January 2020 12:22
Ok Percy thanks a lot, at the moment i've all de-installed but it's installed and tested quickly
I'll inform here the results of my testing experience (and if good i'll put it in production)

"Team Kleijn"
Rob
I'm stopped. At now i've the FritzBox 7581 running with 2 repeaters in mesh. I'm back to the standard presence detection in Domoticz.
During test some devices are presented as their real sitauation, others does not.

"Team Kleijn"
Rob
do you have a mesh network?
maybee try with only the ipadress of the main router and not of the repeaters. that helped in my case
AllesVanZelf
Posts: 265
Joined: Monday 05 February 2018 8:42
Target OS: Raspberry Pi / ODroid
Domoticz version: 12467
Location: Netherlands, near Haarlem
Contact:

Re: Presence detection for Fritzbox script v2.0: using TR064 protocol communication to your router

Post by AllesVanZelf »

Hi, The script was running for a long time, but now I notice that is giving an error:

Code: Select all

checking:Thuis-B
parse error: Invalid numeric literal at line 1, column 60
ma 22 jun 2020 16:20:24 UTCstatus router is not simular to status domoticz.  We change status domoticz for Thuis-B to  On
I've seen this error before on page 3, by felix63, but I do not see the cause of this error.
And I did some tests, but the script is not working. No update of domoticz. Does anyone know the reason for this?

edit:
When I change line 73:

Code: Select all

status_domoticz_device=$(curl -s 'http://'$ip_domoticz'/json.htm?type=devices&rid='${device_idx[i]} | jq -r [.result][][].Data)
to:

Code: Select all

status_domoticz_device=&(curl -s http://127.0.0.1:8090/json.htm?type=devices&rid=70 | jq -r [.result][][].Data )
I get error/output:

Code: Select all

<html><head><title>Unauthorized</title></head><body><h1>401 Unauthorized</h1></body></html>
I notice that when debug is on, domoticz part is empty:

Code: Select all

./presence_detection.sh 
checking:Thuis-B
parse error: Invalid numeric literal at line 1, column 60
di 23 jun 2020 11:22:34 UTCstatus router is not simular to status domoticz.  We change status domoticz for Thuis-B to  Off
debug information:
status domoticz:

-----------------------
status fritzbox:

FritzHosts:
version:            0.6.2
model:              FRITZ!Box 7490
ip:                 192.168.188.1

Details for host:      XX:XX:XX:XX:XX:XX

NewActive              : 0
NewIPAddress           : 192.168.188.61
NewInterfaceType       : None
NewHostName            : Galaxy-S9
NewAddressSource       : DHCP
NewLeaseTimeRemaining  : 0
-----------------------
status repeater:

-----------------------
edit:
When I paste:

Code: Select all

http://192.168.188.80:8090/json.htm?type=command&param=switchlight&idx=70&switchcmd=Off
in the browser on a client computer, I'm able to switch the dummy. But when I place this command in terminal on the raspberry:

Code: Select all

wget -q http://192.168.188.80:8090/json.htm?type=command&param=switchlight&idx=70&switchcmd=On
(Or 'Off'). I get

Code: Select all

[7] 32229
[8] 32230
[9] 32231
[4]   Exit 6                  wget -q http://127.0.0.1:8090/json.htm?type=command
[5]   Klaar                   param=switchlight
[6]   Klaar                   idx=70
[8]-  Klaar                   param=switchlight
And the dummy is not switched.

$#@^% I solved it! Because this seemed to be a permission error from 127.0.0.1 I checked the settings in domoticz settings: Local Network.
It was set to 127.0.0.1. I changed it to 127.0.0.* and saved. Then presence detection was working again. I'm happy now.
Domoticz 2020.1 (12230) on Raspberry Pi 3B with Raspian Buster. Besides Domoticz, Rpi is running Pi-Hole.
sundancekid
Posts: 2
Joined: Sunday 26 July 2020 13:18
Target OS: Raspberry Pi / ODroid
Domoticz version:
Contact:

Re: Presence detection for Fritzbox script v2.0: using TR064 protocol communication to your router

Post by sundancekid »

Good Day,
I'm new to this, so I hope you folks forgive me stupid questions.

I have to say overall that thing works great. The installer is good it worked like a charm. :D

- One little hurdle was the repeater. I use mine in MESH mode, however I had to remove it to be able to get the devices detected. With the Repeater in always had status "ON". From what I understood in debug mode it took the repeater and took that.

- Another thing that people may want to add would be MQTT, I think it could be interesting to add two lines for MQTT, to be able to push the device status in MQTT.

What are your thoughts?
best regards sundanceKid
sundancekid
Posts: 2
Joined: Sunday 26 July 2020 13:18
Target OS: Raspberry Pi / ODroid
Domoticz version:
Contact:

Re: Presence detection for Fritzbox script v2.0: using TR064 protocol communication to your router

Post by sundancekid »

Made a quick and dirty test, and wit works.

It quite nice if you add the the publish to your SHELL.
Now it updates the device status in MQTT on every run of the cronjob.
Best regards max
User avatar
erem
Posts: 230
Joined: Tuesday 27 March 2018 12:11
Target OS: Raspberry Pi / ODroid
Domoticz version: 2021.1
Location: Amsterdam/netherlands
Contact:

Re: Presence detection for Fritzbox script v2.0: using TR064 protocol communication to your router

Post by erem »

installed the latest version to check this out.

I cannot get this to work correctly with my fritz box setup.

The phone Mariannes-iPhone is not active (away), yet it is seen as active.
I checked both router and repeater, and both show the device in the inactive connections.

the software works, if I manually switch off the phone in Domoticz it is switched on again by the software.

the repeater is connected to the router as a lan bridge, not via mesh.
router: FRITZ!OS: 07.13 - Version up to date
repeater: FRITZ!OS: 07.20 - Version up to date

what can i do to make this work?

debug log
Spoiler: show
------------------------------------------------------------
Presence detection for Domoticz using Fritzbox version 2.3
------------------------------------------------------------

checking:MULLERBs-iPhone
2020-09-24T13:31:06 UTC status router and domoticz for MULLERBs-iPhone are simular, we do nothing
debug information:
status domoticz:
On
-----------------------
status fritzbox:

FritzHosts:
version: 0.6.2
model: FRITZ!Box 7581
ip: 10.0.0.138

Details for host: 48:3B:38:15:8F:36

NewIPAddress : 10.0.0.169
NewAddressSource : DHCP
NewLeaseTimeRemaining : 1123200
NewInterfaceType : 802.11
NewActive : 1
NewHostName : iPhone6Rob
-----------------------
status repeater:

FritzHosts:
version: 0.6.2
model: FRITZ!WLAN Repeater 1750E
ip: 10.0.0.152

Details for host: 48:3B:38:15:8F:36

NewIPAddress : 10.0.0.169
NewAddressSource : None
NewLeaseTimeRemaining : 0
NewInterfaceType : 802.11
NewActive : 1
NewHostName : PC-10-0-0-169
-----------------------
checking:Mariannes-iPhone
2020-09-24T13:31:12 UTC status router and domoticz for Mariannes-iPhone are simular, we do nothing
debug information:
status domoticz:
On
-----------------------
status fritzbox:

FritzHosts:
version: 0.6.2
model: FRITZ!Box 7581
ip: 10.0.0.138

Details for host: 60:7E:C9:0F:92:01

NewIPAddress : 10.0.0.180
NewAddressSource : DHCP
NewLeaseTimeRemaining : 1123200
NewInterfaceType : None
NewActive : 0
NewHostName : iPhonevMarianne
-----------------------
status repeater:

FritzHosts:
version: 0.6.2
model: FRITZ!WLAN Repeater 1750E
ip: 10.0.0.152

Details for host: 60:7E:C9:0F:92:01

NewIPAddress : None
NewAddressSource : None
NewLeaseTimeRemaining : 0
NewInterfaceType : Ethernet
NewActive : 1
NewHostName : PC-60-7E-C9-0F-92-01
-----------------------
dependencies installed:
installed python version:
Python 3.8.2
installed JQ version:
jq-1.6
---------------------

config file variables:
---------------------
ip_fritzbox=10.0.0.138
pass_fritzbox=xxxxxxxx
ip_repeater=10.0.0.152
ip_domoticz=10.0.0.167:8080
domoticzpasscode=
device_names=(MULLERBs-iPhone Mariannes-iPhone)
device_macs=(48:3B:38:15:8F:36 60:7E:C9:0F:92:01)
device_idx=(2 3)
---------------------
Regards,

Rob
funky
Posts: 74
Joined: Saturday 06 May 2017 22:38
Target OS: Raspberry Pi / ODroid
Domoticz version: newest
Location: Zeist, Netherlands
Contact:

Re: Presence detection for Fritzbox script v2.0: using TR064 protocol communication to your router

Post by funky »

Hi rob
do you have installed IOS14? since IOS 14 there is a privacy feature which gives your iPhone a random Mac address so tracking it is impossible to track your iPhone at stores. Since this script is using the Mac adres to figure out if the phone is online / not online you have to switch this out.

go to settings > wifi > select (i) next to your wifi network and deselect private adress.

check the Mac adres en change it in the settings of the config file.

Maybe the problem is within the fact you use the repeater as a lan bridge. Can you try it with the repeater disconnected? Does it work then?


if you still have issues please send a PM (ik kom uit Nederland, dus dat kan gewoon in het nederlands)

your sincerely

Gert-Jan (maker of this script)
User avatar
erem
Posts: 230
Joined: Tuesday 27 March 2018 12:11
Target OS: Raspberry Pi / ODroid
Domoticz version: 2021.1
Location: Amsterdam/netherlands
Contact:

Re: Presence detection for Fritzbox script v2.0: using TR064 protocol communication to your router

Post by erem »

@gert-jan

ios 13.6.1, so no issue there.

i traced the problem back to fritz.os 7.13
if i place my phone in airplane mode, the repeater recognizes that and shows

Code: Select all

D:\test>python ./fritzhosts.py -i 10.0.0.152 -p "#System1" -d 48:3B:38:15:8F:36

FritzHosts:
version:            0.6.2
model:              FRITZ!WLAN Repeater 1750E
ip:                 10.0.0.152

Details for host:      48:3B:38:15:8F:36

NewIPAddress           : 10.0.0.169
NewAddressSource       : None
NewLeaseTimeRemaining  : 0
NewInterfaceType       : 802.11
NewActive              : 0
NewHostName            : PC-10-0-0-169
however, fb 7581 still shows

Code: Select all

D:\test>python ./fritzhosts.py -i 10.0.0.138 -p "#System1" -d 48:3B:38:15:8F:36

FritzHosts:
version:            0.6.2
model:              FRITZ!Box 7581
ip:                 10.0.0.138

Details for host:      48:3B:38:15:8F:36

NewIPAddress           : 10.0.0.169
NewAddressSource       : DHCP
NewLeaseTimeRemaining  : 1123200
NewInterfaceType       : Ethernet
NewActive              : 1
NewHostName            : iPhone6Rob
nothing to do with your script. just fritz.os ignoring status on the repeater.
and, since the 7581 is out of maintenance, little hope for a fix.
Regards,

Rob
funky
Posts: 74
Joined: Saturday 06 May 2017 22:38
Target OS: Raspberry Pi / ODroid
Domoticz version: newest
Location: Zeist, Netherlands
Contact:

Re: Presence detection for Fritzbox script v2.0: using TR064 protocol communication to your router

Post by funky »

Ok sometimes there is a little bit of lag on the router. You can try to let the script run with more delay on the crontab
wkossen
Posts: 62
Joined: Friday 06 November 2020 12:12
Target OS: Raspberry Pi / ODroid
Domoticz version:
Contact:

Re: Presence detection for Fritzbox script v2.0: using TR064 protocol communication to your router

Post by wkossen »

I'm happy i found this script. it works much better than other methods i tried so far.

i do have 2 repeaters though, and they are set up in a mesh configuration with the fritzbox. i noticed that one device is not detected, and it is connected via the repeater that is not in the config :). i guess i need two repeaters in the script. how can i add one?

Also, in this 6 page thread i read someone doing some blockly to get a 'anybodyhome' switch working. i tried that, but it doesn't work. so any pointers on that would be great. i think i have my ands and ors right but the device doesn't switch to on with several of the devices present.

Code: Select all

if anybodyhome=off and dev1=on or dev2=on or dev3=on or dev4=on or dev5=on then set anybodyhome=on
else if anybodyhome=on and dev1=off and dev2=off etc set anybodyhome=off.
thanks
Willem
funky
Posts: 74
Joined: Saturday 06 May 2017 22:38
Target OS: Raspberry Pi / ODroid
Domoticz version: newest
Location: Zeist, Netherlands
Contact:

Re: Presence detection for Fritzbox script v2.0: using TR064 protocol communication to your router

Post by funky »

Hey Willem

The script doesn’t support more then 1 repeater but at the end of dec i will make time free to add this feature can you help with testing?
wkossen
Posts: 62
Joined: Friday 06 November 2020 12:12
Target OS: Raspberry Pi / ODroid
Domoticz version:
Contact:

Re: Presence detection for Fritzbox script v2.0: using TR064 protocol communication to your router

Post by wkossen »

Of course I'm willing to test it. yes please. ping me when your ready :)
funky
Posts: 74
Joined: Saturday 06 May 2017 22:38
Target OS: Raspberry Pi / ODroid
Domoticz version: newest
Location: Zeist, Netherlands
Contact:

Re: Presence detection for Fritzbox script v2.0: using TR064 protocol communication to your router

Post by funky »

I will :-)

About your question for if anybody is home.

Make a manually on off virtual switch: Anybody home

Make 2 blocky events like this:

event name: somebody is home
blockly:
IF <name phone 1> = on OR <name phone2> = on
DO set anybody home = on

event name: nobody is home
Blocky:
IF <name phone 1> = off AND <name phone2> = off
DO set anybody home = off
Post Reply

Who is online

Users browsing this forum: No registered users and 0 guests