Python plugin: Presence detection from wireless router

[EscApe]

From memory (I am not at a computer right now): can you run ifconfig on the router and share the output?

[Melotron]

From memory (I am not at a computer right now): can you run ifconfig on the router and share the output?

Code: Select all

admin@RT-AX88U-97B0:/tmp/home/root# ifconfig
br0       Link encap:Ethernet  HWaddr 04:D9:F5:FA:97:B0
          inet addr:192.168.1.1  Bcast:192.168.1.255  Mask:255.255.255.0
          UP BROADCAST RUNNING ALLMULTI MULTICAST  MTU:1500  Metric:1
          RX packets:21820449 errors:0 dropped:142 overruns:0 frame:0
          TX packets:73912868 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:1550169284 (1.4 GiB)  TX bytes:107151329796 (99.7 GiB)

eth0      Link encap:Ethernet  HWaddr 04:D9:F5:FA:97:B0
          inet addr:98.128.172.49  Bcast:98.128.175.255  Mask:255.255.252.0
          UP BROADCAST RUNNING ALLMULTI MULTICAST  MTU:1500  Metric:1
          RX packets:37631574 errors:0 dropped:0 overruns:0 frame:0
          TX packets:21319157 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:54283757532 (50.5 GiB)  TX bytes:1648464670 (1.5 GiB)

eth1      Link encap:Ethernet  HWaddr 04:D9:F5:FA:97:B0
          UP BROADCAST RUNNING ALLMULTI MULTICAST  MTU:1500  Metric:1
          RX packets:10 errors:0 dropped:24 overruns:0 frame:0
          TX packets:1533 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:640 (640.0 B)  TX bytes:294285 (287.3 KiB)

eth2      Link encap:Ethernet  HWaddr 04:D9:F5:FA:97:B0
          UP BROADCAST ALLMULTI MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

eth3      Link encap:Ethernet  HWaddr 04:D9:F5:FA:97:B0
          UP BROADCAST ALLMULTI MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

eth4      Link encap:Ethernet  HWaddr 04:D9:F5:FA:97:B0
          UP BROADCAST RUNNING ALLMULTI MULTICAST  MTU:1500  Metric:1
          RX packets:34393958 errors:0 dropped:6 overruns:0 frame:0
          TX packets:34802352 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:7801058303 (7.2 GiB)  TX bytes:10464656688 (9.7 GiB)

eth5      Link encap:Ethernet  HWaddr 04:D9:F5:FA:97:B0
          UP BROADCAST RUNNING ALLMULTI MULTICAST  MTU:1500  Metric:1
          RX packets:53870463 errors:0 dropped:3541 overruns:0 frame:0
          TX packets:67846957 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:11913517660 (11.0 GiB)  TX bytes:57823937029 (53.8 GiB)

eth6      Link encap:Ethernet  HWaddr 04:D9:F5:FA:97:B0
          UP BROADCAST RUNNING ALLMULTI MULTICAST  MTU:1500  Metric:1
          RX packets:792065 errors:0 dropped:16 overruns:0 frame:0
          TX packets:1201017 errors:0 dropped:6597 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:128861195 (122.8 MiB)  TX bytes:1353468883 (1.2 GiB)

eth7      Link encap:Ethernet  HWaddr 04:D9:F5:FA:97:B4
          UP BROADCAST RUNNING ALLMULTI MULTICAST  MTU:1500  Metric:1
          RX packets:1314190 errors:0 dropped:15 overruns:0 frame:0
          TX packets:1450329 errors:0 dropped:6296 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:683730283 (652.0 MiB)  TX bytes:1790791884 (1.6 GiB)

lo        Link encap:Local Loopback
          inet addr:127.0.0.1  Mask:255.0.0.0
          UP LOOPBACK RUNNING MULTICAST  MTU:65536  Metric:1
          RX packets:669926 errors:0 dropped:0 overruns:0 frame:0
          TX packets:669926 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:157341381 (150.0 MiB)  TX bytes:157341381 (150.0 MiB)

spu_ds_dummy Link encap:UNSPEC  HWaddr 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00
          UP RUNNING NOARP  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:100
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

spu_us_dummy Link encap:UNSPEC  HWaddr 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00
          UP RUNNING NOARP  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:100
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

admin@RT-AX88U-97B0:/tmp/home/root#

[HvdW]

I'm just reading this post out of curiosity.
What do you think of using fing for detection.
Most reliable detection so far that I have found.
The user guide.

Install it on your Android phone as well.

[EscApe]

@Melotron
It looks like the wireless interfaces are named eth*, so that should not be a problem. I don’t know why the wl command is not able to query them. Using the #type=forcegeneric configuration is a work-around. More research is needed to find the correct way to find connected clients on a ax88u router. The architecture might be totally different from other ASUS routers. Maybe you can find something on the web or experiment on the router itself? I will also hit google when i have some extra time.

[EscApe]

I'm just reading this post out of curiosity.
What do you think of using fing for detection.
Most reliable detection so far that I have found.
The user guide.

Looks like another network discovery tool. I don’t know it but it might have the same downsides as using ping (higher battery drain on wireless devices) and other discovery tools (slower to respond, high system and network utilization). It looks nice, but might be overkill for this purpose. However, you could write a tracker for it if you want this as the source for the idetect plugin.

Install it on your Android phone as well.

This plugin is for domoticz, so I don’t understand the relevance of installing it on an android phone.

[Melotron]

@Melotron
It looks like the wireless interfaces are named eth*, so that should not be a problem. I don’t know why the wl command is not able to query them. Using the #type=forcegeneric configuration is a work-around. More research is needed to find the correct way to find connected clients on a ax88u router. The architecture might be totally different from other ASUS routers. Maybe you can find something on the web or experiment on the router itself? I will also hit google when i have some extra time.

Thanks alit! That helped.

Im going to make a few other tests on the router to see if I can it to work without the generic switch.
But Im no expert on it

[hestia]

I've been using the very good plugin for a while with 2 ASUS RT-AC68U and it is ok.
I've added one more and now one of the 3 doesn't work

Error: (iDetect) Failed 3 times to get capabilities for 192.168.1.5

I've checked and they have the same config (for what I could see in the GUI)

The log with debug. I could seen nothing...

Spoiler: show

2020-03-29 21:57:30.316 Status: (iDetect) Started.
2020-03-29 21:57:30.900 Status: (iDetect) Entering work loop.
2020-03-29 21:57:30.901 Status: (iDetect) Initialized version 0.7.7, author 'ESCape'
2020-03-29 21:57:34.220 Status: (iDetect) Using chipset specific wl command on router 192.168.1.2 for interfaces eth1 & eth2 (=wl eth1 eth2)
2020-03-29 21:57:35.866 Status: (iDetect) Using chipset specific wl command on router 192.168.1.4 for interfaces eth1 & eth2 (=wl eth1 eth2)
2020-03-29 21:57:49.329 Status: (iDetect) Awaiting confirmation on absence of 4 devices
2020-03-29 21:58:22.480 Status: (iDetect) Awaiting confirmation on absence of 4 devices
2020-03-29 21:58:54.230 Error: (iDetect) Failed 3 times to get capabilities for 192.168.1.5. Retry interval automatically reduced for this router.

I could connect with SSH
Any check to do?

[EscApe]

I've been using the very good plugin for a while with 2 ASUS RT-AC68U and it is ok.
I've added one more and now one of the 3 doesn't work

Error: (iDetect) Failed 3 times to get capabilities for 192.168.1.5

I've checked and they have the same config (for what I could see in the GUI)

The log with debug. I could seen nothing...

Spoiler: show

2020-03-29 21:57:30.316 Status: (iDetect) Started.
2020-03-29 21:57:30.900 Status: (iDetect) Entering work loop.
2020-03-29 21:57:30.901 Status: (iDetect) Initialized version 0.7.7, author 'ESCape'
2020-03-29 21:57:34.220 Status: (iDetect) Using chipset specific wl command on router 192.168.1.2 for interfaces eth1 & eth2 (=wl eth1 eth2)
2020-03-29 21:57:35.866 Status: (iDetect) Using chipset specific wl command on router 192.168.1.4 for interfaces eth1 & eth2 (=wl eth1 eth2)
2020-03-29 21:57:49.329 Status: (iDetect) Awaiting confirmation on absence of 4 devices
2020-03-29 21:58:22.480 Status: (iDetect) Awaiting confirmation on absence of 4 devices
2020-03-29 21:58:54.230 Error: (iDetect) Failed 3 times to get capabilities for 192.168.1.5. Retry interval automatically reduced for this router.

I could connect with SSH
Any check to do?

The first things that come to mind are:
- Is SSH enabled on the 192.168.1.5 router
- If you are using key-based login, did you check if it works under the user profile that is running Domoticz (root?) for 192.168.1.5.

A debug log might give you some more insights.

[hestia]

thanks for your reply

from domoticz's server I can do this:

$ ssh [email protected]
[email protected]'s password:
admin@RT-AC68U-FC88-Salon:/tmp/home/root# cd /
admin@RT-AC68U-FC88-Salon:/# ls
bin cifs2 etc jffs media mnt proc root sys tmp var
cifs1 dev home lib mmc opt rom sbin sysroot usr www

So I think ssh is ok on the router

I'm not using a key-based login, but pwd in domoticz

I've checked the debug mode(for the plugin) but nothing more, anything else to do to have more log?

[EscApe]

thanks for your reply

from domoticz's server I can do this:

$ ssh [email protected]
[email protected]'s password:
admin@RT-AC68U-FC88-Salon:/tmp/home/root# cd /
admin@RT-AC68U-FC88-Salon:/# ls
bin cifs2 etc jffs media mnt proc root sys tmp var
cifs1 dev home lib mmc opt rom sbin sysroot usr www

So I think ssh is ok on the router

I'm not using a key-based login, but pwd in domoticz

I've checked the debug mode(for the plugin) but nothing more, anything else to do to have more log?

If you have Domoticz setup so it doesn't show debug logging then the debug log for the plugin will not show. Depending on how you auto-start Domoticz the loglevel can be set in a startup file. In my case Domoticz is started by a systemd file. I think it is a init.d file by default.
With systemd my startup command looks like:

Code: Select all

ExecStart=/home/pi/domoticz/domoticz -loglevel status,error

In this case debug logging is nog shown. You can remove the -loglevel parameter to revert to the default logging or add 'debug' to the levels.

In the default init.d startup file it could look like:

Code: Select all

DAEMON_ARGS="$DAEMON_ARGS -loglevel status,error"

Setting the loglevel is nog very well documented but the following page will give you some pointers (even if you are not using syslog).
https://www.domoticz.com/wiki/Set_up_Domoticz_to_syslog

[hestia]

In the default init.d startup file it could look like:

Code: Select all

DAEMON_ARGS="$DAEMON_ARGS -loglevel status,error"

Setting the loglevel is nog very well documented but the following page will give you some pointers (even if you are not using syslog).
https://www.domoticz.com/wiki/Set_up_Domoticz_to_syslog

Even with debug added in the loglevel, I still have no more log from the pluging ; I have more with other things

Code: Select all

2020-03-30 19:28:51.129 Status: (iDetect) Stop directive received.
2020-03-30 19:28:51.129 Status: (iDetect) Exiting work loop.
2020-03-30 19:28:51.229 Status: (iDetect) Stopping threads.
2020-03-30 19:28:51.229 Status: (iDetect) Stopped.
2020-03-30 19:28:51.229 Status: (iDetect) Started.
2020-03-30 19:28:51.395 Status: (iDetect) Initialized version 0.7.7, author 'ESCape'
2020-03-30 19:28:51.398 Status: (iDetect) Entering work loop.
2020-03-30 19:28:54.602 Status: (iDetect) Using chipset specific wl command on router 192.168.1.4 for interfaces eth1 & eth2 (=wl eth1 eth2)
2020-03-30 19:29:06.081 Status: (iDetect) Awaiting confirmation on absence of 3 devices
2020-03-30 19:29:17.393 Status: (iDetect) Awaiting confirmation on absence of 3 devices
2020-03-30 19:29:28.708 Status: (iDetect) Awaiting confirmation on absence of 3 devices
2020-03-30 19:29:40.213 Status: (iDetect) Awaiting confirmation on absence of 3 devices
2020-03-30 19:29:51.878 Status: (iDetect) Awaiting confirmation on absence of 3 devices
2020-03-30 19:30:02.938 Status: (iDetect) Awaiting confirmation on absence of 3 devices
2020-03-30 19:30:25.891 Error: (iDetect) Failed 3 times to get capabilities for 192.168.1.5. Retry interval automatically reduced for this router.
2020-03-30 19:35:35.547 Status: User: xxxxx initiated a switch command (1035/Reveil/Off)
2020-03-30 19:35:35.547 Debug: MAIN SwitchLight idx:1035 cmd:Off lvl:-1
2020-03-30 19:35:35.547 Debug: MAIN SwitchLightInt : switchcmd:Off level:-1 HWid:8 sd:8 0001445B 1 244 62 18
2020-03-30 19:35:35.548 Debug: MAIN SwitchLightInt : switchcmd=="On" || level < 0, new level:10
2020-03-30 19:35:35.553 Debug: SQLH HandleOnOffAction: OnAction:
2020-03-30 19:35:35.553 Debug: SQLH UpdateValueInt Reveil HwID:8 DevID:0001445B Type:244 sType:62 nValue:0 sValue:10
2020-03-30 19:36:04.138 Error: (iDetect) Failed 6 times to get capabilities for 192.168.1.5. Retry interval automatically reduced for this router.

[hestia]

I export the config of the router that was ok and import it to the one ko.
I swap some IP addresses and now all the 3 routers have the error

[EscApe]

I export the config of the router that was ok and import it to the one ko.
I swap some IP addresses and now all the 3 routers have the error

The ssl keys on the routers might have changed, so the Domoticz host no longer trusts the connection. To test this run ssh from the command line on the Domoticz host. Make sure you are first logged on to Domoticz host as root! Because Domoticz runs as root and testing under another user profile will just test the keys of that user profile, not the ones Domoticz is using. You should not get any warnings about keys.

About your logging: I am not sure what is going on. If you set de plugin to debug mode (on plugin setting page) and have no restrictions on Domoticz logging, then we should see a lot more info. Don't forget to restart Domoticz or reboot the Domoticz host. If you are using systemd to startup, then you also need to run 'sudo systemctl daemon-reload' after editing startup files.
You could also try completely removing the -loglevel argument.

[hestia]

Thanks for this new hint I'll do asap...

I've just removed and re-installed the plugin

Code: Select all

2020-03-31 08:16:36.809 Status: (iPresence) Started.
2020-03-31 08:16:36.946 Status: (iPresence) Entering work loop.
2020-03-31 08:16:36.948 Status: (iPresence) Initialized version 2.0, author 'ESCape'
2020-03-31 08:16:37.040 Error: (iPresence) 'onStart' failed 'ImportError'.
2020-03-31 08:16:37.040 Error: (iPresence) ----> Line 427 in '/home/pi/domoticz/plugins/iDetect/plugin.py', function onStart
2020-03-31 08:16:37.040 Error: (iPresence) ----> Line 212 in '/home/pi/domoticz/plugins/iDetect/plugin.py', function onStart
2020-03-31 08:16:37.040 Error: (iPresence) ----> Line 12 in '/home/pi/domoticz/plugins/iDetect/trackers/__init__.py', function <module>
2020-03-31 08:16:37.040 Error: (iPresence) ----> Line 7 in '/home/pi/domoticz/plugins/iDetect/trackers/http_orbi.py', function <module>
2020-03-31 08:16:46.957 Status: (iPresence) Skip this hearbeat ... system is still preparing
2020-03-31 08:16:56.973 Status: (iPresence) Skip this hearbeat ... system is still preparing

I've got an new error

[EscApe]

Thanks for this new hint I'll do asap...

I've just removed and re-installed the plugin

Code: Select all

2020-03-31 08:16:36.809 Status: (iPresence) Started.
2020-03-31 08:16:36.946 Status: (iPresence) Entering work loop.
2020-03-31 08:16:36.948 Status: (iPresence) Initialized version 2.0, author 'ESCape'
2020-03-31 08:16:37.040 Error: (iPresence) 'onStart' failed 'ImportError'.
2020-03-31 08:16:37.040 Error: (iPresence) ----> Line 427 in '/home/pi/domoticz/plugins/iDetect/plugin.py', function onStart
2020-03-31 08:16:37.040 Error: (iPresence) ----> Line 212 in '/home/pi/domoticz/plugins/iDetect/plugin.py', function onStart
2020-03-31 08:16:37.040 Error: (iPresence) ----> Line 12 in '/home/pi/domoticz/plugins/iDetect/trackers/__init__.py', function <module>
2020-03-31 08:16:37.040 Error: (iPresence) ----> Line 7 in '/home/pi/domoticz/plugins/iDetect/trackers/http_orbi.py', function <module>
2020-03-31 08:16:46.957 Status: (iPresence) Skip this hearbeat ... system is still preparing
2020-03-31 08:16:56.973 Status: (iPresence) Skip this hearbeat ... system is still preparing

I've got an new error

Looks like the requests module for your python environment could not be imported. If you have recently updated your Linux and/or python3 version then you might have to reinstall all (required) modules. Unfortunately this is not automated when upgrading to (for instance) python3.7 from 3.6.

[hestia]

to reinstall all (required) modules

could you help me on this? I don't know how...

[EscApe]

to reinstall all (required) modules

could you help me on this? I don't know how...

Assuming you have pip3 installed this should do the trick

Code: Select all

sudo pip3 install requests paramiko

If there are other missing packages you can add them the same way

[hestia]

thanks

I had to do

Code: Select all

sudo apt-get remove python3-pip
sudo apt-get install python3-pip
sudo pip3 install requests paramiko

and

Code: Select all

2020-03-31 13:18:09.730 Status: (iPresence) Started.
2020-03-31 13:18:09.858 Status: (iPresence) Entering work loop.
2020-03-31 13:18:09.859 Status: (iPresence) Initialized version 2.0, author 'ESCape'
2020-03-31 13:18:11.033 Status: (iPresence) Created device for Anyone with unit id 1
2020-03-31 13:18:11.035 Status: (iPresence) Created device for Override with unit id 255
2020-03-31 13:18:11.037 Status: (iPresence) Created device for Pascal with unit id 2
2020-03-31 13:18:11.039 Status: (iPresence) Created device for Camille with unit id 3
2020-03-31 13:18:11.041 Status: (iPresence) Created device for Valérie with unit id 4
2020-03-31 13:18:11.043 Status: (iPresence) Created device for Mathilde with unit id 5
2020-03-31 13:18:11.223 Status: (iPresence) 192.168.1.2 ====> SSH connection established
2020-03-31 13:18:11.611 Status: (iPresence) Tracker activated:192.168.1.2 port 22, user: admin, type: default and options: {}
2020-03-31 13:18:11.932 Status: (iPresence) 192.168.1.3 ====> SSH connection established
2020-03-31 13:18:12.310 Status: (iPresence) Tracker activated:192.168.1.3 port 22, user: admin, type: default and options: {}
2020-03-31 13:18:12.497 Status: (iPresence) 192.168.1.4 ====> SSH connection established
2020-03-31 13:18:12.877 Status: (iPresence) Tracker activated:192.168.1.4 port 22, user: admin, type: default and options: {}

All seems to be ok
Nothing to do on the routers...
Several causes for this issue?
Thanks

[EscApe]

All seems to be ok
Nothing to do on the routers...
Several causes for this issue?

Glad you got it working!
I don't know what the first issue was. 192.168.1.5 wasn't working, but it now has a new IP address(?) So maybe it was an issue with an older key on the Domoticz host, so the host wouldn't trust the newer key on the router. Changing the ip address would have corrected that.
The import error seems unrelated and was probably introduced when reinstalling stuff. DIY domotica will always take more time and effort than you think... let's call it a hobby

[hestia]

DIY domotica will always take more time and effort than you think... let's call it a hobby

yes... for both