Python plugin: Presence detection from wireless router

[Innovator]

@escape
If setting up key based access for the user running Domoticz on the Synology (root??) is not an option you can change a small piece of code in the plugin.
_{it's root}

Look for (will be around line 81 in the version i'm uploading in a few minutes):

Code: Select all

cmd =[self.sshbin, "-o", "ConnectTimeout=" + str(sshtimeout), user+"@"+host, routerscript]

And change it to (with whatever path you are using):

Code: Select all

cmd =[self.sshbin, "-i", "/volume1/path/to/sshfiles/privatekeyfile", "-o", "ConnectTimeout=" + str(sshtimeout), user+"@"+host, routerscript]

_{I have nothing changed use version 0.4.1}
You can experiment from the synology command line to confirm it is working: ssh -i <keyfile-path> usernameonrouter@routerip
_{ssh keyfile is correct working}

I have added the wlanconfig command and ath* interface names to the latest version of the plugin. Will upload it in a few minutes. If anything goes wrong please include some debug logging of the plugin starting up. The part you included only shows that the polling failed, but the reason is a failure while initializing the plugin.

_{At this moment the plugin create no devices. The domticz log shows only
2018-06-13 19:49:53.064 (iDetect Wifi presence detection) Initialized version 0.4.1, author 'ESCape'
2018-06-13 19:49:53.064 (iDetect Wifi presence detection) Entering work loop.
no errors, debug = on
Version 0.4.1. is unfortunately not working}

[EscApe]

There have been some changes that require extra attention when upgrading from an earlier version.

In this order:

Make sure the ioverride.zip file (new since 0.4) is in you plugin directory (besides plugin.py and the orther zip files)
Restart domoticz! (Otherwise settings page is not updated)
go to the hardware settings page for the plugin and set the new parameters (and save)

[devros]

Thanks now it all works (almost) without errors
sometimes i have in log timeout (few times in one log)
should i try modify script to make longer timeouts ?

Code: Select all

2018-06-15 10:03:57.215 Error: (presence) SSH subprocess failed with error: Command '['sshpass', '-p', 'XXXXX', 'ssh', '-o', 'StrictHostKeyChecking=no', '-o', 'ConnectTimeout=1', 'ubnt@XXXXX', "export PATH=/usr/sbin:/usr/bin:/usr/local/sbin:/usr/local/bin:/sbin:/bin:$PATH;wlanconfig ath0 list | grep '^..:..:..:..:..:.. ' | cut -d' ' -f1;wlanconfig eth0 list | grep '^..:..:..:..:..:.. ' | cut -d' ' -f1;exit"]' timed out after 2 seconds
2018-06-15 10:03:57.216 Error: (presence) Failed to poll present phones from router
2018-06-15 10:03:57.216 Error: (presence) No list of connected WLAN devices from router

[EscApe]

@devros

Glad it's working!

Since the plugin runs an external program (ssh) it will block the plugin system whilst ssh is running. From the wiki: "Plugins are single threaded so the whole plugin system will wait." That's why i kept the timeouts short. If the router, network or Domoticz system is very busy it might fail, but at least it will not block the plugin system for long.

If the errors are not that frequent you can just ignore them. It only means you miss one poll cycle and presence/absence will be detected again on the next poll.

The timeouts relevant are:
sshtimeout: time it may take ssh to establish the connection to the router (shorter than alltimeout because real connection issues are most likely to last longer, so you want to skip the poll as soon as possible)
alltimeout: time the entire ssh command may take on the host system (a failsafe in case of unforeseen problems that can not be detected as quick as a connection issue. An example would be a working ssh connection, but hanging command on the router)

You can experiment with the timeouts. If your poll interval is more than 30 second i see no harm in increasing the (alltimeout) of the subprocess to 3-5 seconds. Worst case it will block your plugin system for 10% or even 16,6% of the time, but only if the entire command takes that long for some reason (very busy Domoticz host or router?). The sshtimeout is a different story. If your router is somehow unreachable or very slow to respond (down, rebooting, disconnected) the plugin might wait the full amount of the timeout every single time it tries to poll. On the other hand if it only polls every 30 seconds it will probably do no real harm if you set the sshtimeout to 2-3 seconds.

I poll every 10 seconds, so short timeouts are crucial to prevent the plugin from blocking the plugin system for most of the time. It is all a question of balance. On my dedicated raspberry pi i get very very few errors. The ones that did occur were related to the domoticz host being very busy (compiling domoticz ) or the router rebooting. If everything is functioning normally the whole ssh command will be processed within tenths of a second... at least on my setup.

[devros]

@devros

Glad it's working!

Thanks for full info about timeouts , but noticed that its not working (sorry) with new wlanconfig on my ubitiqui router...
wlanconfig eth0 list returns 'Invalid argument' (propablly disabled)

dunno why, previous (cat /proc/net/arp)
worked great and fast, can you add switch to use ARP for scaning please

Code: Select all

command: ['sshpass', '-p', 'xxx', 'ssh', '-o', 'StrictHostKeyChecking=no', '-o', 'ConnectTimeout=1', '[email protected]', "export PATH=/usr/sbin:/usr/bin:/usr/local/sbin:/usr/local/bin:/sbin:/bin:$PATH;wlanconfig ath0 list | grep '^..:..:..:..:..:.. ' | cut -d' ' -f1;wlanconfig eth0 list | grep '^..:..:..:..:..:.. ' | cut -d' ' -f1;exit"]
2018-06-17 20:19:49.615 (presence) Returncode ssh command: 0
2018-06-17 20:19:49.615 (presence) Output from router: b'80:2a:a8:a4:e8:11\n'
2018-06-17 20:19:49.615 (presence) Considered absent: xxx
2018-06-17 20:19:49.616 (presence) Considered absent: xxxx

[EscApe]

Hi devros,

In your message you write the command isn’t working, yet you include a piece of the log that shows the plugin is returning a MAC address. Should it have returned more addresses? Are you sure the plugin isn’t working? Testing the command from the command line using eth0 might indeed throw an error. Looks like the interface on your router is called ath0 (a instead of e).

Edit:
In the plugin I used ‘wlanconfig ath0 list’ instead of ‘wlanconfig ath0 list sta‘. Could you try if there is a difference between the two commands (with and without sta)?

[devros]

Hi devros,

In your message you write the command isn’t working, yet you include a piece of the log that shows the plugin is returning a MAC address. Should it have returned more addresses? Are you sure the plugin isn’t working? Testing the command from the command line using eth0 might indeed throw an error. Looks like the interface on your router is called ath0 (a instead of e).

Edit:
In the plugin I used ‘wlanconfig ath0 list’ instead of ‘wlanconfig ath0 list sta‘. Could you try if there is a difference between the two commands (with and without sta)?

On my device is ath0 outside network(one ip adress), mine should be
eth0, but with this i get invalid argument...so you plugin works, but something wrong is with my router

Arp scaning worked great

[EscApe]

I would still be interested to see if we can get it working, but that will take some experimenting by someone who owns the router. Did you try the commands I suggested on ath0 and eth0, with and without ‘sta’?

You can (temporarily) ‘sabotage’ the plugin to not use wlanconfig by changing line 163
Change

Code: Select all

test=$(which wlanconfig > /dev/null 2>&1)

To

Code: Select all

test=$(which notwlanconfig > /dev/null 2>&1)

(Or any other non existing commandname)

[devros]

Here is my output, looks like sta make no diference

Code: Select all

XM.v6.1.7# wlanconfig ath0 list
ADDR               AID FREQ TxRATE RxRATE RSSI   IDLE  TXSEQ RXSEQ  CAPS ACAPS      ERP  STATE HTCAPS
80:2a:a8:a4:e8:11   19 5500   117M   130M   38      0   3299 38672   Es             0      80b AP     WME ATH

Code: Select all

XM.v6.1.7# wlanconfig ath0 list sta
ADDR               AID FREQ TxRATE RxRATE RSSI   IDLE  TXSEQ RXSEQ  CAPS ACAPS      ERP  STATE HTCAPS
80:2a:a8:a4:e8:11   19 5500   117M   130M   38      0   1447 38672   Es             0      80b AP     WME ATH

Code: Select all

XM.v6.1.7# wlanconfig eth0 list
Invalid argument
XM.v6.1.7# wlanconfig eth0 list sta
Invalid argument

Thanks

[EscApe]

Weird ... maybe the router has additional interface names for wireless interfaces?
You can find all interfaces using ifconfig and try them one by one with the same command. Maybe the MAC addresses show up on an interface that is now filtered out by the plugin. The plugin assumes eth* wlan* and ath* as acceptable names, but there might be more.

[devros]

Weird ... maybe the router has additional interface names for wireless interfaces?
You can find all interfaces using ifconfig and try them one by one with the same command. Maybe the MAC addresses show up on an interface that is now filtered out by the plugin. The plugin assumes eth* wlan* and ath* as acceptable names, but there might be more.

im using router on my local network only for ethernet conection

[EscApe]

Hi devros,

That explains a lot.. The plugin and the command line tools used is intended for wireless devices, not for ethernet. What is your use case for monitoring the presence of wired (probably stationary?) devices?

There's nothing wrong with the plugin. It is just not intended for ethernet connections. I will keep the wlanconfig command for other ubiquiti owners. To force the plugin to use arp you can sabotage the other commands as described in my earlier post. I might add a 'force arp' option in the future, but that won't be soon. The plugin already uses all available configuration parameters offered by the Domoticz plugin framework.

[devros]

Hi devros,

That explains a lot.. The plugin and the command line tools used is intended for wireless devices, not for ethernet. What is your use case for monitoring the presence of wired (probably stationary?) devices?

There's nothing wrong with the plugin. It is just not intended for ethernet connections. I will keep the wlanconfig command for other ubiquiti owners. To force the plugin to use arp you can sabotage the other commands as described in my earlier post. I might add a 'force arp' option in the future, but that won't be soon. The plugin already uses all available configuration parameters offered by the Domoticz plugin framework.

but i think that wlanconfig monitoring is not working for all ubiquiti users
My ubitiqui works as lan DHCP router connected to tenda (witch works as wifi)

OK thanks for you time... with modified code ARP works great.
you can maybe join some parameters to save position for force ARP switch

[kofec]

Maybe it is worth to add notification to DHCP server:
for ubiquity:
https://community.ubnt.com/t5/EdgeRoute ... -p/2069562
for Openwrt:
https://jpmens.net/2013/10/21/tracking- ... h-dnsmasq/

[EscApe]

@kofec

I’m not sure what you mean. What would you like this type of integration to do?

[EscApe]

@devros

Combining parameters has crossed my mind and I will consider it for a future version. Don’t want to over complicate things though.
The plugin is meant to talk to wireless routers directly (tenda in your case). Does the tenda offer ssh?

Just curious: how long does it take until absence is detected after a device on your network has gone offline? ARP is unusable on my Asus, because it can take forever for it to notice that a device has disconnected (person holding it has left the house).

[drwurn]

Are the instructions on the first page still actual and working?

[Sjaakios]

Hi thanks for this scripts! Have been looking for good presence detection, it's always a struggle...

Had some issues getting the ssh key configured, fixed it based on the above comments (linking directly to the keyfile..). However ended up with the following error: 'onStart' failed 'AttributeError':'module 'Domoticz' has no attribute 'Status''.

Don't know if its my version of domoticz (v3.8153), or because i'm running it on my synology. However, fixed it by replacing the Domoticz.Status -> Domoticz.Log.

Thanks again, let's see if it's going to work

CHeers,
Daan

Code: Select all

 2018-06-19 20:38:57.046 (iDetect) Initialized version 0.4.1, author 'ESCape'
2018-06-19 20:38:57.101 (iDetect) Debug log level set to: 'true'.
2018-06-19 20:38:57.101 (iDetect) Not running on Windows
2018-06-19 20:38:57.141 (iDetect) Checking if [ssh -V] will run: OK
2018-06-19 20:38:57.142 (iDetect) Monitoring {'phone1': 'CC:25:EF:40:7B:F8'} for presence.
2018-06-19 20:38:57.142 (iDetect) Checking router capabilities and wireless interfaces
2018-06-19 20:38:57.142 (iDetect) Using ssh public key authentication (~/.ssh/id_rsa.pub) for OS user running domoticz.
2018-06-19 20:38:57.142 (iDetect) Fetching data from router using ssh
2018-06-19 20:38:57.142 (iDetect) command: ['ssh', '-i', '/root/.ssh/id_rsa', '-o', 'ConnectTimeout=2', 'xx@xxx', '#!/bin/sh\n\t\t\t\t\texport PATH=/usr/sbin:/usr/bin:/usr/local/sbin:/usr/local/bin:/sbin:/bin:$PATH\n\t\t\t\t\ttest=$(which wl > /dev/null 2>&1)\n\t\t\t\t\tif [ $? == 0 ]; then\n\t\t\t\t\t\t\tprintf "wl@"\n\t\t\t\t\t\t\tfor iface in $(ifconfig | cut -d \' \' -f1| tr \':\' \'\n\' | grep \'^eth\\|^wlan\')\n\t\t\t\t\t\t\tdo\n\t\t\t\t\t\t\t\t\ttest=$(wl -i $iface assoclist > /dev/null 2>&1)\n\t\t\t\t\t\t\t\t\tif [ $? == 0 ]; then\n\t\t\t\t\t\t\t\t\t\t\tprintf "$iface "\n\t\t\t\t\t\t\t\t\tfi\n\t\t\t\t\t\t\tdone\n\t\t\t\t\t\t\texit\n\t\t\t\t\tfi\n\t\t\t\t\ttest=$(which iwinfo > /dev/null 2>&1)\n\t\t\t\t\tif [ $? == 0 ]; then\n\t\t\t\t\t\t\tprintf "iwinfo@"\n\t\t\t\t\t\t\tfor iface in $(ifconfig | cut -d \' \' -f1| tr \':\' \'\n\' | grep \'^eth\\|^wlan\\|^ath\')\n\t\t\t\t\t\t\tdo\n\t\t\t\t\t\t\t\t\ttest=$(iwinfo $iface assoclist > /dev/null 2>&1)\n\t\t\t\t\t\t\t\t\tif [ $? == 0 ]; then\n\t\t\t\t\t\t\t\t\t\t\tprintf "$iface "\n\t\t\t\t\t\t\t\t\tfi\n\t\t\t\t\t\t\tdone\n\t\t\t\t\t\t\texit\n\t\t\t\t\tfi\n\t\t\t\t\ttest=$(which wlanconfig > /dev/null 2>&1)\n\t\t\t\t\tif [ $? == 0 ]; then\n\t\t\t\t\t\t\tprintf "wlanconfig@"\n\t\t\t\t\t\t\tfor iface in $(ifconfig | cut -d \' \' -f1| tr \':\' \'\n\' | grep \'^eth\\|^wlan\\|^ath\')\n\t\t\t\t\t\t\tdo\n\t\t\t\t\t\t\t\t\ttest=$(wlanconfig $iface list > /dev/null 2>&1)\n\t\t\t\t\t\t\t\t\tif [ $? == 0 ]; then\n\t\t\t\t\t\t\t\t\t\t\tprintf "$iface "\n\t\t\t\t\t\t\t\t\tfi\n\t\t\t\t\t\t\tdone\n\t\t\t\t\t\t\texit\n\t\t\t\t\tfi\t\t\t\t\t\t\t\t\t\t\n\t\t\t\t\ttest=$(which arp > /dev/null 2>&1)\n\t\t\t\t\tif [ $? == 0 ]; then\n\t\t\t\t\t\t\tprintf "arp"\n\t\t\t\t\t\t\texit\n\t\t\t\t\tfi\n\t\t\t\t\tif [ -f /proc/net/arp ]; then\n\t\t\t\t\t\tprintf "procarp"\n\t\t\t\t\t\texit\n\t\t\t\t\tfi\n\t\t\t\t\tprintf none']
2018-06-19 20:38:57.852 (iDetect) Returncode ssh command: 0
2018-06-19 20:38:57.852 (iDetect) Output from router: b'wl@eth1 eth2 '
2018-06-19 20:38:57.853 (iDetect) Parsed data from router: ['wl', 'eth1 eth2 ']
2018-06-19 20:38:57.853 Error: (iDetect) 'onStart' failed 'AttributeError':'module 'Domoticz' has no attribute 'Status''.
2018-06-19 20:38:57.853 Error: (iDetect) ----> Line 479 in /usr/local/domoticz/var/plugins/idetect/plugin.py, function onStart
2018-06-19 20:38:57.853 Error: (iDetect) ----> Line 311 in /usr/local/domoticz/var/plugins/idetect/plugin.py, function onStart
2018-06-19 20:38:57.853 Error: (iDetect) ----> Line 198 in /usr/local/domoticz/var/plugins/idetect/plugin.py, function routercommand 

[EscApe]

@drwurn

The poll interval is now a pull down choice and an option for a override button has been added. Those changes have not yet been added to the instruction. The basic setup is still accurate.
Do you ask because something is not working?

[EscApe]

@Sjaakios

The Domoticz.Status has been added to the Domoticz plugin framework relatively recent (don’t know the exact version). The error will be gone if you update to the latest domoticz beta. Or you can stick with your work around. Logging levels are I bit less granular but that’s no problem (that’s how it was before the framework change).