Python plugin: Presence detection from wireless router

[EscApe]

@Prinsje

Hopefully someone can shed some light on the roaming issue. I do not have a roaming/mesh-like setup. Since you poll all AP's I would expect the plugin to always find the MAC in question on one of them at any time, but that does not seem to be the case here.

Regarding the USG: Have you tried a ssh session from the raspberry cli to the USG and running 'show arp' (on the USG)? Make sure you are logged on to the raspberry with the same user that is running Domoticz.
I see some post on the internet about limitations of SSH login on the USG, like: https://community.ui.com/questions/cant ... af91ed6f67

Script approaches each AP via its IP with the suffix =unifi-arp.

There is no 'unifi-arp' custom command in the plugin. According to your log the plugin is using wlanconfig on the AP's (probably by auto-detection).
The suffix 'unifiusg-arp', which you wrote further on, does exist to force a 'show arp' command on the USG. I can't remember why I added that custom command. Maybe the USG does not have a standard/full Linux command set(?)The reason is probably somewhere in this thread

[Prinsje]

Thanks EscApe

Regarding the USG: Have you tried a ssh session from the raspberry cli to the USG and running 'show arp' (on the USG)? Make sure you are logged on to the raspberry with the same user that is running Domoticz.

Did indeed tested this with a positive effect. Have in the meantime replaced SSH login with safer method/SSH key and checked user.
ssh [USR]@[IP] makes me enable to login (without a password - hence ssh keying is working); I can then type show arp and it generates an overview of connected devices.

The suffix 'unifiusg-arp', which you wrote further on, does exist to force a 'show arp' command on the USG. I can't remember why I added that custom command. Maybe the USG does not have a standard/full Linux command set(?)

Since pi cli ssh on the USG returned an overview of all connected devices, I resetted my iDetect settings by replacing the APs with the USG domain with suffix =unifiusg-arp .It keeps on generating 127-errors. I have the impression that applying 'ssh [USR]@[IP]' with show arp as a separate statement does generate a workable solution (no reference to -i or -o or show arp as part of the command line, since these result in errors ). Could this be the reason why an error message is being created? Does that require further adjustment of the script?

Code: Select all

2019-09-11 09:29:56.743 Error: (iDetect) Polling 192.[xx] has failed 3 times. Poll interval automatically reduced for this router.

Code: Select all

2019-09-11 09:39:43.497 Status: (iDetect) Started.
2019-09-11 09:39:43.704 Status: (iDetect) Initialized version 0.7.7, author 'ESCape'
2019-09-11 09:39:43.703 Status: (iDetect) Entering work loop.
2019-09-11 09:39:43.745 Status: (iDetect) Using preconfigured custom command on router 192.[IP]: show arp
2019-09-11 09:39:53.713 (iDetect) Polling presense data from 192.[IP]
2019-09-11 09:39:53.713 (iDetect) Fetching data from 192.[IP] using: ssh -i -o ConnectTimeout=3 -p22 [USR]@192.[IP] show arp
2019-09-11 09:39:54.373 (iDetect) SSH subprocess for 192.[IP] failed with error (127):b''
2019-09-11 09:39:54.374 (iDetect) SSH command on 192.[IP] took 660 milliseconds.
2019-09-11 09:39:54.374 (iDetect) Routerinfo:{'prospone': datetime.datetime(2019, 9, 11, 9, 40, 24, 374197), 'port': 22, 'cmd': 'show arp', 'user': '[USR]', 'initialized': True, 'errorcount': 1}
2019-09-11 09:39:54.374 (iDetect) Did not retreive WLAN information from any router
2019-09-11 09:39:54.374 (iDetect) Entire poll took 6% of the poll interval time (661 milliseconds)

[EscApe]

@Prinsje

Hard to tell from a distance... Error 127 indicates the command is not found. Strange, since this custom command was added for the USG specifically. Maybe something has changed in the USG firmware since then(?) It might need a full path now(?)

Running this (with correct user and ip) from the raspberry cli should work when logged in with the user account that is running Domoticz:

Code: Select all

ssh -i -o ConnectTimeout=3 -p22 [USR]@192.[IP] 'show arp'

-o is the option swith to add the timeout parameter
-i tells ssh to use the identification file(s)

You can of course try to see if it makes a difference

Code: Select all

ssh -p22 [USR]@192.[IP] 'show arp'

[xavierlp]

Hello

Do you know if is it possible to use the plugin with a google wifi routeur ?

I don't find the way to connect to it in ssh.

thanks for any help.

Xavier

[EscApe]

Hi Xavier,

I would be surprised if the google Wi-Fi mesh system supports ssh. This plugin will probably not work for you.

[xavierlp]

Thanks for your reply

I did not find anything about this for my side....

[ferrosk]

Hi EscApe

I tried your plugin and I'm getting error:

Code: Select all

(iDetect) Fetching data from [ip] using: ssh -i -o ConnectTimeout=3 -p22 [user]@[ip]
(iDetect) SSH subprocess for [ip] failed with error (255):b'' 

Executing the same command directly from CLI (on RPI running domoticz) results in error:

Code: Select all

$ ssh -i -o ConnectTimeout=3 -p22 [user]@[ip]
Warning: Identity file -o not accessible: No such file or directory.
ssh: Could not resolve hostname connecttimeout=3: Name or service not known

it is because option '-i' must be followed by some argument! When it is missing, like in the command above, then '-o' is interpreted as argument to '-i' option and causing the error.

Checking the source code on Github, I think you have logical error 'plugin.py' see lines 92-95:

Code: Select all

if self.keyfile != '':
    cmd =[self.sshbin, "-o", "ConnectTimeout=" + str(sshtimeout), '-p'+str(port), user+"@"+host, routerscript]
else:
    cmd =[self.sshbin, "-i", self.keyfile, "-o", "ConnectTimeout=" + str(sshtimeout), '-p'+str(port), user+"@"+host, routerscript]

This means option "-i <keyfile>" is used when custom keyfile IS NOT specified in the configuration (variable 'keyfile' is empty)
Correct should be:

Code: Select all

if self.keyfile == '':
    cmd =[self.sshbin, "-o", "ConnectTimeout=" + str(sshtimeout), '-p'+str(port), user+"@"+host, routerscript]
else:
    cmd =[self.sshbin, "-i", self.keyfile, "-o", "ConnectTimeout=" + str(sshtimeout), '-p'+str(port), user+"@"+host, routerscript]

so option "-i <keyfile>" is used when custom keyfile IS specified in configuration (variable 'keyfile' is not empty)

[EscApe]

Hi ferrosk,

Thanks for pointing that out. Looks like a silly mistake indeed:oops:
It has been corrected. Seems like the custom keyfile option was not widely used

[ferrosk]

Hi EscApe,

Seems like the custom keyfile option was not widely used

Due to inverse logic the problem occured just when not using custom keyfile. I think most users use it that way, so I'm surprised it was working for someone before fixed...

Anyway thanks for the plugin! I can confirm it is working fine for me now with Domoticz runnig on rpi (linux) and with wifi router running Openwrt . It does not even need admin (root) privileges on the router. I created new user dedicated for this plugin and set ssh-key authentication. Scripts executed by the plugin on router works fine with normal (non-root) user.

[EscApe]

Hi ferrosk,

It appears to be the other way around... Also to my surprise

After failing to read the custom identity-file (-i switch followed by anything else, like -o in this case) ssh will automatically fall back to using the default identity file (under .ssh). So as long as key based acces was configured it would always work.
Before the fix an intentionally configured custom identity-file would never be used, because of the reverse logic.

Thanks again for pointing out this mistake!

[vamp]

If somebody use this plugin with Mikrotik and CAPsMAN it need to change this line to working:

from

Code: Select all

custom['routeros']="interface wireless registration-table print"

to

Code: Select all

custom['routeros']="caps-man registration-table print"

It only need to check the capsman manager not need to check all APs!


it would be great that this feature will be build in to the next release. (like "routeros-capsman")

[hscholing]

Anyone working with ibeacon and Aruba Networks access points (in Instant mode)?

2019-10-14 22:22:28.160 (Wifi Presense) 192.168.2.4 was not properly initialized. Retrying to get router capabilities (and skipping this poll round).
2019-10-14 22:22:28.161 (Wifi Presense) Using password instead of ssh public key authentication for 192.168.2.4 (less secure!)
2019-10-14 22:22:28.161 (Wifi Presense) Fetching data from 192.168.2.4 using: sshpass -p **secret** ssh -o StrictHostKeyChecking=no -o ConnectTimeout=3 -p22 [email protected]
2019-10-14 22:22:28.161 #!/bin/sh
2019-10-14 22:22:28.161 export PATH=/usr/sbin:/usr/bin:/usr/local/sbin:/usr/local/bin:/sbin:/bin:$PATH
2019-10-14 22:22:28.161 type wl > /dev/null 2>&1 && printf "~wl"
2019-10-14 22:22:28.161 type iwinfo > /dev/null 2>&1 && printf "~iwinfo"
2019-10-14 22:22:28.161 type wlanconfig > /dev/null 2>&1 && printf "~wlanconfig"
2019-10-14 22:22:28.161 type qcsapi_sockrpc > /dev/null 2>&1 && printf "~qcsapi_sockrpc"
2019-10-14 22:22:28.161 type brctl > /dev/null 2>&1 && printf "~brctl"
2019-10-14 22:22:28.161 type arp > /dev/null 2>&1 && printf "~arp"
2019-10-14 22:22:28.161 [ -f /proc/net/arp ] && printf "~procarp"
2019-10-14 22:22:28.161 exit
2019-10-14 22:22:28.161
2019-10-14 22:22:28.939 (Wifi Presense) SSH subprocess for 192.168.2.4 failed with error (255):b''
2019-10-14 22:22:28.939 (Wifi Presense) SSH command on 192.168.2.4 took 777 milliseconds.
2019-10-14 22:22:28.939 (Wifi Presense) Could not retreive available commands on 192.168.2.4
2019-10-14 22:22:28.940 (Wifi Presense) Routerinfo:{'user': 'admin', 'port': 22, 'cmd': '', 'initialized': False, 'prospone': datetime.datetime(2019, 10, 14, 22, 23, 28, 939765), 'errorcount': 2}
2019-10-14 22:22:28.940 (Wifi Presense) Did not retreive WLAN information from any router
2019-10-14 22:22:28.940 (Wifi Presense) Entire poll took 5% of the poll interval time (779 milliseconds)
2019-10-14 22:22:43.471 (Wifi Presense) Prosponed connection to 192.168.2.4 for 0:00:45.468781
2019-10-14 22:22:43.471 (Wifi Presense) Did not retreive WLAN information from any router
2019-10-14 22:22:43.471 (Wifi Presense) Entire poll took 0% of the poll interval time (0 milliseconds)

[EscApe]

@vamp and other Mirotik owners...

If somebody use this plugin with Mikrotik and CAPsMAN it need to change this line to working:

from
CODE: SELECT ALL
custom['routeros']="interface wireless registration-table print"
to
CODE: SELECT ALL
custom['routeros']="caps-man registration-table print"

It only need to check the capsman manager not need to check all APs!


it would be great that this feature will be build in to the next release. (like "routeros-capsman")

Should the caps-man approach replace the existing method (is it better, faster, more reliable)? Or is this an additional method that might not work on every microtik router?

[EscApe]

@hscholing:

Anyone working with ibeacon and Aruba Networks access points (in Instant mode)?

I don't own any Aruba hardware, but with a little bit of googling it seems that ArubaOS does not offer a regular linux shell, but its own set of proprietary commands. Auto detection will fail in such a case. I can add a custom command to the plugin. You (or another Aruba owner) would have to supply a working command line that I can add.

A good starting point seems to be the 'show clients' command. Maybe you can add some formatting to the commandline so it will only return a list of Mac addresses (instead of a table with other info)?

[Jilles]

I can' t seem to get it working.

My setup:

• Synology NAS with Dockerized Domoticz (image from linuxserver)
• SSH key authorization works (tested from inside domoticz container, proof below)
• Asus RT-AC3200 (tried normal firmware and merlin, now 3.0.0.4.382_51640-g679a7e3)
• Tri-band Smart Connect (that is why wl doesn''t show anything, but works with wl eth2 eth3)
• Also tried without Tri-band Smart Connect, then wl works on ssh, but no result in domoticz

I've tried almost everything;

• only ip-address
• ip-address#forcegeneric
• ip-address=arp
• qcsapi_sockrpc, but not supported on the router
• ip-address=wl eth2 eth3 as shown below
• actual user and ip are removed here, but are the same everywhere

These are the logs from Domoticz

Code: Select all

2019-10-19 21:13:08.514 (Presence detection) Fetching data from ip using: ssh -o ConnectTimeout=3 -p22 user@ip export PATH=/usr/sbin:/usr/bin:/usr/local/sbin:/usr/local/bin:/sbin:/bin:$PATH;wl -i eth2 assoclist | cut -d' ' -f2;wl -i eth3 assoclist | cut -d' ' -f2;exit
2019-10-19 21:13:08.562 (Presence detection) SSH subprocess for ip failed with error (255):b''
2019-10-19 21:13:08.562 (Presence detection) SSH command on ip took 47 milliseconds.
2019-10-19 21:13:08.562 (Presence detection) Routerinfo:{'user': 'user', 'port': 22, 'cmd': "export PATH=/usr/sbin:/usr/bin:/usr/local/sbin:/usr/local/bin:/sbin:/bin:$PATH;wl -i eth2 assoclist | cut -d' ' -f2;wl -i eth3 assoclist | cut -d' ' -f2;exit", 'initialized': True, 'prospone': datetime.datetime(2019, 10, 19, 21, 15, 8, 562517), 'errorcount': 8}
2019-10-19 21:13:08.562 (Presence detection) Did not retreive WLAN information from any router

For every way I try, I keep getting the same "SSH subprocess for ip failed with error (255):b".
If I try that exact command directly on my router

Code: Select all

user@RT-AC3200:/tmp/home/root# wl -i eth2 assoclist | cut -d' ' -f2;wl -i eth3 as
soclist | cut -d' ' -f2
XX:XX:XX:XX:XX:EA
XX:XX:XX:XX:XX:97
XX:XX:XX:XX:XX:44
XX:XX:XX:XX:XX:B7
XX:XX:XX:XX:XX:95
XX:XX:XX:XX:XX:5E
XX:XX:XX:XX:XX:44
XX:XX:XX:XX:XX:FE

I really hope someone can help me out, because this plugin is really what I'm looking for.

[EscApe]

@Jilles

The first thing to try/rule out is authentication issues. If you are using key based authentication make sure it is working by running a ssh session to the router from a ssh session on the synology under the same (synology) user profile that is running the Domoticz proces.

[Jilles]

@Jilles

The first thing to try/rule out is authentication issues. If you are using key based authentication make sure it is working by running a ssh session to the router from a ssh session on the synology under the same (synology) user profile that is running the Domoticz proces.

I'm running Domoticz in Docker. The output in the last codeblock was run from a SSH session on the router, setup from inside this docker container as root. This should be similar to the real thing.

[EscApe]

I'm running Domoticz in Docker. The output in the last codeblock was run from a SSH session on the router, setup from inside this docker container as root. This should be similar to the real thing.

It is indeed the same IF you are automatically logged in on the router. Does the router ask for a password when logging in from the docker container to the router?

[Jilles]

[quote=EscApe post_id=226462 time=1571588735 user_id=5790]

I'm running Domoticz in Docker. The output in the last codeblock was run from a SSH session on the router, setup from inside this docker container as root. This should be similar to the real thing.

It is indeed the same IF you are automatically logged in on the router. Does the router ask for a password when logging in from the docker container to the router?
[/quote]

Ok, so I was typing up what I have been doing, but I found that Domoticz is NOT running under root (didn't check that before, my bad). It seems to be running under "abc". Now I just need to find out what to do with this info. Can't seem to su into abc or now where to put the keys... Any ideas here are more than welcome of course.

Code: Select all

[s]2019-10-20 21:36:15.397 (Presence) Debug logging mask set to: PYTHON
2019-10-20 21:36:15.397 (Presence) Not running on Windows
2019-10-20 21:36:15.401 (Presence) The OS user profile running domoticz is: abc
2019-10-20 21:36:15.409 (Presence) Checking if [ssh -V] will run: OK
2019-10-20 21:36:15.409 (Presence) Parsing user and optional keyfile from:Jilles[/s]

Ok apparently that user was not allowed to get a shell (as was stated in /etc/passwd). I changed this to /bin/ash to get a shell. That enabled me to do ssh-keygen -t rsa. I could now also login from user abc on the domoticz host to my router. But still no success.

@EscApe what does that last log line mean? Jilles is the user on my router.

Forget about all that. I figured out how to install sshpass in my alpine based container. Now working with username:pass. Works like a charm!

PS Just realized that I just started screaming for help, sorry for that. That is a bit rude. I really appreciate what everybody (especially you @EscApe) is doing here. So thanks a lot!

[EscApe]

Forget about all that. I figured out how to install sshpass in my alpine based container. Now working with username:pass. Works like a charm!

Happy to hear you got it working!
If you ever want to (re)try key based authentication I think you can just copy the key files you created as root to the .ssh folder in the abc home directory and change ownership... not sure though