Python plugin: Presence detection from wireless router

[EscApe]

@innovator,

Seems the AC828 uses a qualcomm chipset. My first guess would be that the wl command is implemented differently. Could you enter the following on the routers command (for eth0 - eth3) line and share the output?

Code: Select all

wl -i eth0 assoclist;echo $?

Code: Select all

wl -i eth1 assoclist;echo $?

Code: Select all

etc..

The arp command is only used if the wl command can not be found.

[igorkkk]

It is my problem:

Code: Select all

 2018-05-22 17:00:15.300 Error: (IDetect) 'onHeartbeat' failed 'AttributeError':''BasePlugin' object has no attribute 'routercmdline''.
2018-05-22 17:00:15.300 Error: (IDetect) ----> Line 333 in /home/pi/domoticz/plugins/iDetect/plugin.py, function onHeartbeat
2018-05-22 17:00:15.300 Error: (IDetect) ----> Line 291 in /home/pi/domoticz/plugins/iDetect/plugin.py, function onHeartbeat 

Raspberry Pi 3B
ASUS RT-AC88U

[EscApe]

@igorkkk

Can you provide a little more info? Especially the debug log of the plugin while starting up.
From the log you provided i can only conclude that initialization of the plugin failed.

[Innovator]

@innovator,

Seems the AC828 uses a qualcomm chipset. My first guess would be that the wl command is implemented differently. Could you enter the following on the routers command (for eth0 - eth3) line and share the output?

Code: Select all

wl -i eth0 assoclist;echo $?

/tmp/home/root# wl -i eth0 assoclist;echo $?
Usage : wl clm_data_ver
0

Code: Select all

wl -i eth1 assoclist;echo $?

/tmp/home/root# wl -i eth1 assoclist;echo $?
Usage : wl clm_data_ver
0

Code: Select all

etc..

/tmp/home/root# wl -i eth2 assoclist;echo $?
Usage : wl clm_data_ver
0

etc...

The arp command is only used if the wl command can not be found.

The wl clm data_ver gives:
wl clm_data_ver
BRT-AC828
10.4-0.0.2217.273 (IPQ8064.ILQ.3.1 CSU3)
The wl clm data_ver -i eth0 assoclist;echo $? gives:
Usage : wl clm_data_ver

[EscApe]

@innovator

So the wl utility is present, throws an error message, but does not return an exit errorlevel. Thats going to be a challenge... let's take it on

I have found some info on different methods of determining the connected mac addresses for different chipsets on the link below. There will probably be more methods (depending on chipset and drivers).
https://wiki.openwrt.org/doc/howto/wireless.utilities

Can you do some experimenting? What we need to know is:
1. If a certain tool is available (wl, iwinfo, wlanconfig and maybe others?) -> this is already handled for wl and iwinfo
2. which interfaces to monitor -> i'm now looking at eth* and wlan*, but it seems some routers use different interface names. If thats is the case please let me know the exact names.
3. What specific command syntax for the tool will result in a list of mac addresses.
4. How a wrong syntax can be detected (if not by exit errorlevel) -> i will try to figure this out if i can get some examples.

2 en 3 can only be determined by experimenting on the router itself, so i will need your help on those. To find a reliable method for 4, examples of exact(!) erroneous and successful output including the command used and the errorlevel returned will help. The same goes for running the right syntax on the wrong interface.

I am not yet sure if i can check these things in the plugin without breaking or overcomplicating the plugin for already supported routers. Any pointers, examples or other input from the community will be appreciated.


Edit: Found some more examples of commands (change interface names to suit your router)
# Universal (Tested with OpenWRT 14.07 and 15.05.X)
iwinfo wlan0/wl0/ath0 assoclist

# Proprietary Broadcom (wl)
wl -i wl0 assoclist

# Proprietary Atheros (madwifi)
wlanconfig ath0 list sta

# MAC80211
iw dev wlan0 station dump

source: https://serverfault.com/questions/22604 ... nwrt-10-03

[omnisxin]

Hi,Escape
Same error as igorkkk encounted, my router firmware is asus merlin.
domoticz error log as follows, pls take a look where the error is

2018-05-28 21:11:56.573 (iDetect) Started.
2018-05-28 21:11:56.856 (iDetect) Entering work loop.
2018-05-28 21:11:56.858 (iDetect) Initialized version 0.3.0, author 'ESCape'
2018-05-28 21:11:56.862 (iDetect) Debug logging mask set to: PYTHON PLUGIN QUEUE IMAGE DEVICE CONNECTION MESSAGE ALL
2018-05-28 21:11:56.862 (iDetect) Debugging mode enabled
2018-05-28 21:11:56.862 (iDetect) Monitoring {'PHONE3': 'XX:XX:28:C4:98:9E', 'PHONE1': 'XX:2D:5E:FD:CD:53', 'PHONE2': 'XX:51:BA:A9:D1:CB'} for presence.
2018-05-28 21:11:56.862 (iDetect) Pushing 'PollIntervalDirective' on to queue
2018-05-28 21:11:56.862 (iDetect) Checking router capabilities and wireless interfaces
2018-05-28 21:11:56.863 (iDetect) Using password instead of ssh public key authentication (less secure!)
2018-05-28 21:11:56.863 (iDetect) Fetching data from router using ssh
2018-05-28 21:11:56.863 (iDetect) command: ['sshpass', '-p', 'admin', 'ssh', '-o', 'StrictHostKeyChecking=no', '-o', 'ConnectTimeout=2', '[email protected]', '#!/bin/sh\n\t\t\t\t\texport PATH=/usr/sbin:/usr/bin:/usr/local/sbin:/usr/local/bin:/sbin:/bin:$PATH\n\t\t\t\t\ttest=$(which wl > /dev/null 2>&1)\n\t\t\t\t\tif [ $? == 0 ]; then\n\t\t\t\t\t\t\tprintf "wl@"\n\t\t\t\t\t\t\tfor iface in $(ifconfig | cut -d \' \' -f1| tr \':\' \'\n\' | grep \'^eth\\|^wlan\')\n\t\t\t\t\t\t\tdo\n\t\t\t\t\t\t\t\t\ttest=$(wl -i $iface assoclist > /dev/null 2>&1)\n\t\t\t\t\t\t\t\t\tif [ $? == 0 ]; then\n\t\t\t\t\t\t\t\t\t\t\tprintf "$iface "\n\t\t\t\t\t\t\t\t\tfi\n\t\t\t\t\t\t\tdone\n\t\t\t\t\t\t\texit\n\t\t\t\t\tfi\n\t\t\t\t\ttest=$(which iwinfo > /dev/null 2>&1)\n\t\t\t\t\tif [ $? == 0 ]; then\n\t\t\t\t\t\t\tprintf "iwinfo@"\n\t\t\t\t\t\t\tfor iface in $(iwinfo | cut -d \' \' -f1| tr \':\' \'\n\' | grep \'^eth\\|^wlan\')\n\t\t\t\t\t\t\tdo\n\t\t\t\t\t\t\t\t\ttest=$(iwinfo wlan0 assoclist > /dev/null 2>&1)\n\t\t\t\t\t\t\t\t\tif [ $? == 0 ]; then\n\t\t\t\t\t\t\t\t\t\t\tprintf "$iface "\n\t\t\t\t\t\t\t\t\tfi\n\t\t\t\t\t\t\tdone\n\t\t\t\t\t\t\texit\n\t\t\t\t\tfi\t\t\t\t\t\n\t\t\t\t\ttest=$(which arp > /dev/null 2>&1)\n\t\t\t\t\tif [ $? == 0 ]; then\n\t\t\t\t\t\t\tprintf "arp"\n\t\t\t\t\tfi']
2018-05-28 21:11:56.863 Error: (iDetect) 'onStart' failed 'AttributeError'.
2018-05-28 21:11:56.863 Error: (iDetect) ----> Line 329 in /home/pi/domoticz/plugins/Domoticz_iDetect/plugin.py, function onStart
2018-05-28 21:11:56.863 Error: (iDetect) ----> Line 232 in /home/pi/domoticz/plugins/Domoticz_iDetect/plugin.py, function onStart
2018-05-28 21:11:56.863 Error: (iDetect) ----> Line 146 in /home/pi/domoticz/plugins/Domoticz_iDetect/plugin.py, function routercommand
2018-05-28 21:11:56.863 Error: (iDetect) ----> Line 107 in /home/pi/domoticz/plugins/Domoticz_iDetect/plugin.py, function getfromssh
2018-05-28 21:11:56.863 (iDetect) Processing 'PollIntervalDirective' message
2018-05-28 21:11:56.863 (iDetect) Heartbeat interval set to: 10.
2018-05-28 21:12:06.359 (iDetect) Pushing 'onHeartbeatCallback' on to queue
2018-05-28 21:12:06.387 (iDetect) Processing 'onHeartbeatCallback' message
2018-05-28 21:12:06.387 (iDetect) Calling message handler 'onHeartbeat'.
2018-05-28 21:12:06.387 (iDetect) devid2domid: {}
2018-05-28 21:12:06.388 Error: (iDetect) 'onHeartbeat' failed 'AttributeError':''BasePlugin' object has no attribute 'routercmdline''.
2018-05-28 21:12:06.388 Error: (iDetect) ----> Line 333 in /home/pi/domoticz/plugins/Domoticz_iDetect/plugin.py, function onHeartbeat
2018-05-28 21:12:06.388 Error: (iDetect) ----> Line 291 in /home/pi/domoticz/plugins/Domoticz_iDetect/plugin.py, function onHeartbeat
2018-05-28 21:12:16.360 (iDetect) Pushing 'onHeartbeatCallback' on to queue
2018-05-28 21:12:16.408 (iDetect) Processing 'onHeartbeatCallback' message
2018-05-28 21:12:16.408 (iDetect) Calling message handler 'onHeartbeat'.
2018-05-28 21:12:16.408 (iDetect) devid2domid: {}
2018-05-28 21:12:16.408 Error: (iDetect) 'onHeartbeat' failed 'AttributeError':''BasePlugin' object has no attribute 'routercmdline''.
2018-05-28 21:12:16.409 Error: (iDetect) ----> Line 333 in /home/pi/domoticz/plugins/Domoticz_iDetect/plugin.py, function onHeartbeat
2018-05-28 21:12:16.409 Error: (iDetect) ----> Line 291 in /home/pi/domoticz/plugins/Domoticz_iDetect/plugin.py, function onHeartbeat

[EscApe]

Hi omnisxin,

What platform are u using for Domoticz (linux, windows, mac)?
According to your log you are using password based authentication. Did you install sshpass?

I have added soms checking for ssh and sshpass, and some more logging to the plugin. Can you install the new version and have a look at the new logs?

[omnisxin]

Hi
my platform is pi and raspbian, not sure sshpass has been indtalled,how to check it? even else I use rsa with no password,it seems same error.

[EscApe]

sshpass is not installed be default. To install it run:

Code: Select all

sudo apt-get update
sudo apt-get install sshpass

Key authentication requires a more complex setup.

[omnisxin]

Thanks for your replay,
I think all key authentication setup has been done as you described, but the same result, so I added password,as you see ,unforturenately nothing changed
l'll try install sslpass and give you feedback.

thanks again

[EscApe]

Please also upgrade de plugin to the latest version. I suspect the problem is authentication to the router, but the new logging will provide more info.

[quack3d]

Possible to make this work on a Windows Domoticz installation?

Windows 10 now has OpenSSH built in and I've created keys. When I add the iDetect hardware in Domoticz the logs say:

2018-06-01 17:07:32.474 Error: (iDetect) [ssh -V] will not run but is required! (error: [WinError 2] The system cannot find the file specified)
2018-06-01 17:07:32.490 Error: (iDetect) SSH subprocess failed with error: [WinError 2] The system cannot find the file specified
2018-06-01 17:07:32.490 Error: (iDetect) Could not determine router capabilities
2018-06-01 17:07:32.490 Error: (iDetect) 'onStart' failed 'KeyError'.

It also keeps restarting all the other hardware devices it seems.

[EscApe]

Hi quack3d,

In theory it should work. I have tested the ssh command on windows and it runs. Could you check if the domoticz service has the right PATH to find the ssh command? You could also try to replace (all) ssh commands in the plugin with <fullwindowspathto>ssh. Please let me know if that works.

I wouldn't expect a failing plugin to restart other hardware and have never seen this on my setup, but i'm running linux.

[quack3d]

Not sure how to check the path of the Domoticz service. I tried using nssm from https://nssm.cc/ to edit the Domoticz service and added 'ssh=C:\Windows\System32\OpenSSH\' to the Environment tab. Didn't help though. Same error.

Tried using the whole path in the plugin.py file instead. Didn't help. Had to use shell=True all places with subprocess.run (tip found here https://stackoverflow.com/questions/302 ... le_rich_qa).

However, now I get:

018-06-02 00:14:31.927 (iDetect) Returncode ssh command: 1
2018-06-02 00:14:31.927 (iDetect) Output from router: b''
2018-06-02 00:14:31.927 Error: (iDetect) Router command failed with returncode: 1
2018-06-02 00:14:31.927 Error: (iDetect) Raw data from router: b''
2018-06-02 00:14:31.927 Error: (iDetect) Could not determine router capabilities
2018-06-02 00:14:31.943 (iDetect) Creating device 'Anyone'.
2018-06-02 00:14:31.943 (iDetect) Device created fot general/Anyone presence
2018-06-02 00:14:31.943 (iDetect) devid2domid: {}
2018-06-02 00:14:31.943 (iDetect) monitormacs: {'phone1': 'C2:D2:02:02:02:02'}
2018-06-02 00:14:31.943 (iDetect) phone1 not in known device list
2018-06-02 00:14:31.943 (iDetect) creating device for phone1 with unit id 2
2018-06-02 00:14:31.943 (iDetect) Creating device 'phone1'.
2018-06-02 00:14:31.943 Error: (iDetect) 'onStart' failed 'KeyError'.

I also have this: [C:\Windows\System32\OpenSSH\ssh -V] will run, but raises an error: 1

I think I need to specify where I store the key.pub I created, no? Currently it's in C:\Users\name\ . I also tried to rename it and put it in C:\Users\name\.ssh\id_rsa.pub . The content of the .pub is pasted into the router config page field called Authorized Keys (asuswrt-merlin on a AC68U).

[EscApe]

I can confirm the error messages. However, the "[C:\Windows\System32\OpenSSH\ssh -V] will run, but raises an error: 1" might just mean the shell you added (shell=True) is not directly passing on the errors. The underlying problem could still be the same. Python seems unable to run the ssh.exe (even if it is in its PATH, i checked).

This will take a lot more exploring for which i do not have the time right now. If you can figure out the right approach, then I can see if your changes can be incorporated in the plugin. Otherwise Windows compatibility will be investigated at a later time. Sorry.

[EscApe]

Python on Windows seems te have no access to the C:\Windows\System32\OpenSSH directory. I did some manual testing in the Python window (ran the same tests again after starting Python window as Administrator).

Code: Select all

>>> test = subprocess.run(["dir", "C:\Windows\System32\OpenSSH"], shell=True)

Will result in "File not Found"

Code: Select all

>>> test = subprocess.run(["dir", "C:\Windows\System32"], shell=True)

Will result in directory listing

I don't see any restrictions in the directory's properties that would explain this.
Maybe someone with more Windows knowledge can help us???

[quack3d]

Myabe this? : https://stackoverflow.com/questions/309 ... le_rich_qa

I tried using this in a shell:

Code: Select all

import ctypes
class disable_file_system_redirection:
    _disable = ctypes.windll.kernel32.Wow64DisableWow64FsRedirection
    _revert = ctypes.windll.kernel32.Wow64RevertWow64FsRedirection
    def __enter__(self):
        self.old_value = ctypes.c_long()
        self.success = self._disable(ctypes.byref(self.old_value))
    def __exit__(self, type, value, traceback):
        if self.success:
            self._revert(self.old_value)


disable_file_system_redirection().__enter__()

Then the test = subprocess.run(["dir", "C:\Windows\System32\OpenSSH"], shell=True) worked.

[EscApe]

Great find! Maybe it is possible to prevent redirection for the entire Domoticz and/or python environment?
Seems this might also affect other plugins and Domoticz itself (if for instance calling an external program from lua).

Adding this to the plugin is not a priority for me. It would take some time to integrate this code and not let it interfere with linux installations. I might have to include some form of OS detection and also I want to keep the number of imported modules as low as possible.

You could of course add it to your local copy. I would be very interested to hear if it fixes things.

[quack3d]

I added it between import subprocess and class BasePlugin: .

The only thing I get in the logs now is:

2018-06-03 18:24:31.982 (iDetect) Started.
2018-06-03 18:24:32.084 (iDetect) Initialized version 0.3.1, author 'ESCape'
2018-06-03 18:24:32.084 (iDetect) Entering work loop.

No new device is created.

[EscApe]

I got the same result in different section of the plugin. Seems like it won't be that easy to manipulate the OS from within the plugin. I think the plugin or even the plugin framework is crashing/hanging without logging the failure(?)

Globally disabling the redirection might be the better direction to investigate.