Python plugin: Presence detection from wireless router

[Johan1974]

Hello,
I am trying to install Domoticz iDetect v2 on my Raspberry pi4, but when installing Paramiko

Code: Select all

sudo pip3 install requests paramiko 

I get the following error:

Code: Select all

pip is configured with locations that require TLS/SSL, however the ssl module in Python is not available.
Collecting paramiko
  Could not fetch URL https://pypi.python.org/simple/paramiko/: There was a problem confirming the ssl certificate: Can't connect to HTTPS URL because the SSL module is not available. - skipping
  Could not fetch URL https://www.piwheels.org/simple/paramiko/: There was a problem confirming the ssl certificate: Can't connect to HTTPS URL because the SSL module is not available. - skipping
  Could not find a version that satisfies the requirement paramiko (from versions: )
No matching distribution found for paramiko

What's going wrong ...

[EscApe]

Hi Johan1974,

This does not appear to be a problem related to the plugin. Googling the first line of the error messages will give you a lot of leads to investigate.

[Geitje]

I'm using this plugin for months now on my Orbi. Works flawlessly!

I would like to do a request. It would be a great extension to the plugin I think. I would like to monitor my wifi network for new devices that connect. For example to get a notification when a unknown device connects to my LAN. This is a long wished feature for me, and I think it is not too hard to add this to this plugin? It would be sort of opposite to the function iDetect has now. Checking for unknown devices instead of checking for known devices. Please let me know what you think of this feature...

I know Orbi has an "access control" function. I have been using this so far, but it has many known bugs, it is excluding devices randomly. Also I could use Netgear Armor, but I find that too much a giant of a package which checks al my tracks.

[EscApe]

Hi Geitje,

Nice to hear you like the plugin.

Thanks for your suggestion.However, I think a dedicated tool would better suit your use case and would be easier to write than adding this to my plugin. Here is why:

As you already stated "It would be sort of opposite to the function iDetect has now". It might be possible to implement an alarm device that triggers when a MAC address is detected that is not in the list of MAC addresses to monitor. However, that would mean you will have to monitor every(!) know MAC address in your network. For a lot of users that would lead to very long configurations and unnecessary devices in Domoticz. They might even exceed the technical limitations of the configuration form or database (have nog checked this) or in extreme cases the maximum number of devices a plugin can add. It gets worse for users that use arp or brctl as monitoring method. You would have to add every network device (wired as well as wireless) as a known device. If other tracker methods are ever added (like bluetooth tracking) it would become even more complicated.

[Geitje]

You are right, a new plugin would be better. Are you willing to write it (I myself have not the knowledge to do so...)? Large parts of the code can be the same I supppose?. I can help debugging it...

[EscApe]

You are right, a new plugin would be better. Are you willing to write it (I myself have not the knowledge to do so...)? Large parts of the code can be the same I supppose?. I can help debugging it...

Hi Geitje,

Sorry, I don't have plans to develop anything like that.

Im also not sure of your use case will provide you with the safety you expect. Access control based on MAC-addresses does not offer any real security because MAC-addresses can be easily spoofed. Thats is why whitelisting know devices on the router is widely considered to only offer a false sense of security. If an attacker is capable of logging into your password protected wifi, then he will definitely be capable of some simple MAC-address snooping and spoofing. Assuming your password isn't "1234" or "welkom"

[Geitje]

Ok, thanks.

[mvzut]

@Melotron
It looks like the wireless interfaces are named eth*, so that should not be a problem. I don’t know why the wl command is not able to query them. Using the #type=forcegeneric configuration is a work-around. More research is needed to find the correct way to find connected clients on a ax88u router. The architecture might be totally different from other ASUS routers. Maybe you can find something on the web or experiment on the router itself? I will also hit google when i have some extra time.

Thanks alit! That helped.

Im going to make a few other tests on the router to see if I can it to work without the generic switch.
But Im no expert on it

I realize I am reacting to a very old post, but I was wondering if you, Melotron, or others have found a way to get the ASUS AX88U router working with iDetect without the #type=forcegeneric option? Mine is not working without it, it mentions a problem with ifconfig:

Code: Select all

SSH returned error:sh: ifconfig: not found

and

Code: Select all

SSH returned error:type -h <optional command> for help

Although ifconfig works fine when I log into the router's console.

[EscApe]

Hi mvzut,

You PATH might be different when logging on interactively. Can you run "type ifconfig" on the router to see where it's located? To test this theory you could hardcode the path in tracker_cli_helper.py (<path>/ifconfig instead of just ifconfig on line 19, 24, 30 etc... depending on your chipset).

I did not take into account that ifconfig could be outside of the PATH, so if your test confirms my suspicion I will try some possible fixes.

[HM31]

[...]So I changed the tracker command to 192.168.1.1#ssh=cat /tmp/clientlist.json [...]

Thx for the trick !
Works fine for me on an ASUS AiMesh network with 3 different routers with ASUS genuine firmware (RT-AC86U, RT-AC1900P and RT-AC3100).

Is there a way to have only one device with two different MAC addresses ? I don't care if my laptop is connected using dock-station or WiFi, my aim is just to know that the laptop is up and running without adding extra device.

[HM31]

Hi,

Just try to create a second hardware device because I naively thought I could create some sort of presence detection "by zone".

Then when I wanted to remove the physical device I had just created for test purpose and I got an message that the removal gave an error. I did it again (stubborn? Yes just a little ...) and there Domoticz no longer responding I restarted it and ... lost everything ...?! ;-(

Fortunately a few commands to copy my last backup db and everything came back (except the device which was causing me problem since it was not in my backup).

Is this plugin intended for multiple instances?

[EscApe]

Is this plugin intended for multiple instances?

It was never intended for multiple instances, but I see no reason why it should not work. Let alone fail in such a disastrous way. Glad you had a backup!

I have also seen some odd behavior using multiple (different) python plugins. Sometimes stopping a plugin crashes Domoticz. Other times it doesn’t, and there is not one particular order or combination that results in a reproducible fault. I have never seen it crash with just one plugin enabled. Maybe the plug-ins are not as isolated as we would like them to be.

As for this plugin. Your use case should have worked in theory. I don’t know why it didn’t.

[stephanvdplas]

I'm sorry to report your plugin doesn't work on my RuckusWireless R300.

Code: Select all

2021-06-05 10:05:27.742 Status: Set UserVariable Paneel_bit = 1
2021-06-05 10:06:18.681 iDetect hardware started.
2021-06-05 10:06:18.681 Status: iDetect: (iDetect) Started.
2021-06-05 10:06:18.681 Status: iDetect: (iDetect) Entering work loop.
2021-06-05 10:06:18.792 Status: iDetect: (iDetect) Initialized version 2.0, author 'ESCape'
2021-06-05 10:06:20.302 iDetect: (iDetect) Debug logging mask set to: PYTHON
2021-06-05 10:06:20.302 iDetect: (iDetect) Operation system is: win32
2021-06-05 10:06:20.332 iDetect: (iDetect) The OS user profile running domoticz is: nt authority\system
2021-06-05 10:06:20.332 iDetect: (iDetect) Parsing user and optional keyfile from:super
2021-06-05 10:06:20.332 iDetect: (iDetect) Getting icon requested for Anyone: idetect-home from file:ihome.zip
2021-06-05 10:06:20.337 iDetect: (iDetect) New image: ID: -1, Base: '', Name: , Description: ''
2021-06-05 10:06:20.342 iDetect: (iDetect) Getting icon requested for Override: idetect-override from file:ioverride.zip
2021-06-05 10:06:20.352 iDetect: (iDetect) New image: ID: -1, Base: '', Name: , Description: ''
2021-06-05 10:06:20.352 iDetect: (iDetect) Getting icon requested for Stephan: idetect-unithome from file:iunit.zip
2021-06-05 10:06:20.362 iDetect: (iDetect) New image: ID: -1, Base: '', Name: , Description: ''
2021-06-05 10:06:20.362 iDetect: (iDetect) start get or create wireless device
2021-06-05 10:06:20.362 iDetect: (iDetect) Stephan monitor tag_id:04:D6:AA:10:C2:E4, domoticz unit:2
2021-06-05 10:06:20.362 iDetect: (iDetect) start get or create wireless device
2021-06-05 10:06:20.362 iDetect: (iDetect) Lizette monitor tag_id:44:91:60:65:CF:A1, domoticz unit:3
2021-06-05 10:06:20.362 iDetect: (iDetect) Monitoring {'04:D6:AA:10:C2:E4': <plugin.tag_device object at 0x096BF370>, '44:91:60:65:CF:A1': <plugin.tag_device object at 0x096BF610>} for presence.
2021-06-05 10:06:20.362 iDetect: (iDetect) Tracker configuration:192.168.5.2
2021-06-05 10:06:20.362 iDetect: (iDetect) tracker:192.168.5.2
2021-06-05 10:06:20.362 iDetect: (iDetect) options:
2021-06-05 10:06:20.362 iDetect: (iDetect) 192.168.5.2 Tracker is of the ssh kind
2021-06-05 10:06:20.362 iDetect: (iDetect) 192.168.5.2 ====> SSH Fetching data using:
2021-06-05 10:06:20.362 export PATH=/usr/sbin:/usr/bin:/usr/local/sbin:/usr/local/bin:/sbin:/bin:$PATH
2021-06-05 10:06:20.362 type wl
2021-06-05 10:06:20.362 type iwinfo
2021-06-05 10:06:20.362 type wlanconfig
2021-06-05 10:06:20.362 type wl_atheros
2021-06-05 10:06:20.362 type qcsapi_sockrpc
2021-06-05 10:06:20.362 type ip
2021-06-05 10:06:20.362 type brctl
2021-06-05 10:06:20.362 type arp
2021-06-05 10:06:20.362 [ -f /proc/net/arp ] && type cat
2021-06-05 10:06:20.362 echo 0
2021-06-05 10:06:20.362 exit
2021-06-05 10:06:20.362 iDetect: (iDetect) 192.168.5.2 ====> SSH not connected ... connecting
2021-06-05 10:06:20.362 iDetect: (iDetect) 192.168.5.2 ====> SSH start connect
2021-06-05 10:06:20.342 Status: iDetect: (iDetect) Created device for Anyone with unit id 1
2021-06-05 10:06:20.352 Status: iDetect: (iDetect) Created device for Override with unit id 255
2021-06-05 10:06:20.362 Status: iDetect: (iDetect) Created device for Stephan with unit id 2
2021-06-05 10:06:20.362 Status: iDetect: (iDetect) Created device for Lizette with unit id 3
2021-06-05 10:06:21.192 Status: iDetect: (iDetect) 192.168.5.2 ====> SSH connection established
2021-06-05 10:06:26.312 iDetect: (iDetect) 192.168.5.2 Could not retreive available commands
2021-06-05 10:06:26.312 iDetect: (iDetect) 192.168.5.2 tracker will autodetect ssh cli
2021-06-05 10:06:26.312 iDetect: (iDetect) 192.168.5.2 Data will be received and interpreted by <bound method BasePlugin.onDataReceive of <plugin.BasePlugin object at 0x006EE5F0>>
2021-06-05 10:06:26.312 iDetect: (iDetect) Trackers initialized as:{'192.168.5.2': <trackers.ssh_autodetect.ssh_autodetect object at 0x096BF670>}
2021-06-05 10:06:26.312 iDetect: (iDetect) Plugin initialization done.
2021-06-05 10:06:26.312 Status: iDetect: (iDetect) 192.168.5.2 ====> SSH resetting connection
2021-06-05 10:06:26.312 Status: iDetect: (iDetect) Tracker activated:192.168.5.2 port 22, user: super, type: default and options: {}
2021-06-05 10:06:26.312 Error: iDetect: (iDetect) 192.168.5.2 ====> SSH failed with exception:
2021-06-05 10:06:28.113 iDetect: (iDetect) onHeartbeat called
2021-06-05 10:06:28.113 iDetect: (iDetect) 0 devices are present (excluding ignored devices)
2021-06-05 10:06:30.372 iDetect: (iDetect) 192.168.5.2 Not (yet) ready for polling
2021-06-05 10:06:38.082 iDetect: (iDetect) onHeartbeat called
2021-06-05 10:06:38.082 iDetect: (iDetect) 0 devices are present (excluding ignored devices)
2021-06-05 10:06:40.382 iDetect: (iDetect) 192.168.5.2 Not (yet) ready for polling
2021-06-05 10:06:48.072 iDetect: (iDetect) onHeartbeat called
2021-06-05 10:06:48.072 iDetect: (iDetect) 0 devices are present (excluding ignored devices)
2021-06-05 10:06:50.389 iDetect: (iDetect) 192.168.5.2 Not (yet) ready for polling
2021-06-05 10:06:58.072 iDetect: (iDetect) onHeartbeat called
2021-06-05 10:06:58.072 iDetect: (iDetect) 0 devices are present (excluding ignored devices)
2021-06-05 10:07:00.400 iDetect: (iDetect) 192.168.5.2 Not (yet) ready for polling
2021-06-05 10:07:08.072 iDetect: (iDetect) onHeartbeat called
2021-06-05 10:07:08.072 iDetect: (iDetect) 0 devices are present (excluding ignored devices)
2021-06-05 10:07:10.412 iDetect: (iDetect) 192.168.5.2 Not (yet) ready for polling
2021-06-05 10:07:18.112 iDetect: (iDetect) onHeartbeat called
2021-06-05 10:07:18.112 iDetect: (iDetect) 0 devices are present (excluding ignored devices)
2021-06-05 10:07:20.422 iDetect: (iDetect) 192.168.5.2 Not (yet) ready for polling
2021-06-05 10:07:28.083 iDetect: (iDetect) onHeartbeat called
2021-06-05 10:07:28.083 iDetect: (iDetect) 0 devices are present (excluding ignored devices)

I've written my own (powershell) script that does work, so for me it is no big deal, but if you want to incorperate RW devices, maybe we can troubleshoot together?

[EscApe]

I've written my own (powershell) script that does work, so for me it is no big deal, but if you want to incorperate RW devices, maybe we can troubleshoot together?

If you know the cli command to get a list of MAC addresses on the ruckus, then you can specify that command in the plugin configuration. See documentation on https://github.com/d-EScape/Domoticz_iDetect about custom commands (ssh= configuration parameter). The router specific command could also be added to the plugin as a custom tracker. I will gladly help you do that if you can figure out the correct command.

The autodetection works for most mainstream routers using a generic linux platform, but it doesn’t recognize brand specific OS commands. Can you share your script? Maybe it can tell us in which categorie the Ruckus falls.

[stephanvdplas]

If you know the cli command to get a list of MAC addresses on the ruckus, then you can specify that command in the plugin configuration. See documentation on https://github.com/d-EScape/Domoticz_iDetect about custom commands (ssh= configuration parameter). The router specific command could also be added to the plugin as a custom tracker. I will gladly help you do that if you can figure out the correct command.

The autodetection works for most mainstream routers using a generic linux platform, but it doesn’t recognize brand specific OS commands. Can you share your script? Maybe it can tell us in which categorie the Ruckus falls.

Here is my (powershell) script. It is a bit cumbersome, because we have to use a shell stream and cannot directly enter the box via SSH.
As you can see, it reads the output for 16 WLAN ID's (max 8 2,4GHz and max 8 5GHz SSID's are possible) and then parses the output looking for all defined MAC addresses. The definition part of the script is omitted for obvious reasons.
I hope this will help you.

Code: Select all

    $ruckussession = New-SSHSession -ComputerName $RuckusIP -Credential $cred
    $sshshellstream = New-SSHShellStream -Index $ruckussession.SessionId
    $sshshellstream.WriteLine($username)
    sleep 2
    $sshshellstream.WriteLine($pwd)
    sleep 2
    for ($i=0; $i -le 15; $i++) {$sshshellstream.WriteLine("get station wlan$i list")}
    sleep 2
    $wlanoutput = $sshshellstream.Read()
    sleep 2
    Get-SSHSession | Remove-SSHSession
    if ($wlanoutput -Like "*Ruckus R300*") 
    {
		Add-Content -Path $LogFileName -Value " " 
		Get-Date -Format "dddd MM/dd/yyyy HH:mm:ss" | Add-Content -Path $LogFileName -NoNewline
		for ($j=0; $j -lt $Macaddresses.length; $j++)
    	{
            $Macaddress = $Macaddresses[$j]
            $idxnr = $idxnrs[$j]
    		$Naam = $Namen[$j]
    		$currentstate = ((wget ('http://' + $domoticzIP + ':' + $domoticzPort + '/json.htm?type=devices&rid=' + $idxnr ) -UseBasicParsing).Content | convertfrom-json).result.status
		    if ($wlanoutput -Like "*$Macaddress*") 
            {
   	    		Add-Content -Path $LogFileName -NoNewline -Value ", $Naam is thuis" 
		    	if ($currentstate -eq 'Off') 
                {
					Write-Host "$Naam GAAT AAN"
	                wget ('http://' + $domoticzIP + ':' + $domoticzPort + '/json.htm?type=command&param=switchlight&idx=' + $idxnr + '&switchcmd=On') -UseBasicParsing
    			}
	    	} else 
            {
   	    		Add-Content -Path $LogFileName -NoNewline -Value ", $Naam is weg" 
    			if ($currentstate -eq 'On') 
                {
					Write-Host "$Naam GAAT UIT"
		    		wget ('http://' + $domoticzIP + ':' + $domoticzPort + '/json.htm?type=command&param=switchlight&idx=' + $idxnr + '&switchcmd=Off') -UseBasicParsing
			    }
    		}
	    }
    }

[EscApe]

It looks like Ruckus uses proprietary commands and “get station wlan$i list” is the relevant part here. Since I do not have access to or knowledge of the Ruckus command set you need te find a way to run the loop on the router itself in a single cli line. You could then configure the plugin “tracker” as:
<routerip>#ssh=<singlecommandline>

Just for testing you could try

Code: Select all

192.168.5.2#ssh=get station wlan1 list

For some more testing (without actual loop programming).

Code: Select all

192.168.5.2#ssh=get station wlan1 list;get station wlan2 list;get station wlan3 list

(The ruckus might not accept multiple commands separated by semicolon like Linux.)

I don’t know what the limit of the command line will be, but you could try te expand on this or at least try different numbers up to wlan15, to see if you can at least get some usable data. This way we can be sure it the ruckus ssh server is compatible with the paramiko ssh client and it’s a good starting point for further programming.

[THBR]

Hello,

I'm using this plugin for some months now on my Netgear R7000. Works now flawlessly without any special adjustment!

Only problem is that when I want to change some settings in the hardware plugin, for instance grace/poll time or debug mode, I can’t save these changes (by the “update” button), nothing happens. The workaround is: adding a new hardware plug in, restart Domoticz, delete the original pluging and restart Domoticz again....

Customizing and updating my other hardware plugins, for instance Zigbee2MQTT of Buienradar NL works fine.
Looks like the plugin is now somehow protected against overwriting (was not the case after the first install)?

My hardware/software
Domoticz version: 2021.1
Python Version: 3.7.3
Latest iDetect v2 plugin
RPi 3b
Netgear R7000 accespoint with latest Fresh Tomato firmware

Thanks, Theo (NL).

[EscApe]

Hoi Theo,

Saving a setup changes works just fine in my setup which is also Domoticz 2021.1 with python 3.7.3, so I don’t really have a starting point to investigate. Are there any debug messages? Maybe this could be caused by a database or domoticz corruption(?)

[THBR]

Hoi Theo,

Saving a setup changes works just fine in my setup which is also Domoticz 2021.1 with python 3.7.3, so I don’t really have a starting point to investigate. Are there any debug messages? Maybe this could be caused by a database or domoticz corruption(?)

Thanks for responding!

Could you please tell me where/how to find these debug massage(s)? Debug mode in the plugin is turned to On.
In the Domoticz log (under "All" and filter "Idectect") everything looks ok, no error messages.

[EscApe]

Maybe a debug log on Domoticz level wil reveal anything interesting, maybe it won't. It does not necessarily have te be an error! You can enable Domoticz debug logging on the command line (and thus started script). For al parameters have a look at https://www.domoticz.com/wiki/Command_line_parameters

You will need -loglevel (0 =All, 1 =Status+Error, 2 =Error) => 0
and -log <filepath>

I can not reproduce your situation with the same version, and no one else has reported anything similar. Both the plugin and Domoticz versions tested have been around for a while, so for now I it seems most likely that something in your setup got corrupted/broken. I know a re-install is a massive undertaking when you have a large setup, but you might want to consider It anyway.

There is one (far fetched!) thing I can think of, and that is a tracker that takes long to respond and a longer than usual timeout(?) That might cause Domoticz to wait for the plugin to stop and restart a bit longer, but it should stil restart with the new values... or end in an error message about the plugin failing to restart. How long have you waited for the page to reflect the button press? This seems very far fetched and mostly theoretical, since i kept most timeouts short (a few seconds) and all ssh connections are ended before the plugin stops.