Python plugin: Presence detection from wireless router

[EscApe]

I have added a (crude) tracker for Netgear Orbi to the 'modular' (beta) branch on GitHub.

This mainly serves as a proof of concept and example on how to add other types of (non-ssh) trackers to iDetect. The Netgear Orbi is queried using https.
I do not own or use an Orbi, but had acces to one and could test it successfully. Real Orbi owners are invited to further enhance the tracker module.

Hopefully the now available examples (various custom ssh models, ping tracker, orbi tracker) will inspire tinkerers and owners of various router makes and models to create a module for their router and share it as a pull request on GitHub. If you need any help on that please leave a message (I know the documentation is lacking )

EDIT: I have added some documentation on how to add your own class to fake_tracker.py in the trackers directory.

[Geitje]

So cool you're developing in advance of my requests probable! I'm not really experienced in these things so most likely would not have succeeded on my own in this. Did some pi work this week, but need a part from the store before I can make my backup and start testing, but when I have it I will certainly try it on my Orbi...

[EscApe]

@Geitje,

Thanks! Without the plugin by pipiche I would not have known how to query the Orbi system, so I would like to share credit here
Best of luck setting up your pi and welcome to the addictive world of tinkering

[drwurn]

Hi,

I have the plugin working on my orbi setup.
There are a lot of device changes, both on ios and android at exact the same times.

Screenshot_20191201-113054_Samsung Internet.jpg (77.6 KiB) Viewed 2740 times

[EscApe]

There are a lot of device changes, both on ios and android at exact the same times.

Did you have a look at the logs? Are there any errors around the time of the changes? If not my first guess would be that your poll times and grace period(s) are about the same. A tag will show offline if presence was not confirmed within its grace period, so if both are set to eg 60 seconds the confirmation might be a few (milli)seconds late because polling takes time too. There are some other timing things at play (within a 5 second range) that can make it seem to work most of the time, but just miss its deadline occasionally. So general rule of thumb: make sure there are serval polls within the grace period.

In my (ssh) setup the grace period is A few seconds more than four times the poll interval. This way even an occasional poll failure or WiFi disconnect will not cause a false negative. I don’t have a lot of experience with the orbi but am still running a test over a vpn connection to my parents house (they have an orbi) with a grace period of 120 seconds and poll interval of 50 seconds. So there are always two polls within the grace period. It has been rock solid for two days, even over a vpn. The same ratio with shorter times should be no problem on a local network.

[drwurn]

Didn't have any errors.
Changed the grace to 45 and poll is 15. Now gonna look how this is holding up.

[Geitje]

Today I installed the plugin an configured it. I prefer using SSH public auth., but for now use password.
The plugin initializes correctly, but cannot reach my Orbi router for some reason. I logged out of my router before starting the plugin.

Code: Select all

2019-12-01 21:44:39.478 (iDetect Wifi presence detection) Debug logging mask set to: PYTHON
2019-12-01 21:44:39.478 (iDetect Wifi presence detection) Not running on Windows
2019-12-01 21:44:39.503 (iDetect Wifi presence detection) The OS user profile running domoticz is: root
2019-12-01 21:44:39.536 (iDetect Wifi presence detection) Checking if [ssh -V] will run: OK
2019-12-01 21:44:39.536 (iDetect Wifi presence detection) Parsing user and optional keyfile from:admin
2019-12-01 21:44:39.561 (iDetect Wifi presence detection) Checking if [sshpass -V] will run: OK
2019-12-01 21:44:39.561 (iDetect Wifi presence detection) Monitoring {'PhoneMTel': {'ignore': False, 'mac': 'XX:XX:XX:XX:XX:XX', 'lastseen': None}, 'LaptTel': {'ignore': False, 'mac': 'XX:XX:XX:XX:XX:XX', 'lastseen': None}} for presence.
2019-12-01 21:44:39.562 (iDetect Wifi presence detection) 2 of them will control the Anyone home switch
2019-12-01 21:44:39.562 (iDetect Wifi presence detection) Router configuration:192.168.1.1
2019-12-01 21:44:39.562 (iDetect Wifi presence detection) Using password instead of ssh public key authentication for 192.168.1.1 (less secure!)
2019-12-01 21:44:39.562 (iDetect Wifi presence detection) Fetching data from 192.168.1.1 using: sshpass -p **secret** ssh -o StrictHostKeyChecking=no -o ConnectTimeout=3 -p22 [email protected]
2019-12-01 21:44:39.607 (iDetect Wifi presence detection) SSH subprocess for 192.168.1.1 failed with error (255):b''
2019-12-01 21:44:39.607 (iDetect Wifi presence detection) SSH command on 192.168.1.1 took 45 milliseconds.
2019-12-01 21:44:39.607 (iDetect Wifi presence detection) Could not retreive available commands on 192.168.1.1
2019-12-01 21:44:39.608 (iDetect Wifi presence detection) Router initialized as:{'192.168.1.1': {'initialized': False, 'errorcount': 1, 'user': 'admin', 'prospone': datetime.datetime(2019, 12, 1, 21, 44, 51, 607871), 'port': 22, 'cmd': ''}}
2019-12-01 21:44:39.608 (iDetect Wifi presence detection) iDetect Wifi presence detection - PhoneTel is stil in use
2019-12-01 21:44:39.608 (iDetect Wifi presence detection) iDetect Wifi presence detection - LaptTel is stil in use
2019-12-01 21:44:39.608 (iDetect Wifi presence detection) Devicenames and their Domoticz (subdevice) index: {'PhoneTel': 2, 'LaptTel': 3}
2019-12-01 21:44:39.608 (iDetect Wifi presence detection) MAC addresses to monitor: {'PhoneTel': {'ignore': False, 'mac': 'XX:XX:XX:XX:XX:XX', 'lastseen': None}, 'LaptTel': {'ignore': False, 'mac': 'XX:XX:XX:XX:XX:XX', 'lastseen': None}}
2019-12-01 21:44:39.608 (iDetect Wifi presence detection) Plugin initialization done.
2019-12-01 21:44:39.203 Status: (iDetect Wifi presence detection) Started.
2019-12-01 21:44:39.473 Status: (iDetect Wifi presence detection) Entering work loop.
2019-12-01 21:44:39.475 Status: (iDetect Wifi presence detection) Initialized version 0.7.7, author 'ESCape'
2019-12-01 21:44:49.521 (iDetect Wifi presence detection) Prosponed connection to 192.168.1.1 for 0:00:02.086432
2019-12-01 21:44:49.521 (iDetect Wifi presence detection) Did not retreive WLAN information from any router
2019-12-01 21:44:49.522 (iDetect Wifi presence detection) Entire poll took 0% of the poll interval time (1 milliseconds)
2019-12-01 21:45:04.508 (iDetect Wifi presence detection) 192.168.1.1 was not properly initialized. Retrying to get router capabilities (and skipping this poll round).
2019-12-01 21:45:04.508 (iDetect Wifi presence detection) Using password instead of ssh public key authentication for 192.168.1.1 (less secure!)
2019-12-01 21:45:04.508 (iDetect Wifi presence detection) Fetching data from 192.168.1.1 using: sshpass -p **secret** ssh -o StrictHostKeyChecking=no -o ConnectTimeout=3 -p22 [email protected]
2019-12-01 21:45:04.553 (iDetect Wifi presence detection) SSH subprocess for 192.168.1.1 failed with error (255):b''
2019-12-01 21:45:04.553 (iDetect Wifi presence detection) SSH command on 192.168.1.1 took 44 milliseconds.
2019-12-01 21:45:04.553 (iDetect Wifi presence detection) Could not retreive available commands on 192.168.1.1
2019-12-01 21:45:04.553 (iDetect Wifi presence detection) Routerinfo:{'initialized': False, 'errorcount': 2, 'user': 'admin', 'prospone': datetime.datetime(2019, 12, 1, 21, 46, 4, 553818), 'port': 22, 'cmd': ''}
2019-12-01 21:45:04.553 (iDetect Wifi presence detection) Did not retreive WLAN information from any router
2019-12-01 21:45:04.554 (iDetect Wifi presence detection) Entire poll took 0% of the poll interval time (45 milliseconds)

[EscApe]

@geitje

Only the ‘modular’ branch (still in beta) supports orbi. You can install it from github. You are using the older (stable) version.

As far as I know the orbi does not support ssh at all. The modular branch gets the connected mac addresses from orbi using https.

[pipiche]

@geitje

Only the ‘modular’ branch (still in beta) supports orbi. You can install it from github. You are using the older (stable) version.

As far as I know the orbi does not support ssh at all. The modular branch gets the connected mac addresses from orbi using https.

No orbi do not accept ssh, eventually telnet (if you enable it)

Orbi do not allow concurent access to the web server, so for instance if you have The plugin getting access to Orbi and at the same time you are accessing the Orbi via smartphone apps or Web, then you get in the concurent access and one of the https request will fail

[Geitje]

Ah, I thought I was on modular, not so. Now I am, and managed to get it working . Delay when removing/connecting my phone from/to the network is acceptable: around 3mins. Tomorrow testing will continue...

[Geitje]

I use idetect with my Orbi system now for some days. At first with the standard settings, I had the same problem as @drwurn: many on/offs. Now grace period is 130s, poll time 30s. It works flawless and stable! I like this plugin...

I have one request. Is want to use this plugin for my IPcams to send snapshots when not at home, and not sending them when at home. Right now, if I arrive at home, I get many snapshots during the grace period. In fact most of the time after about 6 minutes it changes my status to "at home". I can shorten the grace-period, but then I get more false on/off statuschanges. My question: is it possible to be able to set separate grace periods for online/offline status change? That way, the grace period for getting online I can set very brief (I suppose there is no false positive change to getting online), while the grace period for getting offline should be longer, as those are often false positive...

[EscApe]

is it possible to be able to set separate grace periods for online/offline status change?

The grace period only affects the offline status. A device will be shown online immediately after it is seen by the poll, so in your case it should take 30 seconds at most. Any other delay might be caused by the time it takes the router to show the device in the list. Directly querying the chipset is mostly instant, but 6 minutes seems very slow for any method. it seems unlikely that the orbi is taking that long. No (connection) errors in your log?

[Geitje]

OK, did not know that. I did not check delay for going online very precise. I was supposing it would be the same period as going offline. I will check and report if it is that long. EDIT: indeed for going online it is only 30 seconds or so....

The grace period only affects the offline status.

[EscApe]

Since the new version has been running stable on my setup and I have not received any complaints from other users I am merging the 2.0 'modular' version into the master branch on GitHub. The new version might surprise users that have not followed this thread and are auto updating. If so: pleas have a look at https://github.com/d-EScape/Domoticz_iDetect for the new requirements, features and configuration syntax.

Spoiler: you need to install the paramiko python module for any ssh type of detection to continue working.

[Mrrodz]

Hi im using a unify usg and 2 access points, it has been working flawlessly for a very long time. since the update i get

Error: (iDetect) 192.168.10.4 ====> SSH returned error:Not supported

it still seems to detect my phone on the network but unsure of this error.

any idea where i can look to resolve this.

thanks

[Prutsium]

Hi im using a unify usg and 2 access points, it has been working flawlessly for a very long time. since the update i get

Error: (iDetect) 192.168.10.4 ====> SSH returned error:Not supported

it still seems to detect my phone on the network but unsure of this error.

any idea where i can look to resolve this.

thanks

Just read the message above your message: Spoiler: you need to install the paramiko python module for any ssh type of detection to continue working.

[Mrrodz]

Hi Prutsium,

i have already installed paramiko via pip. Detection is working but im getting this error and unsure why

thanks

[EscApe]

Hi Mrrodz,

Good to hear that detection is working. To diagnose the error message I will need some more information. Can you enable debug mode and share the debug log? I'm especially interested in the configuration used and the commands send and response received over ssh (you can remove any passwords and/or share as PM for safety).

A configuration like <router ip>#type=unifiusg-arp should only execute ishow arp (old style configuration should also work)
Do you get any messages/errors when running ishow arp from the command line on the router?

One thing that has changes is that I removed some (seemingly) redundant path and exit statements around some commands. That may or may not be a problem on the unify. You could try the following configuration to test this:

Code: Select all

<routerip>#ssh=ishow arp;exit

Please share the results.

[Rob67ert]

Hi,

I have installed the plug-in. A nice learning curve for me. Got it to work.
But in my log i found: Error: (iDetect) 192.168.178.2 ====> SSH returned empty response. Transport active: True
Is there anything that is not correct?

Robert

[Geitje]

You say you got it to work, so it means it detects your device's presence?
I get errors now and then, I suppose my router (Netgear Orbi) sometimes is not replying (busy) or someone else is using the account (my router only allows 1 login a a time). Could these be the case for your router also? For me these errors are not causing the plugin to malfunction, in your case?