Python plugin: Presence detection from wireless router

[wizzard72]

I have been trying to get the loop working on the commandline. So far I got the following working:

Code: Select all

for i in 1 2 3; do qcsapi_sockrpc get_station_mac_addr wifi0 $i; done

[EscApe]

That was indeed the idea behind the seq command used, but seq is not available on asuswrt. See my previous post. It offers a different approach that hopefully works.

[wizzard72]

@wizzard72

My bad The router os does not have the seq command.

If that was the only problem, then this should work (at least the loop works on my asus with dummy echo commands)

Code: Select all

pollscript=pollscript + ";concount=$(qcsapi_sockrpc get_count_assoc " + knownif + ");i=1;while [[ $i -le $concount ]]; do qcsapi_sockrpc get_station_mac_addr " + knownif + " $i;i=$((i+1));done"

If it doesn't can you try the following script on the router itself and share the output?

Code: Select all

concount=$(qcsapi_sockrpc get_count_assoc wifi0)
i=1
while [[ $i -le $concount ]]; do
        qcsapi_sockrpc get_station_mac_addr wifi0 $i
        i=$((i+1))
done

If the script on the router works but the plugin doesn't then i'm very interested in the debug log. You can share it as PM if you want to keep mac addresses and such secret.

EDIT: Did you try if get_station_mac_addr is indeed faster (real-time'ish) then get_assoc_records? Otherwise getting this loop to work might be a waste of time for both of us.

I missed this post

This is working:

Code: Select all

pollscript=pollscript + ";concount=$(qcsapi_sockrpc get_count_assoc " + knownif + ");i=1;while [[ $i -le $concount ]]; do qcsapi_sockrpc get_station_mac_addr " + knownif + " $i;i=$((i+1));done"

Edit:
The 'get_assoc_records' is a longer list and don't update regular, so this is not useful for device detection

[EscApe]

This is working:

Code: Select all

pollscript=pollscript + ";concount=$(qcsapi_sockrpc get_count_assoc " + knownif + ");i=1;while [[ $i -le $concount ]]; do qcsapi_sockrpc get_station_mac_addr " + knownif + " $i;i=$((i+1));done"

That's good news!
So the AC87 is now queried on both chipsets and the plugin is responding quickly to absence/presence?

Thanks again for all of you digging around for information en experimenting! I think the plugin should be ready for "production" now, but would love to hear from other testers if they encounter any trouble on the AC87 or other routers.

The latest changes and some extra error handling are in this new version. It is running fine on my (single brand chipset) AC86 and should also work for the AC87 and many other routers. I'm not sure if anybody else is testing, but if i don't get any complaints i will soon publish this version on GitHub.

For now it should still be considered 'experimental' but i would like to encourage everyone to try this version and share your findings. You can easily go back to the previous version, which is still on GitHub.

(remove the .txt extension and replace the original plugin.py with this one. It still needs the other (icon) files included with the original plugin.

[wizzard72]

That's good news!
So the AC87 is now queried on both chipsets and the plugin is responding quickly to absence/presence?

Yes it's working perfectly. It's fast and accurate. Thank you for adjusting the plugin so it works with the AC87U. It's very appreciated.

[mvzut]

I can also confirm it works on my AC87U!

However, for normal use I have to revert to the #forcegeneric flag, as it will not detect devices which are picked up by my (wired) second access point. Would be great if a better solution could once be found for this. Adding the plugin a second time for my other access point probably won't work, it's a pretty old router. Plus it gives me double the amount of switches which requires extra logic.

[Prutsium]

I can also confirm it works on my AC87U!

However, for normal use I have to revert to the #forcegeneric flag, as it will not detect devices which are picked up by my (wired) second access point. Would be great if a better solution could once be found for this. Adding the plugin a second time for my other access point probably won't work, it's a pretty old router. Plus it gives me double the amount of switches which requires extra logic.

+ on that dealing with the same issue with an external AC87 and my own AC68.

Would be great if the ETH devices would also be picked up instead of using #forcegeneric

[wizzard72]

Maybe something for the roadmap:
* "iDetect - Anyone" switch is made (created) by the plugin and is controlled bij the plugin. If there is another presence method, for example bluetooth, it would be nice that these dummy switches are monitored by the plugin, so that the Anyone switch is not switched off while the bluetooth switch(es) has the state "On".

[wizzard72]

This is working:

Code: Select all

pollscript=pollscript + ";concount=$(qcsapi_sockrpc get_count_assoc " + knownif + ");i=1;while [[ $i -le $concount ]]; do qcsapi_sockrpc get_station_mac_addr " + knownif + " $i;i=$((i+1));done"

That's good news!
So the AC87 is now queried on both chipsets and the plugin is responding quickly to absence/presence?

Thanks again for all of you digging around for information en experimenting! I think the plugin should be ready for "production" now, but would love to hear from other testers if they encounter any trouble on the AC87 or other routers.

The latest changes and some extra error handling are in this new version. It is running fine on my (single brand chipset) AC86 and should also work for the AC87 and many other routers. I'm not sure if anybody else is testing, but if i don't get any complaints i will soon publish this version on GitHub.

For now it should still be considered 'experimental' but i would like to encourage everyone to try this version and share your findings. You can easily go back to the previous version, which is still on GitHub.

(remove the .txt extension and replace the original plugin.py with this one. It still needs the other (icon) files included with the original plugin.

This updated plugin works good also

[BarryT]

In here the AC87U is not working correctly as well.
I have to use the #forcegeneric option.

Normal on a AC87U using as dhcp server with a AC68U as AP, does not seen anybody:

Code: Select all

2018-08-29 17:55:54.277 Status: (IemandThuisDetector) Started.
2018-08-29 17:55:54.635 Status: (IemandThuisDetector) Entering work loop.
2018-08-29 17:55:54.637 Status: (IemandThuisDetector) Initialized version 0.4.5, author 'ESCape'
2018-08-29 17:55:55.357 Status: (IemandThuisDetector) Using chipset specific (wl) command on router for interfaces ['eth1']
2018-08-29 17:55:55.357 Status: (IemandThuisDetector) Using chipset specific (qcsapi_sockrpc) command on router for interfaces ['wifi0']
2018-08-29 17:56:04.619 Status: (IemandThuisDetector) Homecount=0; Gonecount=0; Totalmacs=2
2018-08-29 17:56:04.619 Status: (IemandThuisDetector) Awaiting confirmation on absence of 2 devices

#Forcegeneric, working:

Code: Select all

2018-08-29 17:58:09.767 Status: (IemandThuisDetector) Started.
2018-08-29 17:58:10.111 Status: (IemandThuisDetector) Entering work loop.
2018-08-29 17:58:10.113 Status: (IemandThuisDetector) Initialized version 0.4.5, author 'ESCape'
2018-08-29 17:58:10.634 Status: (IemandThuisDetector) Using generic (brctl) instead of chipset specific command on router (slower response to absence)
2018-08-29 17:58:20.193 Status: (IemandThuisDetector) Homecount=1; Gonecount=1; Totalmacs=2
2018-08-29 17:58:20.193 Status: (IemandThuisDetector) Awaiting confirmation on absence of 1 devices

[EscApe]

@wizzard72

You can utilize the 'Override' switch for something like that. Set the Override button to 'override indefinitely' and control it through some other script/api.

[EscApe]

@BarryT

Actually it is working as intended, since idea behind the plugin is directly monitoring the wifi chipset from the OS of the device. The router in your case can not monitor a chip that is in a different device.

If you only use wifi on the AP then you should let the plugin connect to the AP instead of the router. The plugin is all about wifi chipsets and has nothing to do with dhcp or routing.

[BarryT]

@BarryT

Actually it is working as intended, since idea behind the plugin is directly monitoring the wifi chipset from the OS of the device. The router in your case can not monitor a chip that is in a different device.

If you only use wifi on the AP then you should let the plugin connect to the AP instead of the router. The plugin is all about wifi chipsets and has nothing to do with dhcp or routing.

Ah, okay.. Thnx
It is working on the ac87, but with the generic option.
I also did a test on the 68, but this is only an access point without dhcp, so it will not see any clients

Will try this for a while, i hope its better then my old arp script that not fit with my s8.
The s8 is going to sleepmode after not using it, so the arp will be disconnected sometime.

[EscApe]

@prutsium & mzut

Thanks for the confirmation!

I guess it would be possible to still use one of the generic commands like brctl even if the hardware is also being monitored directly (little programming required). But i don't think it will offer you any advantage over the #forcegeneric option.

If i understand correctly devices are connecting to the AP and to the Router(?)
The generic commands would detect presence in both cases quickly. It will be slow in detecting when you leave (absence). The chipset monitoring would detect absence quickly, but the generic command (like brctl) would simply contradict that as long as it thinks you are still at home (it 'sees' the same mac addresses).

The #forcegeneric approach is probably the best option right now. Otherwise we would still need acces to the chipset management tools in the OS'es of both the Router and the AP.
Even if the plugin would let you specify multiple routers, it would also have to detect the capabilities for each one and if even one of them falls back to generic tools the same problem (contradicting info) would occur.

All in all it is not impossible, but it would take a lot of time to implement and would benefit only a few very specific setups. Anyone is off course welcome to fork the current plugin and give it a go... I will fully support him/her.

[mvzut]

Before I used this plugin, I was using a shell script (which I found somewhere) which ran locally on my router. This script was using the contents of /proc/net/arp to check if devices were online or not. It then set the corresponding virtual switches in Domoticz via http commands. This was cumbersome to setup and maintain (which is the reason why I switched to the plugin now), but I'm pretty sure it was slightly faster to respond to absence than the #forcegeneric method in this plugin.

Although my programming skills are limited, I see in the plugin's code that arp is one of the methods you can use for the #forcegeneric method. But when I start the plugin, it says it's using the brctl method. Why do you give this method priority over the arp method? Is there any way I can override this? Maybe you can implement additional flags like #forcearp, #forceprocarp and #forcebrctl? In this way, advanced users can experiment which method works best for them.

[EscApe]

Just published a new version to GitHub. It includes the multiple chipset support and adds some more error handling.

@mvzut: I also added an (undocumented) #forcearp option. Use it instead of #forcegeneric to ignore the hardware tools and the brctl method. In my experience ARP is unreliable for absence detection. It took minutes and sometimes (many) hours to finally detect someone had disconnected from my AC68 and AC86. Brctl turned out to be much more reliable in detecting absence, but can occasionally report absence if a connection is shortly dropped. That can be fixed with a higher grace period (more confirmation polls) in the plugin settings. ARP might behave better on other routers.
If you can keep an eye on the arp reliability i would love to hear how it behaves in your setup over a longer period of time.

[mvzut]

If you can keep an eye on the arp reliability i would love to hear how it behaves in your setup over a longer period of time.

Great!
You might be right that brctl is more reliable in the end, but since my previous setup worked pretty acceptable, I'm going to check the arp solution for a while. I'll update you with my findings!

[sapnho]

Hi EscApe, I am using the Unifi Wifi system with a Cloud Key controller and three Access Points.

I can connect to every single Unifi access point (they all have a different IPs) to get accurate information about the phone presence but not the Unifi Cloud Key controller to get the status.

I checked IP address, user name and passwords by ssh'ing into the Cloud Key controller from the Terminal and I am getting connected, but no presence data seems to be passed on to Domoticz.

Any suggestion what I should try? Thanks for your work!

[hjzwiers]

I have an Asus AC86U and AC68U now working in AiMesh configureation. The AC86U is the main router, and now only the phones are detected connected to the main router, and not the mesh node. Any way in the future it can "see" both? Probably too much to want

Thnx

[EscApe]

@sapnho & hjzwiers

In both your cases i think the #forcegeneric option is the best way to go for now, unless you can find a command on your router that gives you a real-time inventory of all wireless devices connected to mesh members. The generic commands (like brctl) look beyond the router itself. Chipset specific wifi tools on the router (like wl) don't.

The strength of this plugin is the direct communication with the hardware in the router. I know of no tools that do the same for mesh networks, but there might be some proprietary tools in the (main) router. If you can find a reliable method then i can incorporate that into the plugin.

How bad is the #forcegeneric approach in a mesh setup? Are the response times acceptable?