Python plugin: Presence detection from wireless router

[Prutsium]

As my stupid cable provider provided me with a new router (Fritzbox 6490) i had to change my network setup.
Now the Asus is back to dumb AP mode and most functions are off (including the DHCP) so i was affraid i would lose all the presence functions.

But somehow the Asus is still able to find all devices in the network even when it's not acting as DHCP server and just a dumb gateway.
Even the devices that connect to the FritzBox directly seems to be found.

Pitty there is no option (anymore) to get SSH at the Fritzbox but at least this way it seems to work.

[EscApe]

@dextm80

De plugin uses SSH, not telnet. If that is also unavailable, then the plugin will indeed not work. Do you have another router in your setup? The arp tables might be a bit slower to update, but it might be usable.

[EscApe]

@Prutsium

The plugin does not use DHCP (arp, brctl etc have nothing to do with DHCP) .Is the plugin using arp or brctl on ANY of your routers (should be visible in the log)? That would explain why it detects devices connected tot other acces points. Arp and brctl info will propagate throughout the network, since switches and routers have to know where network clients are located (by mac address).

DHCP assigns IP addresses to clients... something the plugin doesn't care about.

But is always nice to find out something is still working when you didn't expect it to

[Prutsium]

Yupp forgot to mention i was using it (also before) on BRCTL due the setup at that time (Routers behind the Asus).

So anyway still happy it works as i depend on the presence detection heavily for my Domotica setup.

[dextm80]

@dextm80

De plugin uses SSH, not telnet. If that is also unavailable, then the plugin will indeed not work. Do you have another router in your setup? The arp tables might be a bit slower to update, but it might be usable.

yes, i ve a linksys and tplink router that is used it for switch

[Freekers]

Glad it's working

You might be able to tweak arp table management on the routerboard. Maybe set some time-outs(?) I don't know the platform, but it seems well documented and very tweakable. Refreshing the arp table on every polls seems a bit drastic. The plugin might miss mac addresses if the table isn't refreshed quick enough.

Querying the wireless interfaces will probably work much better than the generic arp command. You could investigate how to get the associated mac addresses from the wireless chipset. That is where the plugin is at is best.

Another question to really support RouterOS in the plugin is how to detect the routerboard as a routerboard with a command that will consistently identify the routerboard and that will consistently fail on other routers/firmwares. That way it can be included in the auto detection mechanism of the plugin.
Eg.
To check if a router has a broadcom chipset the plugin will try to run: 'type wl' on the router. type will return an error code if the wl command is not available. The plugin uses this info (and more checks like that) to identify the router. Since routeros offers no real linux-like shell the 'type'' test will not work.

I agree, refreshing the arp table on every poll is a bit drastic.

I've found a way to query the wireless interface using SSH. By executing the command

Code: Select all

interface wireless registration-table print

you get the following table:


This table is instantly updated when the connection of a device drops or reconnects. If it's possible to incorporate this command instead of the arp table, that would be awesome and indeed the best solution !

I've also found a command to detect the routerboard, which will consistently identify it as a routerboard using the command

Code: Select all

system routerboard print

The output of this command is as follows:


Please let me know if you need any more debug output.

Kind regards.

[Prutsium]

@Prutsium

The plugin does not use DHCP (arp, brctl etc have nothing to do with DHCP) .Is the plugin using arp or brctl on ANY of your routers (should be visible in the log)? That would explain why it detects devices connected tot other acces points. Arp and brctl info will propagate throughout the network, since switches and routers have to know where network clients are located (by mac address).

DHCP assigns IP addresses to clients... something the plugin doesn't care about.

But is always nice to find out something is still working when you didn't expect it to

Seems the new way was not very stable as i see various dropouts that my phone went offline (sleep mode).

Did anyone try this: https://github.com/gasperphoenix/FritzB ... eDetection
Not sure if it works but would be nice addition to iDetect
(Will start trying it to see if it even works)

[EscApe]

@Freekers

Great finds!
The latest version (just uploaded) uses the wireless command you supplied.

<routerip>=routeros will force the routeros wireless query
<routerip>=routeros-arp will force arp in the routeros syntax

For now it will remain a custom setting. Using the auto detection script on the routerboard seems like a bad idea in the current approach, because the linux commands will just generate lots of errors and i don't want to break up the script into even more parts (with ssh session overhead for every one).

[EscApe]

@Prutsium

Yeah, thats the downside of the brctl method. I can imagine it will be more susceptible to (sleepmode) dropouts in a multi router environment because the sleeping device will produce less traffic in the network. A longer 'grace period' in the plugin settings might help.

The detection tool you are referring to uses the information from a http source/webpage on the fritzbox. It will probably work better on the fritzbox because the information is very likely based on the same tools my plugin uses over ssh. I have no short term plans to include other methods than ssh in my plugin. It would be nice to include http (also for status pages on mesh setups) or even ping as a fallback for some specific devices.

I did see some info on the web (google) on how to enable/install ssh on a fritzbox but the info looked outdated and might only be usable on much older routers.

[Freekers]

@Freekers

Great finds!
The latest version (just uploaded) uses the wireless command you supplied.

<routerip>=routeros will force the routeros wireless query
<routerip>=routeros-arp will force arp in the routeros syntax

For now it will remain a custom setting. Using the auto detection script on the routerboard seems like a bad idea in the current approach, because the linux commands will just generate lots of errors and i don't want to break up the script into even more parts (with ssh session overhead for every one).

I've just tested the new version; it works perfectly! Thank you very very much for the great support & quick fixes. Awesome!

[Freekers]

Have you considered adding your plugin to the Plugin Manager for easy installation & updating ?
viewtopic.php?t=22339

Kind regards.

[EscApe]

Hi Freekers,

Have considered it (briefly) but since the installation can not be 100% automated I didn't add it. The updating part is a good point. I might reconsider Figuring out how to get it added is not a priority for me right now though.

[Freekers]

Would it be possible to make the ssh key location an user configurable option? I'm asking this because the ssh key location on Synology devices is different from the normal location. I have to manually edit the python file to reflect the correct path. This way, updating becomes a bit harder, because I cannot do a git pull since it will overwrite my changes

Hi Freekers,

Have considered it (briefly) but since the installation can not be 100% automated I didn't add it. The updating part is a good point. I might reconsider Figuring out how to get it added is not a priority for me right now though.

Adding the plugin to the plugin manager is quite easy You just submit a pull request to the plugin manager's repository with your plugin name & github URL and that's it

[EscApe]

New in version 0.7.7

If you need to specify a custom key file name for ssh authentication, you can add file name (full path) after the router username in the username field, separated by # (e.g. admin#/home/pi/.ssh/id_rsa)

@Freekers,
I had another look at the plugin manager. Since I am not planning to use the plugin manager myself and do not want to publish something untested I will not add my plugin to the plugin manager myself. Feel free to add it though.

I'm running out of (sensible) options to add custom configuration variables and the plugin needs some cleaning up after adding all these extra's

[mikeoo]

Using the plug-in now for a while and 95% it is working fine and live it.

Using it with Unify Edgerouter X and ARP option because other command are not working with the EdgeRouter X as fas a i could find and know.

Sometimes when a phone leave the house iDetect keeps thinking it is still at home.

Today a phone come back at 06:30 am (dog walk) and iDetects registered that phone is back. Phone leaves again the house at 09:30 am and came back a 13:50 pm. Last Status in Domoticz is 06:30 am and did not change anymore while phone was gone for a few hourse. Next day all is working fine and working fine for days and the i could happen again.

As a test I run a other script what is also using ARP and when the above problem happens i check the status for this phone that uses the other script and that one is working fine and the phone status is then off while iDetect is On.

So 2 scripts both using ARP are getting difference result sometimes.

Any tips how the find/tweak this because i am lost. Mabe other users use an Edgerouter also and have other experiences and tips.

[EscApe]

@mikeoo,

I noticed exactly the same reliability issue you described when using the ARP method on my Asus router (as mentioned in the instructions). But that should not be any different for other scripts... unless the other script uses another method than running the ARP command on the router.

Have you looked at that script and can you share the method it is using (or the entire script)?
Is the Edgerouter the only router you are polling?

[mikeoo]

@mikeoo,

I noticed exactly the same reliability issue you described when using the ARP method on my Asus router (as mentioned in the instructions). But that should not be any different for other scripts... unless the other script uses another method than running the ARP command on the router.

Have you looked at that script and can you share the method it is using (or the entire script)?
Is the Edgerouter the only router you are polling?

I just find out i could SSH to the Unfi AP point. I was under the imprecision the this was not possible because i use the Unify Controller software.
So the good news is they iDetect now scan all 3 AP and log is telling:

Code: Select all

 2018-11-14 16:18:26.577 Status: (iDetect) Using chipset specific wlanconfig command on router 192.168.0.201 for interfaces ath0 & ath1 & ath3 & ath4 & eth0 (=wlanconfig ath0 ath1 ath3 ath4 eth0)
2018-11-14 16:18:27.222 Status: (iDetect) Using chipset specific wlanconfig command on router 192.168.0.202 for interfaces ath0 & ath1 & ath3 & ath4 & eth0 (=wlanconfig ath0 ath1 ath3 ath4 eth0)
2018-11-14 16:18:27.889 Status: (iDetect) Using chipset specific wlanconfig command on router 192.168.0.203 for interfaces ath0 & ath1 & ath3 & ath4 & eth0 

So it finally use the chipset and not ARP anymore if i get it right

So hopefully all will work much better now.

Next step is to find out how to create SSH key authentication.
I use a Raspberry Pi for Domoticz and 3 Unify AP but need to find out how and were the place the ssh key files.
The Unify Controller have also an option for SSH keys but don't know if this is the same way. Not my strongest part of networking
https://help.ubnt.com/hc/en-us/articles ... Fi-Devices

[EscApe]

So it finally use the chipset and not ARP anymore if i get it right

Thats’s great! It should be much more reliable and quicker to respond.

Since I don’t own any unifi ap’s I can offer no help on the ssh key configuration. Maybe someone else on the (unifi) forum?

[mikeoo]

So it finally use the chipset and not ARP anymore if i get it right

Thats’s great! It should be much more reliable and quicker to respond.

Since I don’t own any unifi ap’s I can offer no help on the ssh key configuration. Maybe someone else on the (unifi) forum?

Will try tomorrow and when i have succes i will post it for other if it is easy to do
But for your info Unify AP lite and AP long range working with only filling the ip-adresses for all AP in the field for router. Handy for others with same setup.


Tnx for the help and plugin

[mikeoo]

Finally solved the SSH key problem and because it took some time to find out everything i create an how to for it.

How to setup iDetect SHH from Raspberry and a Unifi AP Lite and Unify AP LR.

1. Connect to Raspberry with Putty.
2. Command: sudo su (log in as root)
3. Follow http://www.linuxproblem.org/art_9.html to generate the SHH key.
4. Give command: cat ~/.ssh/id_rsa.pub
5. Select everything with your mouse while holding the mouse button and drag over the text. Release left mouse and click right mouse once.
6. Log on to Unifi Controller and go to Settings --> Site --> Device Authentication
7. Enable SSH authentication and give it a username and password if it is not already filled in.
8. Click on Add new SSH Key
9. Give it a name like: Unify AP of what your like.
10 Now past the text from point 5 in the field key and check that it starts with ssh-rsa
11. Click on Add
12. Click on Apply Changes.
13. Done

Now setup iDetect in Domoticz for SHH. use the same username as you use in step 7.
If you have more then one Unify AP the SSH key will be copied to all AP automatically when you do it with Unify Controller

If you don't use Unify Controller.
1. Connect with Putty or WinSCP to your Unify AP.
2. Browse to /var/etc/dropbear
3. Put the SHH key (step 4) in authorized_keys. When file is not there create the file. CHMOD 0600
4. Repeat this steps for every Unify AP.

When it is not working.
1. Connect to Raspberry with Putty.
2. Command: sudo su (log in as root)
3. give command: ssh username@ip-adres-AP and type: yes
4. Repeat this for every Unify AP.

These steps you need to do only once. With this steps you trust the ssh connection.