Python plugin: Presence detection from wireless router

[mikeoo]

Result /usr/sbin/wlanconfig eth0 list; echo $?
CODE: SELECT ALL

Schuur-BZ.v4.0.80# /usr/sbin/wlanconfig eth0 list; echo $?
Not supported
0

Thanks!
So we found the culprit. Running wlanconfig on eth0 returns an error message, but not an errorlevel (still returning 0). So the plugin assumes it is a usable interface, when in fact it isn’t. The presence detection will still work without any issues. You can ignore the error message.

I don’t know if this is specific to the unifi or a generic issue for all routers using the wlanconfig command. Since I don’t know every make and model router that uses the atheros chipset I can not simply exclude eth0 from the detection script. A specific tracker for the unifi is an option but I would still prefer auto detection over this approach. Even when it throws an ignorable error.

The auto detection could be enhanced to not only check the errorlevel but also the string returned. I will look into that,but since functionality is not affected in any way it is not a top priority.

I understand by what i don't get. Why is it giving the error suddenly. Before last night it was working fine and AP of other software was not updated.
I use Berryboot so i can boot different images if Domoticz crash. I have now an older Domoticz running from last week with running V1 of iDetect and with same AP's it is running without error.

Code: Select all

 2020-02-09 11:30:21.099 Status: (iDetect) Using chipset specific wlanconfig command on router 192.168.0.203 for interfaces ath0 & ath1 & ath2 & ath3 & eth0 (=wlanconfig ath0 ath1 ath2 ath3 eth0)
2020-02-09 11:30:21.255 Status: Python EventSystem: Initalizing event module.
2020-02-09 11:30:21.256 Status: EventSystem: Started
2020-02-09 11:30:21.256 Status: EventSystem: Queue thread started... 
 2020-02-09 11:32:31.274 Status: (iDetect) Using chipset specific wlanconfig command on router 192.168.0.201 for interfaces ath0 & ath1 & ath2 & ath3 & eth0 (=wlanconfig ath0 ath1 ath2 ath3 eth0)
2020-02-09 11:32:31.849 Status: (iDetect) Using chipset specific wlanconfig command on router 192.168.0.202 for interfaces ath0 & ath1 & ath2 & ath3 & eth0 (=wlanconfig ath0 ath1 ath2 ath3 eth0) 

So version V1 have no problems with eth0 not present.
I will try to make a clone in Berryboot from the older Domoticz and runs that one and upgrading iDetect from V1 to V2 to see if the error is coming back directly.

The problem is the de log of Domoticz is getting full with errors so you can't see any real error anymore generated with Domoticz.

[EscApe]

So version V1 have no problems with eth0 not present.

Actually, you probably had exactly the same “problem”, but logging was different in V1. I wouldn’t spend to much time on testing V1 again. If the logging is bothering you I could make a quick patch to not log these messages.

[mikeoo]

So version V1 have no problems with eth0 not present.

Actually, you probably had exactly the same “problem”, but logging was different in V1. I wouldn’t spend to much time on testing V1 again. If the logging is bothering you I could make a quick patch to not log these messages.

The would be nice if you could do that. Did the test and V1 upgrade to V2 and error is back. So would love the patch so i don't see the error and Domoticz log will be clean with real logs

[EscApe]

So version V1 have no problems with eth0 not present.

Actually, you probably had exactly the same “problem”, but logging was different in V1. I wouldn’t spend to much time on testing V1 again. If the logging is bothering you I could make a quick patch to not log these messages.

The would be nice if you could do that. Did the test and V1 upgrade to V2 and error is back. So would love the patch so i don't see the error and Domoticz log will be clean with real logs

Can you test something for me? I don't like releasing stuff untested but don't own anything that uses the wlanconfig command.
In the ../helpers/tracker_cli_helper.py file replace the interface auto-detection for wlanconfig (lines 28-32) with the part below. I think this should work around the problem that wlanconfig does not return an errorlevel even if it throws an error.
Copy past should work best, because there are a lot of different style parentheses in there that can be easily overlooked.

Code: Select all

interface_check['wlanconfig']="""
for iface in $(ifconfig | cut -d ' ' -f1| tr ':' '\n' | grep -E '^eth|^wlan|^ath|^wl');do
	testif=`wlanconfig $iface list 2>&1`
	errif=$?
	[ "$testif" != "Not supported" ] && [ $errif == 0 ] && printf "~$iface"
done
"""

[mikeoo]

Actually, you probably had exactly the same “problem”, but logging was different in V1. I wouldn’t spend to much time on testing V1 again. If the logging is bothering you I could make a quick patch to not log these messages.

The would be nice if you could do that. Did the test and V1 upgrade to V2 and error is back. So would love the patch so i don't see the error and Domoticz log will be clean with real logs

Can you test something for me? I don't like releasing stuff untested but don't own anything that uses the wlanconfig command.
In the ../helpers/tracker_cli_helper.py file replace the interface auto-detection for wlanconfig (lines 28-32) with the part below. I think this should work around the problem that wlanconfig does not return an errorlevel even if it throws an error.
Copy past should work best, because there are a lot of different style parentheses in there that can be easily overlooked.

Code: Select all

interface_check['wlanconfig']="""
for iface in $(ifconfig | cut -d ' ' -f1| tr ':' '\n' | grep -E '^eth|^wlan|^ath|^wl');do
	testif=`wlanconfig $iface list 2>&1`
	errif=$?
	[ "$testif" != "Not supported" ] && [ $errif == 0 ] && printf "~$iface"
done
"""

Of course i would test is. Glad you help
I is working but no logging at all now. Only when it starting you see this. When i Phone is gone you don't see it anymore.
But better nothing for now than a lot of errors.

I you need more testing let me know

Code: Select all

2020-02-09 15:02:26.688 Status: (iDetect) Started.
2020-02-09 15:02:26.932 Status: (iDetect) Entering work loop.
2020-02-09 15:02:26.934 Status: (iDetect) Initialized version 2.0, author 'ESCape'
2020-02-09 15:02:29.188 Status: (iDetect) 192.168.0.201 ====> SSH connection established
2020-02-09 15:02:29.310 Status: (iDetect) Tracker activated:192.168.0.201 port 22, user: migroen, type: default and options: {}
2020-02-09 15:02:29.564 Status: (iDetect) 192.168.0.202 ====> SSH connection established
2020-02-09 15:02:29.688 Status: (iDetect) Tracker activated:192.168.0.202 port 22, user: migroen, type: default and options: {}
2020-02-09 15:02:29.924 Status: (iDetect) 192.168.0.203 ====> SSH connection established
2020-02-09 15:02:30.045 Status: (iDetect) Tracker activated:192.168.0.203 port 22, user: migroen, type: default and options: {} 

[DJBenson]

Awesome plugin! I've been using the check_device_online.py method for a long time but never found it particularly reliable (cron attempting to start multiple instances and the active instance dying leaving the PID file in place meaning it never started again).

I have one question - is it possible to expose the "Anyone" device as a presence detection node to Homebridge? At the moment it's presented as a switch but it would be good if Homebridge (and by extension Siri) saw the device as a presence detection node so it could be included in routines?

Mmm on further inspection it would seem this is somewhat constrained by Domoticz as there is no "Presence" device, only motion sensor (not sure if that would work as a motion sensor is its own device class in Homekit).

Perhaps this is more a question for the eDomoticz dev - a way to map Domoticz device classes to Homekit device classes might be possible.

[EscApe]

@mikeoo

I is working but no logging at all now.

The modification changes nothing related to logging. It only prevents eth0 from being falsely detected as a wireless interface. Could it be that you did not enable debug logging this time?

The fix has been merged on GitHub, so it is now available after updating

[EscApe]

Awesome plugin! I've been using the check_device_online.py method for a long time but never found it particularly reliable (cron attempting to start multiple instances and the active instance dying leaving the PID file in place meaning it never started again).

I have one question - is it possible to expose the "Anyone" device as a presence detection node to Homebridge? At the moment it's presented as a switch but it would be good if Homebridge (and by extension Siri) saw the device as a presence detection node so it could be included in routines?

Mmm on further inspection it would seem this is somewhat constrained by Domoticz as there is no "Presence" device, only motion sensor (not sure if that would work as a motion sensor is its own device class in Homekit).

Perhaps this is more a question for the eDomoticz dev - a way to map Domoticz device classes to Homekit device classes might be possible.

Look like you answered your own question
I'm glad you like the plugin. I don't see any other (more relevant) types of devices in Domoticz either. Please let me know if you discover anything we both missed at this time.

[gjaa]

Is there someone who got this working with a FritzBox?
I can't ssh to the FritzBox

Gerard

[mikeoo]

@mikeoo

I is working but no logging at all now.

The modification changes nothing related to logging. It only prevents eth0 from being falsely detected as a wireless interface. Could it be that you did not enable debug logging this time?

The fix has been merged on GitHub, so it is now available after updating

Tnx again. Debug is off indeed. But normally i saw in the logging of Domoticz when a one leaves the house something like waiting for 1 phone absence. Don't know the exact one and when a phone is back in the house.

That part is not showing anymore but maybe that was only in V1.

[EscApe]

Is there someone who got this working with a FritzBox?
I can't ssh to the FritzBox

Gerard

Hi Gerard,

I have seen examples of querying the Fritzbox for connected clients using http. That method could be used if someone with access to a Fritzbox (for development and testing) would write a tracker for it. There is a http example in the trackers.

[gschmidt]

Hi,

My router is a pfSense based system.
When I login with SSH on this router, a menu is openened.
Selecting no. 8 is the Shell

SSH.JPG (61.31 KiB) Viewed 1382 times

Is there a work-around trick to get into the shell?

[EscApe]

Hi,

My router is a pfSense based system.
When I login with SSH on this router, a menu is openened.
Selecting no. 8 is the Shell

Is there a work-around trick to get into the shell?

I don’t know if there are other pfSense users here. You might have a better chance of getting an answer on a pfSense related forum.

[gschmidt]

OK...I had to create an additional user with admin rights on pfSense.
Then you can ssh straight into the shell

But the Domoticz log shows the following error:

Code: Select all

 2020-02-12 21:07:53.757 Error: (iDetect) 192.168.1.1 ====> SSH returned error:export: Command not found.
2020-02-12 21:07:53.757 wl: not found
2020-02-12 21:07:53.757 iwinfo: not found
2020-02-12 21:07:53.757 wlanconfig: not found
2020-02-12 21:07:53.757 qcsapi_sockrpc: not found
2020-02-12 21:07:53.757 ip: not found
2020-02-12 21:07:53.757 brctl: not found 

The router has no WLAN interface, I use 2 AP's instead
The only command that works is (just like the windows cmd command)

Code: Select all

arp -a

This returns a list of devices in this format:

Code: Select all

(192.168.1.72) at d4:63:c6:78:1d:2d on igb1 expires in 873 seconds [ethernet]

igb1 is the LAN interface of the mini-pc

I don't know exactly what iDetect is looking for, but can I create a custom tracker with syntax options?

[EscApe]

OK...I had to create an additional user with admin rights on pfSense.
Then you can ssh straight into the shell

But the Domoticz log shows the following error:

Code: Select all

 2020-02-12 21:07:53.757 Error: (iDetect) 192.168.1.1 ====> SSH returned error:export: Command not found.
2020-02-12 21:07:53.757 wl: not found
2020-02-12 21:07:53.757 iwinfo: not found
2020-02-12 21:07:53.757 wlanconfig: not found
2020-02-12 21:07:53.757 qcsapi_sockrpc: not found
2020-02-12 21:07:53.757 ip: not found
2020-02-12 21:07:53.757 brctl: not found 

The router has no WLAN interface, I use 2 AP's instead
The only command that works is (just like the windows cmd command)

Code: Select all

arp -a

This returns a list of devices in this format:

Code: Select all

(192.168.1.72) at d4:63:c6:78:1d:2d on igb1 expires in 873 seconds [ethernet]

igb1 is the LAN interface of the mini-pc

I don't know exactly what iDetect is looking for, but can I create a custom tracker with syntax options?

The plugins auto detection will indeed fail on ssh hosts without the usual Linux commands.
You could configure the tracker as

Code: Select all

192.168.1.1#ssh=arp -a

If you are satisfied with the results you can turn it into a custom tracker with that syntax

[gschmidt]

Code: Select all

192.168.1.1#ssh=arp -a

returns:

Code: Select all

2020-02-12 22:34:31.560 Error: (iDetect) 192.168.1.1 ====> SSH returned error:usage: arp [-n] [-i interface] hostname
2020-02-12 22:34:31.560 arp [-n] [-i interface] -a
2020-02-12 22:34:31.560 arp -d hostname [pub]
2020-02-12 22:34:31.560 arp -d [-i interface] -a
2020-02-12 22:34:31.560 arp -s hostname ether_addr [temp] [reject | blackhole] [pub [only]]
2020-02-12 22:34:31.560 arp -S hostname ether_addr [temp] [reject | blackhole] [pub [only]]
2020-02-12 22:34:31.560 arp -f filename
2020-02-12 22:34:31.560
2020-02-12 22:34:31.560 Error: (iDetect) 192.168.1.1 ====> SSH returned empty response. Transport active: True 

[gschmidt]

Working!...I forgot that the pfsense user cannot be admin, but it should be the new user I created with admin rights

[RolandMegens]

Is there someone who got this working with a FritzBox?
I can't ssh to the FritzBox

Gerard

I just discovered: https://github.com/hydex80/presence_det ... /README.md

Script is working perfect for my Fritz!Box 5490.
Fritz!Box -> Diagnostics -> Secuity -> look for 'Provider services (TR069)' and the script above does the trick!

[gschmidt]

If you are satisfied with the results you can turn it into a custom tracker with that syntax

With ARP pfSense reports after approx. 20 minutes that the device is not present in the network.
But pfSense has the possibility to install packages.
There is a package NMAP, which scans quickly whether a device is online or not.
It also runs with ssh commands:

Offline

Code: Select all

[2.4.4-RELEASE][[email protected]]/home/gsc: nmap 192.168.1.178
Starting Nmap 7.70 ( https://nmap.org ) at 2020-02-15 21:49 CET
Note: Host seems down. If it is really up, but blocking our ping probes, try -Pn
Nmap done: 1 IP address (0 hosts up) scanned in 0.13 seconds

Online:

Code: Select all

[2.4.4-RELEASE][[email protected]]/home/gsc: nmap 192.168.1.178
Starting Nmap 7.70 ( https://nmap.org ) at 2020-02-15 21:50 CET
Nmap scan report for 192.168.1.178
Host is up (0.0017s latency).
All 1000 scanned ports on 192.168.1.178 are closed

Nmap done: 1 IP address (1 host up) scanned in 2.05 seconds
[2.4.4-RELEASE][[email protected]]/home/gsc: 

Different arguments are possible. See link:

https://securitytrails.com/blog/top-15- ... mote-hosts

Would it be possible to create a custom tracker for Nmap?

[EscApe]

With ARP pfSense reports after approx. 20 minutes that the device is not present in the network.

What are your poll interval and grace period settings for the plugin? As a rule of thumb use 3-4 polls within the grace period. Having just 1 poll within the grace period dramatically increases the chance to miss a connected device on a poll connection delay/error and any (short) connection drop by a phone.
If your devices go offline long enough to disappear from the arp table (which is already unusual on my asus router) then maybe you can modify the arp timeout on pfSense(?)

But pfSense has the possibility to install packages.
There is a package NMAP, which scans quickly whether a device is online or not.

It might be even easier to use nmap from the domoticz host itself, but wouldn’t the already available ping approach work just as well if you already know the (fixed) ip addresses? I have been experimenting with some network discovery but it is a resource heavy and slow process for the entire network, which is not desirable if you want to poll frequently.