Yet another presence detection method

[Wob76]

Hi,

Thanks for your work, I have it in and running.

Just wondering how everyone is handling the Presence switches in Domoticz?

I wanted to try and simplify adding extra devices later for monitoring in the future, so I created a Presence script that toggles a "Occupied" switch if any of devices ending in "Phone" are on.

This occupied switch is then used in my heating\cooling scripts to enable\disable overnight monitoring, along with a selector switch with Off\On\Auto. So I can override the presence sensing.

I have another switch called "Guests" which is toggled by devices that end in "Phone (Guest)". I plan to add a few people who regularly stay in our guest room, this will then control the heating zone for that room.

So my new Presence script runs once a min and seems to be working as expected, but I have since had a thought if I should just be using Sub\Slave switches.

Regards,
Wob

[Toyman]

Thanks A LOT. The ONLY presence script that actually works with iPhones

[DAVIZINHO]

edit, no problem. works great!!!

[freakshock]

This is a great method! Works great for my Android and iPhone devices.
I would also like to add some wired devices via this method.
Not as a presence detection function of course, but to see if these devices are online.

Is there any way to modify the code to also look for wired MAC adresses?
My skills are lacking here

[WyldeBill]

Hi there, great script and I have got it working on the 2.5Ghz interface of my ASUS AC87U router. However the AC87U does not report the 5Ghz interface as eth2 and therefore the wl command fails. Initial investigation indicates that this is because the 5Ghz interface is not provided via the Broadcom chip (which supports the wl command).

Was wondering if anyone had managed to get this working on the ASUS AC87 router, it appears to need to use the qcsapi client.

Thanks in advance

WB

[WyldeBill]

I have now had time to investigate detecting devices on the 5Ghz interface for the RT-AC87U router. I have an amended section of script to replace the 5Ghz section in the original script. This may not be the most elegant of scripts as Shell scripts are not my forte, however it works - please feel free to improve.

Hope this helps someone else

ATB

WB

Code: Select all

# look in 5GHz (wifi0) network for device

node_idx=0
interface=wifi0
nodemax=20
loopend=QCS
inc=1

#loops through upto 20 devices on the interface 
until [ $node_idx -gt $nodemax ] 

do
   
    #get_associated_device_mac_addr returns a mac adress for each node of the interface 
    for x in `/usr/sbin/qcsapi_sockrpc get_associated_device_mac_addr $interface $node_idx | awk '{print $1}'`; do

	    if [ $x = $mac ]; then
        	
		device_present=On

	    else
		
		#when the end of the wifi devices is reached QCS is returned so stop looping
	        if [ $x = $loopend ]; then

		   node_idx=$nodemax

		fi
	    
	    fi
	done

    #increment the node id
    node_idx=`expr $node_idx + $inc`

    done

[rrozema]

I've been experimenting and found this detection method to work almost as I want it. I have made a few adjustments and need some help making some more.

Like many others I've got multiple wifi access points and thus detecting a phone on either the 2.4 or 5GHz interfaces doesn't work well, I'd have to run scripts on all 3 access points. My network however is the same over all access points (i.e. my devices are all in the same ip-range, getting an address from the same dhcp server while roaming the access points). To fix this issue I've replaced running wl against the wifi interfaces by looking up the mac address in the arp cache on bridge br0 in my main router. This way the mac address will be found when it is on either 2.4GHz, 5GHz or on the wired network (from another access point). In my RT-AC66U (merlin 380.68_4) arp entries time out after 2 minutes, which results in an acceptable delay for turning off my presence of 2 to 3 minutes. The switch will turn On at most a minute after my phone connects to wifi because of the 1 minute schedule, and Off within 2 to 3 minutes after my phone disconnects. I have no idea how this change affects "sleeping" iphone detection, but for me it works like a charm. I'll post my revised script below. Because arp returns the mac address somewhat differently formatted and I have little experience in scripting in these languages, you have to pass the MAC address formatted like AB:CD:EF:01:02, so using capitals and with : instead of - in between the hex pairs. My domoticz is accessible from my lan without user name and passw, so I don't have to specify this in the url's. If your domoticz is configured differently, adjust the script accordingly.

Code: Select all

#!/bin/sh

mac=$1
idx=$2
device_present=Off
domoticz_status=`curl -s "http://192.168.128.100:8080/json.htm?type=devices&rid=$idx" | grep Status | awk '{print $3}' | sed 's/[",]//g'`

# look in bridge (br0) network for device
for x in `arp -i br0 -na | awk '{print $4}'`; do
echo "x: $x, mac: $mac"
 if [ $x = $mac ]; then
        device_present=On
 fi
done

echo "Domoticz: $domoticz_status, router: $device_present"

# tell domoticz the new device status
if [ $domoticz_status != $device_present ]; then
 echo "New status $device_present"
 curl -s "http://192.168.128.100:8080/json.htm?type=command&param=switchlight&idx=$idx&switchcmd=$device_present" > /dev/null
 fi
fi

This is how far I've gotten and what I wanted to share with you. Now the part that I need some assistance on to make it work even better :

As I've got at least 6 devices I need to configure, plus the fact that the rate at which people change phones at my home is rather high, I would like to make the entire process a little bit more maintenance friendly by having to set up the script on the router only once and later on I only need to create more switches, remove switches or modify existing switches in Domoticz to change which devices are being monitored.

In order to do this I want to adjust the presence_detection script such that it first queries domoticz for all devices ( selects only those which name starts with 'Presence' and for each of these reads both the "idx" value plus the "Description" value for the devices' json returned by domoticz. "idx" of course holds the switches idx value, required to get and set the switches Status. "Description" I fill with the mac address of the device I need monitored. I did find a way to find and loop over the device names, but for retrieving the "idx" and "Description" values I realy need to parse the json structure returned by domoticz and I don't know how to do that using just shell commands. If anyone has a suggestion or maybe some example code on how to parse the both values from the json I'd be much obliged.

Code: Select all

#!/bin/sh

mac=$1
idx=$2
for device in `curl -s "http://192.168.128.100:8080/json.htm?type=devices" | grep '"Name".*"Presence' | awk -F'"' '{print $4}'`; do
   echo "Presence device: $device"
   # to do: read the idx and Description values, then query each switch and if needed set the switch accordingly.
done

And here's an example of a single device of which there are multiple returned by the "devices" query to domoticz. I need both the value for "idx" and "Description" for each of these:

Code: Select all

      {
         "AddjMulti" : 1.0,
         "AddjMulti2" : 1.0,
         "AddjValue" : 0.0,
         "AddjValue2" : 0.0,
         "BatteryLevel" : 255,
         "CustomImage" : 0,
         "Data" : "Off",
         "Description" : "DC:0B:34:82:AF:1B",
         "Favorite" : 0,
         "HardwareID" : 9,
         "HardwareName" : "Dummy",
         "HardwareType" : "Dummy (Does nothing, use for virtual switches only)",
         "HardwareTypeVal" : 15,
         "HaveDimmer" : true,
         "HaveGroupCmd" : true,
         "HaveTimeout" : false,
         "ID" : "000140A8",
         "Image" : "Light",
         "IsSubDevice" : false,
         "LastUpdate" : "2017-11-17 12:40:01",
         "Level" : 0,
         "LevelInt" : 0,
         "MaxDimLevel" : 100,
         "Name" : "Presence Richard",
         "Notifications" : "false",
         "PlanID" : "0",
         "PlanIDs" : [ 0 ],
         "Protected" : false,
         "ShowNotifications" : true,
         "SignalLevel" : "-",
         "Status" : "Off",
         "StrParam1" : "",
         "StrParam2" : "",
         "SubType" : "Switch",
         "SwitchType" : "On/Off",
         "SwitchTypeVal" : 0,
         "Timers" : "false",
         "Type" : "Light/Switch",
         "TypeImg" : "lightbulb",
         "Unit" : 1,
         "Used" : 1,
         "UsedByCamera" : false,
         "XOffset" : "0",
         "YOffset" : "0",
         "idx" : "88"
      }

EDIT: Although this script turns the swithces on and off, after some time of usage I don't think it is doing exactly what I want it to do: My phones sometimes do not go into the internet for periods longer than 2 minutes, resulting in their arp entries being cleared from the router's arp cache. Every time this happens, my presence switch is turned off, even though I'm still at home. i.e. this method is not reliable enough for me. I'll be looking for another method still.

[galadril]

I've been experimenting and found this detection method to work almost as I want it. I have made a few adjustments and need some help making some more.

Great work richard

And here's an example of a single device of which there are multiple returned by the "devices" query to domoticz. I need both the value for "idx" and "Description" for each of these:

There is jq for parsing json on the command line:
jq '.Body'

so for the values it would be:
jq '.result[0].idx'
jq '.result[0].Description'

Visit this for jq: https://stedolan.github.io/jq/
they even have an online tester:

https://jqplay.org/


hope that this assists you in your quest
see you on work!

[woody4165]

Sorry for my ignorance, this nice script requests two parameters, Mac address and idx
In the script I see also $3 and $4

What they refer to?

When I run it I get, as result of

Code: Select all

for x in `arp -i br0 -na | awk '{print $4}'`; do
echo "x: $x, mac: $mac"

I get

Code: Select all

x: entries, mac: xx:xx:xx:xx:xx:xx

What entries stand for?

Thanks

[rrozema]

Sorry for my ignorance, this nice script requests two parameters, Mac address and idx
In the script I see also $3 and $4

What they refer to?

Those $3 and $4 are not script parameters, those are parameters to the awk command. They represent the 3rd and 4th "word" from the input given to awk. What "arp -i br0 -na | awk '{print $4}'" does:
- execute arp -i br0 -na
- take the output and "pipe" it into the next command (using the "pipe" command or |-symbol)
- awk '{print $4}' then takes the 4th word from it's input and outputs that

Run the individual parts of the commands from your router's cli and you'll see what exactly is going on.

When I run it I get, as result of

Code: Select all

for x in `arp -i br0 -na | awk '{print $4}'`; do
echo "x: $x, mac: $mac"

I get

Code: Select all

x: entries, mac: xx:xx:xx:xx:xx:xx

What entries stand for?

Thanks

Apparently, in your case the word "entries" was the 4th word output by arp -i br0 -na. Can you run said command in your router and see what exactly the output is that makes it return this word? For me the output is multiple lines like these:

Code: Select all

? (192.168.128.20) at D0:66:78:79:68:52 [ether]  on br0
? (192.168.128.28) at 30:C5:B8:EE:6A:32 [ether]  on br0
? (192.168.128.22) at 4C:49:7D:AF:B7:C9 [ether]  on br0

I suspect you're getting some error message instead.

But the text output here isn't that important, this print statement is more like some debugging, it shows you what texts are found from the arp output. The next if-statement matches the word found with the mac address you've specified, and only if this matches the code is activated. All entries that don't match are simply ignored.

If it isn't an error message, it could be that you have a different firmware version in your router, resulting in different output from the arp command. My firmware version is ASUSWRT-Merlin RT-AC66U 380.68-4 Wed Oct 4 19:01:30 UTC 2017. What is yours? (It is shown when I log on using telnet/ssh)

[woody4165]

Thank for your reply.
When I run the arp command I get
arp: in 36 entries no match found

What can be my issue?

[rrozema]

Thank for your reply.
When I run the arp command I get
arp: in 36 entries no match found

What can be my issue?

Are you running the arp command in your router or in your raspberry? You should be running it in your router. Try running

Code: Select all

ifconfig

to list all available interfaces. In my router I have a bridge over all interfaces, called br0 (with a zero at the end). This one is particulary handy for this purpose as it includes both all wifi interfaces and the wired interfaces. i.e. wether the call came in via one of the wifi tranceivers or from another access point via the LAN connectors, all come through this bridge interface. The first column returned by ifconfig lists the interface names. This is the name that you can use with the -i parameter for arp.

[woody4165]

What do you mean by running arp in my router?
I have no access (telnet or ssh) to my Fritz router.

With ifconfig I get

Code: Select all

bond0     Link encap:Ethernet  HWaddr xx:xx:xx:xx:xx:xx 
          UP BROADCAST MASTER MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

eth0      Link encap:Ethernet  HWaddr xx:xx:xx:xx:xx:xx   
          inet addr:192.168.xxx.xxx  Bcast:192.168.xxx.xxx  Mask:255.255.xxx.xxx
          inet6 addr: xx:xx:xx:xx:xx:xx /64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:932237 errors:0 dropped:259086 overruns:0 frame:0
          TX packets:954891 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:81454906 (77.6 MiB)  TX bytes:1121825048 (1.0 GiB)
          Interrupt:51 

lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:65536  Metric:1
          RX packets:223401 errors:0 dropped:0 overruns:0 frame:0
          TX packets:223401 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:27416526 (26.1 MiB)  TX bytes:27416526 (26.1 MiB)

wlan0     Link encap:Ethernet  HWaddr xx:xx:xx:xx:xx:xx   
          UP BROADCAST MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

[rrozema]

What do you mean by running arp in my router?
I have no access (telnet or ssh) to my Fritz router.

The script needs to be run on your router, via telnet or ssh. That's the idea of the methods presented in this thread: your router is in the path between your phones and the internet, so it can be used to monitor your phone's activities on the network and alert domoticz when a state changes. Various methods to monitor the presence of the phone have been presented, but they all only work correctly when run via telnet or ssh from your router. If you don't have access to your router's cli via telnet or ssh this method is of no use to you.

[commentator]

when adding this script to cron it doesn't work.
After changing the domoticz_status line to

Code: Select all

domoticz_status=$(curl -s "http://dom.otic.zser.verip/json.htm?type=devices&rid="$idx"" | grep '"Data" :' | awk '{ print $3 }' | sed 's/[!@#\$%",^&*()]//g')

and off course replacing the serverip it is working as it should be

[rrozema]

I've created an icon for my presence switches and thought I'd share them with you all.

Presence48_On.png (2.1 KiB) Viewed 4096 times

for 'On' and

Presence48_Off.png (3.25 KiB) Viewed 4096 times

for 'Off'. You can insert them into your domoticz by uploading

Presence.zip

(8.28 KiB) Downloaded 132 times

in your menu 'Settings' -> 'More options' -> 'Custom Icons'.

[Surroot]

I've created an icon for my presence switches and thought I'd share them with you all. Presence48_On.png for 'On' and Presence48_Off.png for 'Off'. You can insert them into your domoticz by uploading Presence.zip in your menu 'Settings' -> 'More options' -> 'Custom Icons'.

Thank you very much!
I already uploaded them to my setup, I really like the icons

[domoticzcom1234]

All, great script.

Please be aware that using the command "arp -a" returns all MAC addresses in low case from the ARP table. The Web GUI list the MAC's in uppercase.

So when using the script matching both MAC's will fail.

Solution: change all MAC addresses you monitor to lower cases and insert these in the init-start script.

[heggink]

Hey @surroot, when you flashed with Merlin, were all the settings retained or did you have to reconfigure the router from scratch? I have so many reserved IPs, reconfiguring scares me shitless .

[Surroot]

Hey @surroot, when you flashed with Merlin, were all the settings retained or did you have to reconfigure the router from scratch? I have so many reserved IPs, reconfiguring scares me shitless .

Hey @heggink,

sorry I don't know it anymore, but I think that you have to reconfigure your router after flashing.
But it could also be that you can safe your router's config and upload it to Merlin afterwards, maybe Merlin supports that feature, you have to look it up.

Good luck!
Best regards,
Surroot