Page 1 of 7

Yet another presence detection method

Posted: Wednesday 18 January 2017 14:37
by Surroot
Edits:

28-01-2017: Added code for protected switches. Thanks user timop!


Hey there!

I know there are a few other presence detection methods in the forum, but I want to show just another approach:
I wanted to track if my girlfriend and I am at home or not, to enable or disable a camera motion sensor in Domoticz. So I needed a reliable way of presence detection.
I've tried both versions of presence detection from the Domoticz-Wiki (Ping- and SNMP-Method), but soon I realised that both had its own problems with detecting devices in the network. For example:
  • switching on and off the virtual switch although the device was still in the network
  • device not detected when its in sleep mode
  • snmp device status did not update for about 20mins (when I disconnected the phone from the network)
Therefore I took another approach which I found here and works just perfectly: Let my Asus router do the detection work instead of the raspberry pi.
My home router is an Asus RT-AC68U which I have flashed with the Asus Merlin custom firmware, but every router for example with OpenWrt, DD-WRT, etc. should work as long as you can SSH into it and read out the connected wifi devices. Note: My router has a Broadcom Wifi-chipset and the script provided in step h. should work on all Broadcom devices. This post is written for routers with Asus Merlin firmware. On OpenWrt, DD-WRT etc. some details have to be made differently!

So here we go:
  1. Create a Dummy Hardware and add a virtual switch for it in Domoticz. Please have a look at the Wiki for how to do this.
  2. Flash Asus Merlin firmware (it is really easy but please take care of the information provided at their website)
    1. visit the Asus Merlin website, download the latest firmware and extract it
    2. log into your router by going to its local address (for example https://192.168.1.1:8443), accept the certificate and enter your username and password
    3. click on the "Administration" panel on the left --> "Firmware Upgrade" --> "Choose File" --> select extracted file and hit Upload. It can take a few minutes for your router to upgrade and reboot. You have to manually power-cycle you router after the update has finished
  3. start the SSH daemon on your router by logging into it and then going to "Administration" --> "System" --> adapt the settings according to the image:
    Screen Shot 2017-01-18 at 13.36.12.jpg
    Screen Shot 2017-01-18 at 13.36.12.jpg (19.87 KiB) Viewed 14170 times
    You can also choose another port, it doesn't matter, you just have to remember it for the next steps.
  4. Also click the "Enable JFFS custom scripts and configs" badge
  5. Click "Reboot" at the top of the page and wait until the reboot finished
  6. SSH into your router
    • Windows: download putty
      • IP: your router ip
      • Port: port which you have entered above
      • Connection Type: SSH
      • Hit Open and enter your username and password which you also use to login into the router website
    • Mac or Linux-Systems: open a terminal and type:

      Code: Select all

      ssh -p 2222 router-username@router-ip-address
      Hit the Enter key on your keyboard. Type the router password (you wont see what you are writing, just type the password) and hit Enter again.
    • You have to type "yes" (without quotes) when there is the question if you want to add this device to the known-hosts file and hit the Enter key
  7. You should now see a blinking cursor. Type the following (Enter after each line):

    Code: Select all

    cd /jffs/scripts/
    nano init-start
    Paste the following into this file and replace MAC-ADDRESS by the mac address you want to detect and IDX with the virtual switch ID which we have created in step a. (e.g. cru a PRESENCE "*/1 * * * * /jffs/scripts/presence_detection A1:B2:3C:D4:5F:67 27"):

    Code: Select all

    #! /bin/sh
    cru a PRESENCE "*/1 * * * * /jffs/scripts/presence_detection MAC-ADDRESS IDX"
    Press CTRL+O then Enter to save the file. Then CTRL+X to exit nano.
  8. Now:

    Code: Select all

    nano presence_detection
    Paste this into the file and replace domoticzUser:domoticzPassword@domoticzIP:domoticzPort with the right settings (e.g. domoticz:[email protected]:8080)

    Code: Select all

    #!/bin/sh
    
    mac=$1
    idx=$2
    device_present=Off
    domoticz_status=`curl -s "http://domoticzUser:domoticzPassword@domoticzIP:domoticzPort/json.htm?\
    type=devices&rid=$idx" | grep Status | awk '{print $3}' | sed 's/[",]//g'`
    
    # look in 2.4GHz (eth1) network for device
    for x in `wl -i eth1 assoclist | awk '{print $2}'`; do
     if [ $x = $mac ]; then
            device_present=On
     fi
    done
    
    # look in 5GHz (eth2) network for device
    for x in `wl -i eth2 assoclist | awk '{print $2}'`; do
     if [ $x = $mac ]; then
            device_present=On
     fi
    done
    
    # tell domoticz the new device status
    if [ $domoticz_status != $device_present ]; then
     if [ $device_present = "On" ]; then
            curl -s "http://domoticzUser:domoticzPassword@domoticzIP:domoticzPort/json.htm?\
            type=command&param=switchlight&idx=$idx&switchcmd=On" > /dev/null
     else
            curl -s "http://domoticzUser:domoticzPassword@domoticzIP:domoticzPort/json.htm?\
            type=command&param=switchlight&idx=$idx&switchcmd=Off" > /dev/null
     fi
    fi
    Again press CTRL+O then Enter to save the file. Then CTRL+X to exit nano.

    If you have a protected switch (thanks user timop!) you have to replace the following three lines:

    Code: Select all

    replace
    domoticz_status=`curl -s "http://domoticzUser:domoticzPassword@domoticzIP:domoticzPort/json.htm?\
    type=devices&rid=$idx" | grep Status | awk '{print $3}' | sed 's/[",]//g'`
    with
    domoticz_status=`curl -s "http://domoticzUser:domoticzPassword@domoticzIP:domoticzPort/json.htm?\
    type=devices&rid=$idx&passcode=switchPassword" | grep Status | awk '{print $3}' | sed 's/[",]//g'`
    
    replace
    curl -s "http://domoticzUser:domoticzPassword@domoticzIP:domoticzPort/json.htm?\
    type=command&param=switchlight&idx=$idx&switchcmd=On" > /dev/null
    with
    curl -s "http://domoticzUser:domoticzPassword@domoticzIP:domoticzPort/json.htm?\
    type=command&param=switchlight&idx=$idx&switchcmd=On&passcode=switchPassword" > /dev/null
    
    replace
    curl -s "http://domoticzUser:domoticzPassword@domoticzIP:domoticzPort/json.htm?\
    type=command&param=switchlight&idx=$idx&switchcmd=Off" > /dev/null
    with
    curl -s "http://domoticzUser:domoticzPassword@domoticzIP:domoticzPort/json.htm?\
    type=command&param=switchlight&idx=$idx&switchcmd=Off&passcode=switchPassword" > /dev/null
    
    Where "switchPassword" stands you have to enter the password that you have set in Domoticz (Setup->Settings->Light/Switch Protection).

    Now we have to set the correct permissions. Enter:

    Code: Select all

    chmod 755 init-start
    chmod 755 presence_detection
    Then we test the script we have just created by typing (replace MAC-ADDRESS by the mac address you want to detect and IDX with the virtual switch ID):

    Code: Select all

    ./presence_detection MAC-ADDRESS IDX
    If there is no error after executing the script and I have not forgotten to describe a step and the device you want to detect is in your network, the virtual switch in Domoticz should change its state.
After these steps you have to reboot your router (you can do this again from the routers website). After the reboot the router calls the /jffs/scripts/init-start script which adds a cronjob which calls the presence_detection script every minute.

I hope someone can use it. Its the solution I like best at the moment as the switches are updated nearly instantly (the only delay comes from the cron timeout).

Best regards,
Surroot

Re: Yet another presence detection method

Posted: Wednesday 18 January 2017 19:47
by mehtadone
Hi,

Thank you for the work on this.

When I try run the script, I get:

admin@RT-AC66U:/jffs/scripts# ./presence_detection
./presence_detection: line 3: syntax error: unexpected "|"

or

./presence_detection XX:XX:XX:XX:XX:XX 19
./presence_detection: line 3: syntax error: unexpected "|"

Any ideas?

Re: Yet another presence detection method

Posted: Wednesday 18 January 2017 19:52
by Surroot
Hey!
Sorry, it didn't work because I forgot to add the arguments for the script...
I edited the post, should work now!

Re: Yet another presence detection method

Posted: Wednesday 18 January 2017 19:56
by mehtadone
Ahh I just edited my post. Its the same problem.

Re: Yet another presence detection method

Posted: Wednesday 18 January 2017 20:36
by Szwuntex
mehtadone wrote:Ahh I just edited my post. Its the same problem.
You have to put

Code: Select all

domoticz_status=`curl -s "http://domoticzUser:domoticzPassword@domoticzIP:domoticzPort/json.htm?type=devices&rid=$idx" | grep Status | awk '{print $3}' | sed 's/[",]//g'`
in one line without a "new row" as it is in the original code.

You have the same problem in a couple of other rows also.

Re: Yet another presence detection method

Posted: Wednesday 18 January 2017 20:39
by Szwuntex
I'm running this on my system and it is working great!

A big thank you to Surroot!

Re: Yet another presence detection method

Posted: Wednesday 18 January 2017 20:42
by Surroot
in one line without a "new row" as it is in the original code.

You have the same problem in a couple of other rows also.
Yes thats maybe the next problem, Merlin firmware is a bit strange in that way, never had it on another Linux.
It sometimes adds a new line when the line is too long and that gives errors.
If a line is too long maybe you need to enter:

Code: Select all

domoticz_status=`curl -s "http://domoticzUser:domoticzPassword@domoticzIP:domoticzPort/json.htm?type=devices&rid=$idx" \
 | grep Status | awk '{print $3}' | sed 's/[",]//g'`
Note the backslash at the end of the first line!
I'm running this on my system and it is working great!

A big thank you to Surroot!
Happy to hear that, you're welcome! :D

Re: Yet another presence detection method

Posted: Wednesday 18 January 2017 21:35
by mehtadone
Great. Thanks Surroot. Got it working in the end. Was all line ending issues.

Let see if this is more reliable. Fingers crossed and great work.

Re: Yet another presence detection method

Posted: Thursday 19 January 2017 15:27
by Calzor Suzay
It's not quite clear but can you run the line below for each mac address to idx you need watching?
So if you had three devices it would be...

Code: Select all

cru a PRESENCE "*/1 * * * * /jffs/scripts/presence_detection MAC-ADDRESS1 IDX1"
cru a PRESENCE "*/1 * * * * /jffs/scripts/presence_detection MAC-ADDRESS2 IDX2"
cru a PRESENCE "*/1 * * * * /jffs/scripts/presence_detection MAC-ADDRESS3 IDX3"

Re: Yet another presence detection method

Posted: Thursday 19 January 2017 17:24
by Surroot
Calzor Suzay wrote:It's not quite clear but can you run the line below for each mac address to idx you need watching?
Yes that's the way it works ;)
One cru command adds one cronjob, they run in parallel.

Re: Yet another presence detection method

Posted: Saturday 21 January 2017 2:15
by Calzor Suzay
Just to fix my code on multiples you need to specify a unique name for the cron job, what I had originally overwrote to one line.
So should be...

Code: Select all

cru a PRESENCE1 "*/1 * * * * /jffs/scripts/presence_detection MAC-ADDRESS1 IDX1"
cru a PRESENCE2 "*/1 * * * * /jffs/scripts/presence_detection MAC-ADDRESS2 IDX2"
cru a PRESENCE3 "*/1 * * * * /jffs/scripts/presence_detection MAC-ADDRESS3 IDX3"
Implemented and will be testing :)

Re: Yet another presence detection method

Posted: Saturday 21 January 2017 18:43
by Surroot
Calzor Suzay wrote:

Code: Select all

cru a PRESENCE1 "*/1 * * * * /jffs/scripts/presence_detection MAC-ADDRESS1 IDX1"
cru a PRESENCE2 "*/1 * * * * /jffs/scripts/presence_detection MAC-ADDRESS2 IDX2"
cru a PRESENCE3 "*/1 * * * * /jffs/scripts/presence_detection MAC-ADDRESS3 IDX3"
Yes thank you, I didn't see it, they have to have unique names :)
Let me know how it works for you!

Re: Yet another presence detection method

Posted: Saturday 21 January 2017 19:33
by Calzor Suzay
Well so far it picked up myself and the wife leaving the house and activated a switch that then monitors PIRs and door sensors for activation to alert me and set scenes etc.
The real test will be if it picks us up as we get home fast enough that the front door trigger doesn't go off :lol:
That will probably be dependent on how quick our phones pick up the WiFi but it does broadcast outside :geek:

I hope it works as prior to this I had a brillinat LUA script (from here) going off to iCloud to then retrieve our location to then report it back, it worked but a little long winded.

Re: Yet another presence detection method

Posted: Saturday 28 January 2017 9:08
by elfinko
Hi.
Tested many, but still havent found a good working precence detection script for my mixed android / iphone enviroment.
This looks to be the solution! but Im getting unknown operand script errors...

[: "ActTime": unknown operand

Can someone provide a copy/pasteable version of the script? Or take a screenshoot of a working version in the nano editor?

Re: Yet another presence detection method

Posted: Saturday 28 January 2017 18:07
by timop
Thanks this works great with RTN66U !
I got some unknown operand when my IDX was wrong when running command ./presence_detection MAC-ADDRESS1 IDX1
after fixing it works now.

Re: Yet another presence detection method

Posted: Saturday 28 January 2017 19:34
by Surroot
elfinko wrote:Can someone provide a copy/pasteable version of the script? Or take a screenshoot of a working version in the nano editor?
The code is the same as in the first post, but as I said in another post, nano on Merlin is a bit strange according to long lines.
Here you have a screenshot:
Script.jpg
Script.jpg (265.59 KiB) Viewed 13567 times
timop wrote:Thanks this works great with RTN66U !
Happy to hear its working great for you! :D

Best regards,
Surroot

Re: Yet another presence detection method

Posted: Saturday 28 January 2017 19:43
by timop
here is working code, just change address and port and at user account and password to address if there's any.
http://codepad.org/j6m4ayte

Re: Yet another presence detection method

Posted: Saturday 28 January 2017 20:01
by Calzor Suzay
So far it's working great for me and wife :)

Re: Yet another presence detection method

Posted: Saturday 28 January 2017 20:14
by timop
I added little feature (Surroot you can add it for example if wanted :) ), if use phones as protected switch like me. added passcode for address:

Code: Select all

#!/bin/sh
pcode=password  # add this in settings\Light/Switch Protection
mac=$1
idx=$2
device_present=Off
domoticz_status=`curl -s "http://IPAddress:port/json.htm?type=devices&rid=$idx&passcode=$pcode" | grep Status | awk '{print $3}' | sed 's/[",]//g'`

# look in 2.4GHz (eth1) network for device
for x in `wl -i eth1 assoclist | awk '{print $2}'`; do
 if [ $x = $mac ]; then
        device_present=On
 fi
done

# look in 5GHz (eth2) network for device
for x in `wl -i eth2 assoclist | awk '{print $2}'`; do
 if [ $x = $mac ]; then
        device_present=On
 fi
done

# tell domoticz the new device status
if [ $domoticz_status != $device_present ]; then
 if [ $device_present = "On" ]; then
        curl -s "http://IPAddress:port/json.htm?type=command&param=switchlight&idx=$idx&switchcmd=On&passcode=$pcode" > /dev/null
 else
        curl -s "http://IPAddress:port/json.htm?type=command&param=switchlight&idx=$idx&switchcmd=Off&passcode=$pcode" > /dev/null
 fi
fi

Re: Yet another presence detection method

Posted: Saturday 28 January 2017 21:07
by Surroot
timop wrote:I added little feature (Surroot you can add it for example if wanted :) ), if use phones as protected switch like me. added passcode for address
Thanks timop, thats great! Have already been looking for it :lol:
I add it to the original post!