Pass2PHP

[Egregius]

The only true thing about response time is the logfile.

For example: at ,922 the signal of the pir comes in, at ,962 the switch command is sent. Difference between them is the response time = 42 msec.
2016-12-19 15:22:34.922 (ZWAVE) Light/Switch (pirkeuken)
2016-12-19 15:22:34.964 User: Admin initiated a switch command (61/wasbak/On)
2016-12-19 15:22:34.964 OpenZWave: Domoticz has send a Switch command! NodeID: 6 (0x06)

For other stuff like telegram, kodi, whatever there are way more variables that can cause a delay.

[Sneezydevil]

In my silly quest for more speed I found little difference between version 2.x (with Opcache) and 3.0 (with Opcache). Without opchache version 3 is even a microfaction slower.

I'm looking for a faster way to do the base64 encoding, thats the one thing holding me back at the moment.

Out of the 20msec it takes to trigger a switch, 7msec are just for loading the base64 file.

On the other hand you can't really notice 7msec can you .

[Egregius]

Wich base64 and json lua files are you using?
I tried to remove all unnessescary stuff but as I don't know lua that's not so easy.
My versions are at https://github.com/Egregius/LUA-Pass2PH ... er/scripts

It is true that most of the time get lost doing that, but I can't see another way of pushing the lua tables to an external script.
Maybe of we could skip the json encode and use native lua things to make a string like array('switch1'=>'On','switch2'=>'On') etc
Or even better a multilevel array:
array('Switch1'=>array('Status'=>'On','idx'=>123,'time'=>1234567890),'Switch2'=>array('Status'=>'On','idx'=>124,'time'=>1234568890).
Then we would only need one base64 encode/decode to handle everything.

7msec in your case is almost half of the time...

[Egregius]

pass2php 3.1 might come faster than I guessed...

2016-12-19 18:03:12.106 (ZWAVE) Light/Switch (pirkeuken)
2016-12-19 18:03:12.142 User: Admin initiated a switch command (61/wasbak/On)
2016-12-19 18:03:12.142 OpenZWave: Domoticz has send a Switch command! NodeID: 6 (0x06)

Found a possible way to avoid the json encode. Need some hours to see if everything still works...

[Sneezydevil]

I am using the same base64 file and it is hard to tell if the json file is the same without the comments. I was at 40-45 msec with those, I then switched to cjson which removed 20msec.

[Egregius]

I've read about cjson as being very fast but didn't search how to get it on my syno.

[Nautilus]

For example: at ,922 the signal of the pir comes in, at ,962 the switch command is sent. Difference between them is the response time = 42 msec.
2016-12-19 15:22:34.922 (ZWAVE) Light/Switch (pirkeuken)
2016-12-19 15:22:34.964 User: Admin initiated a switch command (61/wasbak/On)
2016-12-19 15:22:34.964 OpenZWave: Domoticz has send a Switch command! NodeID: 6 (0x06)

As a comparison (using z-wave PIR to trigger MiLight spots with MiLight Wifi bridge v5):

Code: Select all

2016-12-19 23:37:41.592 (AotecZstick) Light/Switch (PIR Musiikkihuone)
-> 275ms
2016-12-19 23:37:41.867 EventSystem: Script event triggered: Door back
-> 34ms
2016-12-19 23:37:41.901 User: Admin initiated a switch command (129/Musiikkihuoneen spottivalot/On)
-> 314ms
2016-12-19 23:37:42.215 (MiLight2) Lighting Limitless/Applamp (Musiikkihuoneen spottivalot)

The delay between last two steps is something I hope RFLink will shorten, it is now there as "given", I assume. Also the time between LUA registering the event change and then executing the switch command is quite fast, 34ms (this is something that would take much longer if passing the "On" command via commandArray). But what I do not fully understand is the 275ms between PIR "On" and then event system firing. In your 42ms there is first the event system firing which would have to be considerably faster than this. I wonder why...? Maybe because my scripts are in the database and not in the scripts folder?

Also strange, in Domoticz log the first line is actually showing up as third so they are not in chronological order.

Edit: I moved this script to the scripts folder and result now is:

Code: Select all

2016-12-20 00:06:28.230 (AotecZstick) Light/Switch (PIR Musiikkihuone)
-> 55ms
2016-12-20 00:06:28.285 EventSystem: Script event triggered: /home/pi/domoticz/scripts/lua/script_device_PIRMusiikkihuone.lua
-> 38ms
2016-12-20 00:06:28.323 User: Admin initiated a switch command (129/Musiikkihuoneen spottivalot/On)
-> 395ms
2016-12-20 00:06:28.718 (MiLight2) Lighting Limitless/Applamp (Musiikkihuoneen spottivalot)

Quite a big difference, 275ms vs. 55ms. Still doesn't explain what takes the 55ms as it is a very simple script. Can it be this slow to do the logic operations with LUA?

[Egregius]

File based scripts are faster than those in database.
Remember that EACH script is executed.
For the fun: place 1000 files names script_device_A0001.lua to script_device_A1000.lua and see how much slower it will go...
That's way I'm only using one.

Find another thing to make it even faster, but don't know yet if it's wurth all the extra effort.
2016-12-19 23:18:00.929 (ZWAVE) Light/Switch (pirkeuken)
2016-12-19 23:18:00.951 User: Admin initiated a switch command (61/wasbak/On)
2016-12-19 23:18:00.952 OpenZWave: Domoticz has send a Switch command! NodeID: 6 (0x06)

Storing all statusses, idx's and updates in a cache with a time script. Then with a device script only sent the updated device without base64 or json.
But, this would need all scripts to be changed...
First block is old system with $s array, second retrieving from cache

Code: Select all

<?php
if($s['pirkeuken']=="On")
{
    if($s['keuken']=='Off'&&$s['wasbak']=='Off'&&$s['werkblad']=='Off'&&$s['kookplaat']=='Off'&&$s['zon']<500)sw($i['wasbak'],'On','wasbak');
}

if(cget('spirkeuken')=="On")
{
    if(cget('skeuken')=='Off'&&cget('swasbak')=='Off'&&cget('swerkblad')=='Off'&&cget('skookplaat')=='Off'&&cget('szon')<500)sw(cget('iwasbak'),'On','wasbak');
}

[Nautilus]

File based scripts are faster than those in database.
Remember that EACH script is executed.
That's way I'm only using one.

Yes, that was something I learned way back, but never (until now I mean) went through the trouble to find out how much. And in this case, the device script in the scripts folder was the only one there so it should've been the first one to execute, still took 55ms

Edit: As Blockly is supposed to be the fastest native alternative, tried that as well, result:

Code: Select all

2016-12-20 00:39:18.812 (AotecZstick) Light/Switch (PIR Musiikkihuone)
2016-12-20 00:39:18.829 EventSystem: Event triggered: TeslBlock_1

Response to device change clearly faster, 17ms, but then the actual switch command seemed to take ages (maybe should have disabled the parallel LUA scripts first). Anyhow, seems that pass2php is something to dive into at least for the stuff that one would like to response fast Never really realized that most of the time is wasted waiting for the scripts to start running (at least after the first delay of waiting the commandArray to be returned is tackled with json...)

[Egregius]

This looks also promising...

Code: Select all

c = ''
s = ''
i = ''
t = ''
for k,v in pairs(devicechanged) do c = c..k.."|"..v.."#" end
for k,v in pairs(otherdevices) do s = s..k.."|"..v.."#" end
for k,v in pairs(otherdevices_idx) do i = i..k.."|"..v.."#" end
for k,v in pairs(otherdevices_lastupdate) do t = t..k.."|"..v.."#" end
os.execute('curl -s --data "c='..c..'&s='..s..'&i='..i..'&t='..t..'" http://127.0.0.1/secure/pass2php.php &')
commandArray={}
return commandArray

Code: Select all

$c=ex($_REQUEST['c']);
$s=ex($_REQUEST['s']);
$i=ex($_REQUEST['i']);
$t=ex($_REQUEST['t']);
function ex($x){
    $return=array();
    $pieces=explode('#',$x);
    foreach($pieces as $piece){
        $keyval=explode('|',$piece);
        if(count($keyval)>1)$return[$keyval[0]]=$keyval[1];
        else $return[$keyval[0]]='';
    }
    return $return;
}
 

So, without base64 and json encode/decode. Response time 26msec

[Sneezydevil]

Very nice, I think soon there is no more room for improvements, haha.

For now I am sticking with json and base64.

I modified the cjson lua module to do both the json and base64 encoding, and I don't think I can get this much faster.

Some results:

The old way with base64.lua and json.lua = 38msec

Code: Select all

2016-12-19 22:50:49.435 LUA: Encoding started
2016-12-19 22:50:49.473 LUA: Encoding finished - jf-json

With cjson and base64.lua = 16msec

Code: Select all

2016-12-19 22:50:50.466 LUA: Encoding started
2016-12-19 22:50:50.482 LUA: Encoding finished - cjson

With the modified module = 2msec

Code: Select all

2016-12-19 22:56:51.484 LUA: Encoding started
2016-12-19 22:56:51.486 LUA: Encoding finished - cb64json

This means pass2php gets called after only 2msec.

I will keep this running for a while to make sure its stable.

[Egregius]

Wow! Can you post how you did that? Is that possible on Syno?
I'm at 25msec (total response time) now.
Maybe also good to mention how many devices? I have 109 used devices, the more the more time logically.

[Egregius]

Now,... also added a print statement for the 'encoding'
Not much to improve there.

Code: Select all

2016-12-20 14:36:16.838  LUA: Encode Start
2016-12-20 14:36:16.840  LUA: encode ready
2016-12-20 14:36:16.847  LUA: curl ready

[Sneezydevil]

But thats with your new way right? It was loading the files that was causing the delay for me, and you no longer do that anyway. Don't think it's worth the hassle for you.

I have about 70 devices in use, but that doesn't seem to have alot of impact.

If I encode just the changed device it takes 2msec.
If I encode all my devices it takes 2msec.
If I encode everything twice it takes 3msec

Don't know if it will work on a syno, but I guess it should be possible.

I attached the module I currently use, I extract it and place it in:

Code: Select all

/usr/local/lib/lua/5.2/

The top of the lua script looks like this:

Code: Select all

local cb64json = require "cb64json"
changed=cb64json.b64encode(cb64json.encode(devicechanged))
devices=cb64json.b64encode(cb64json.encode(otherdevices))
idx=cb64json.b64encode(cb64json.encode(otherdevices_idx))
lastupdate=cb64json.b64encode(cb64json.encode(otherdevices_lastupdate))

[Egregius]

Like I tought, doesn't work, other platform...

Code: Select all

2016-12-20 15:00:30.008  Error: EventSystem: in /usr/local/domoticz/var/scripts/lua/script_device_pass2php.lua: error loading module 'cb64json' from file '/usr/local/lib/lua/5.2/cb64json.so':
	/usr/local/lib/lua/5.2/cb64json.so: wrong ELF class: ELFCLASS32

Indeed, just a concatenate of all statusses etc:

Code: Select all

c = ''
s = ''
i = ''
t = ''
for k,v in pairs(devicechanged) do c = c..k.."|"..v.."#" end
for k,v in pairs(otherdevices) do s = s..k.."|"..v.."#" end
for k,v in pairs(otherdevices_idx) do i = i..k.."|"..v.."#" end
for k,v in pairs(otherdevices_lastupdate) do t = t..k.."|"..v.."#" end
os.execute('curl -s --data "c='..c..'&s='..s..'&i='..i..'&t='..t..'" http://127.0.0.1/secure/pass2php.php &')
commandArray={}
return commandArray

[Westcott]

Hi SneezyDevil,
WOW!
4ms to load your lib and do the 4 encodes on my Pi2.

I've now installed cjson as well.

[sincze]

Hi SneezyDevil,
WOW!
4ms to load your lib and do the 4 encodes on my Pi2.

I've now installed cjson as well.

Do we have a wiki for my ARM based system.

Sent from my SM-G925F using Tapatalk

[sincze]

Did the following now:

Extracted the so file from cb64json.tar.gz to

Code: Select all

/usr/local/lib/lua/5.2/

Modified the script_device_pass2php.lua to:

Code: Select all

local cb64json = require "cb64json"
changed=cb64json.b64encode(cb64json.encode(devicechanged))
devices=cb64json.b64encode(cb64json.encode(otherdevices))
idx=cb64json.b64encode(cb64json.encode(otherdevices_idx))
lastupdate=cb64json.b64encode(cb64json.encode(otherdevices_lastupdate))
c = ''
s = ''
i = ''
t = ''
for k,v in pairs(devicechanged) do c = c..k.."|"..v.."#" end
for k,v in pairs(otherdevices) do s = s..k.."|"..v.."#" end
for k,v in pairs(otherdevices_idx) do i = i..k.."|"..v.."#" end
for k,v in pairs(otherdevices_lastupdate) do t = t..k.."|"..v.."#" end
os.execute('curl -s --data "c='..c..'&s='..s..'&i='..i..'&t='..t..'" http://127.0.0.1/secure/pass2php.php &')
commandArray={}
return commandArray

And as a result of your guys work i get:

Code: Select all

2016-12-21 18:08:21.929 User: Admin initiated a switch command (193/Lamp (Keukentafel)/On)
2016-12-21 18:08:21.930 (RFXCom) Lighting 2 (Lamp (Keukentafel))
2016-12-21 18:08:23.027 (RFLink Gateway) Light/Switch (Lamp (Keukentafel) (RF))

Seems pretty okay to me right?

and:

Code: Select all

2016-12-21 18:13:38.180 User: Admin initiated a switch command (193/Lamp (Keukentafel)/Off)
2016-12-21 18:13:38.152 (RFLink Gateway) Temp + Humidity (TH7_Berging (RF))
2016-12-21 18:13:38.321 (RFXCom) Lighting 2 (Lamp (Keukentafel)) 

[Egregius]

You're only using my part of it.
Don't need to double it.
Now your sending c s t and i
If you want to use vb64json that should be changed devices idx and lastupdate as in the example of Sneezydevil.
Keep in mind that when you base64/json encode you also need to enable the decode again in the php file.

[sincze]

You're only using my part of it.
Don't need to double it.
Now your sending c s t and i
If you want to use vb64json that should be changed devices idx and lastupdate as in the example of Sneezydevil.
Keep in mind that when you base64/json encode you also need to enable the decode again in the php file.

A yes.. came to that conclusion when I arrived home and not everything was working
Reverted back to old plain old pass2php 3.0

Code: Select all

2016-12-21 19:30:13.040 => Pass2PHP de functie reversed: Motion_Keuken
2016-12-21 19:30:13.046 => SETLEVEL 40
2016-12-21 19:30:13.048 => Pass2PHP de functie reversed: Motion_Keuken
2016-12-21 19:30:13.055 User: Admin initiated a switch command (193/Lamp (Keukentafel)/Set Level)
2016-12-21 19:30:13.596 (MySensors LAN) Lux (Light Level)
2016-12-21 19:30:13.897 RFLink: 2 level: 15
2016-12-21 19:30:13.865 (RFXCom) Lighting 2 (Lamp (Keukentafel))
2016-12-21 19:30:15.015 User: Admin initiated a switch command (226/Motion Keuken/On)
2016-12-21 19:30:15.016 (Virtual Device) Lighting 2 (Motion Keuken)
2016-12-21 19:30:15.286 => Pass2PHP de functie reversed: Motion_Keuken
2016-12-21 19:30:15.288 => Pass2PHP de functie reversed: Motion_Keuken
2016-12-21 19:30:15.293 => Motion Keuken, Lamp aangezet 40%
2016-12-21 19:30:15.295 => SETLEVEL 40
2016-12-21 19:30:15.298 => SETLEVEL 40

And everything was backup and running as it should. Have to read the Sneezydevil post more carefully i think.