Pass2PHP

[Egregius]

I don't understand it's even working if everything in the files is inside a function(){...}
Have a look at my examples at https://github.com/Egregius/LUA-Pass2PH ... e/pass2php

[sincze]

I don't understand it's even working if everything in the files is inside a function(){...}
Have a look at my examples at https://github.com/Egregius/LUA-Pass2PH ... e/pass2php

Aha that is our difference now in configuration. I see it
I stayed with "function" based devices (hence need global variables) and cjson where you moved to different method.
All my devices are also stored in subdirectory /pass2php need to be removed of the function() at the beginning of the file.

Let me see if we can make it work on my setup, Still needs some redecoration I think.

Code: Select all

$c=ex($_REQUEST['c']);$s=ex($_REQUEST['s']);
foreach($c as $device=>$status)
    if(@include '/volume1/web/secure/pass2php/'.$device.'.php'){apcu_store('t'.$device,time);$dev=$device;}
$split=microtime(true);
if(!isset($dev))die();
include '/volume1/web/secure/pass2php/__CRON.php'; 

I don't have ACPU so I would still need $t=ex($_REQUEST['t'])
You are using __CRON.php instead of Zon with values 1 and 0 now??

Suggestions?

[Egregius]

You can still use memcached if you keep the cset and cget functions.
Or you keep the $t and $i array, it's only couple of milliseconds difference.

Yes, before all my logic was in 'zon'. As I don't store the solar energy in domoticz anymore I renamed the file to __CRON so it'll be always on top of the dir instead of the bottom.

You don't need to follow each brain twitch I have, you can use the basics and work from there in your mindset.

[sincze]

You can still use memcached if you keep the cset and cget functions.
Or you keep the $t and $i array, it's only couple of milliseconds difference.

Yes, before all my logic was in 'zon'. As I don't store the solar energy in domoticz anymore I renamed the file to __CRON so it'll be always on top of the dir instead of the bottom.

You don't need to follow each brain twitch I have, you can use the basics and work from there in your mindset.

Hi i changed my pass2php to the following with your general script_device_lua calling it. That goes okay

Code: Select all

<?php 
$start=microtime(true);error_reporting(E_ALL);
ini_set("display_errors","on");date_default_timezone_set('Europe/Brussels');
define('time',$_SERVER['REQUEST_TIME']);$actions=0;
define('domoticz','http://192.168.2.10:8080/');
define('webserver','http://127.0.0.1/');

$c=ex($_REQUEST['c']);
$s=ex($_REQUEST['s']);

foreach($c as $device=>$status)
{//lg('For each loop starting: '.$device);
    
    $devicer=str_replace(" ","_",$device);

    if(@include '/var/www/php/pass3php/'.$device.'.php')
    {
        lg('Hi, I did find: '.$device);
        cset('t'.$device,time);
        $dev=$device;
    }
    
    elseif(@include '/var/www/php/pass3php/'.$devicer.'.php')
    {
        lg('Hi, I did find: '.$devicer);
        cset('t'.$devicer,time);
        $dev=$device;
    }
    
//lg('Hi, passed for each statement with: '.$device.'and'.$c);
}

This will call the appropriate device.php in the subdirectory.

Code: Select all

2016-12-29 00:05:32.430 User: Admin initiated a switch command (621/ZonweringAuto/On)
2016-12-29 00:05:32.659 --->> Yes I found you: what a result
2016-12-29 00:05:32.662 --->> ZonweringAuto ON
2016-12-29 00:05:32.665 --->> Hi, I did find: ZonweringAuto

mm had to do a little modification to have it work, Added $i otherwise no light would switch,,,

Code: Select all

$c=ex($_REQUEST['c']);
$s=ex($_REQUEST['s']);
$i=ex($_REQUEST['i']);

Also the following is not working anymore...
The php is called from within the main script.
Could it be that function donker() that is defined in the main pass2php is not accessible from here for some reason ??? ah it is late...

Code: Select all

<?php

    if (($s['Motion Detector Trap']=="On")&&donker()&&($s['Lamp LED Trap']!="On"))
    {  
        sw($i['Lamp LED Trap'],'On');
           if($debug=="true")
           {
              lg('Motion Detector Trap and it is dark so swith the light to on');
           }
    }
    elseif (($s['Motion Detector Trap']=="Off")&&($s['Lamp LED Trap']!="Off"))
    {
         sw($i['Lamp LED Trap'],'Off');
        if($debug=="true")
           {
              lg('Motion Detector Trap OFF');
           }
    } 

[Egregius]

Code: Select all

2016-12-29 00:05:32.430 User: Admin initiated a switch command (621/ZonweringAuto/On)
2016-12-29 00:05:32.659 --->> Yes I found you: what a result
2016-12-29 00:05:32.662 --->> ZonweringAuto ON
2016-12-29 00:05:32.665 --->> Hi, I did find: ZonweringAuto

You still have other scripts or blocklies active?

mm had to do a little modification to have it work, Added $i otherwise no light would switch,,,

Code: Select all

$c=ex($_REQUEST['c']);
$s=ex($_REQUEST['s']);
$i=ex($_REQUEST['i']);

I store all IDX's in cache. Someone here or at tweakers give me that idea. Idea behind it was that there's to much data sent upon each device changed. The lesser data nescessary the faster it'll go as you need to concatenate, sent and split less stuff. He even stores the otherdevices table in cache so he only needs to send the devicechanged table. I tried several times to do the same but every time ran into issues. As leaving out the otherdevices table only gived me about 3 msec speed difference I don't bother anymore to search for it.

Also the following is not working anymore...
The php is called from within the main script.
Could it be that function donker() that is defined in the main pass2php is not accessible from here for some reason ??? ah it is late...

If donker is a function in pass2php.php it should be available.

Another nice thing about the new @include for your case: you don't need the underscores anymore, you can have spaces in the filenames, just tested with a device "test scha ke laar" and a "pass2php/test scha ke laar.php" file and worked

[sincze]

Code: Select all

2016-12-29 00:05:32.430 User: Admin initiated a switch command (621/ZonweringAuto/On)
2016-12-29 00:05:32.659 --->> Yes I found you: what a result
2016-12-29 00:05:32.662 --->> ZonweringAuto ON
2016-12-29 00:05:32.665 --->> Hi, I did find: ZonweringAuto

You still have other scripts or blocklies active?

A yes some unconverted blocky and lua scripts are still stored and running. The converted lua -> pass2php ones have been disabled.

mm had to do a little modification to have it work, Added $i otherwise no light would switch,,,

Code: Select all

$c=ex($_REQUEST['c']);
$s=ex($_REQUEST['s']);
$i=ex($_REQUEST['i']);

I store all IDX's in cache. Someone here or at tweakers give me that idea. Idea behind it was that there's to much data sent upon each device changed. The lesser data nescessary the faster it'll go as you need to concatenate, sent and split less stuff. He even stores the otherdevices table in cache so he only needs to send the devicechanged table. I tried several times to do the same but every time ran into issues. As leaving out the otherdevices table only gived me about 3 msec speed difference I don't bother anymore to search for it.

Yes and that part i totally missed. I think this is done in __CRON.php

Code: Select all

if(apcu_fetch('cron604800')<time-604799){
    apcu_store('cron604800',time);
    $cron.=' + 604800';
    $domoticz=json_decode(file_get_contents(domoticz.'json.htm?type=devices&used=true'),true);
    if($domoticz){
        foreach($domoticz['result'] as $dom){
            $name=$dom['Name'];
            if(strtotime($dom['LastUpdate'])!=apcu_fetch('t'.$name))apcu_store('t'.$name,strtotime($dom['LastUpdate']));
            if($dom['idx']!=apcu_fetch('i'.$name))apcu_store('i'.$name,$dom['idx']);
        }
    }
}
 

So I need to rewrite that to memcached. I already did see you would store the time in a similar way in the for each loop.
And then i do not need the $i=ex($_REQUEST['i']); anymore? Correct? and do something like

Code: Select all

{sw(cget('islapen'),'Off') 

Also the following is not working anymore...
The php is called from within the main script.
Could it be that function donker() that is defined in the main pass2php is not accessible from here for some reason ??? ah it is late...

If donker is a function in pass2php.php it should be available.

Ok need to figure out what is missing by adding debug lines.

Another nice thing about the new @include for your case: you don't need the underscores anymore, you can have spaces in the filenames, just tested with a device "test scha ke laar" and a "pass2php/test scha ke laar.php" file and worked

Aha nice. make the code a little bit less complicated. I will start with that later today.

Code: Select all

foreach($c as $device=>$status)
    if(@include '/var/www/php/pass3php/'.$device.'.php')
    {
        lg('Hi, I did find: '.$device);
        cset('t'.$device,time);
        $dev=$device;
    }
 

btw what happens after a reboot and acpu is empty? Will it retrieve all the idx values of the active devices only after cron time has been reached? Sorry for all the questions but I try to understand what is going on Sometimes I see in your code $i then sw(cget('islapen').. just figuring things out here.

[Egregius]

Look at my __CRON.php
If the cache cron864000 is older than 864000 seconds it grabs all statusses, idxs and timestamps to pre-fill the cache.

Since this morning I only use the $c array
Responsetime now 19 msec.
As I'm not a developer I sometimes need to revert changes and try again later. Spend at least 16 hours on it last week without success. This morning in less than 2 hours converted everything so I could avoid the $s array.
Greatest benefit of storing everything in cache goes to my floorplan. Creating the page takes now 0,7 milliseconds against 29 msecs before!
Regarding pass2php the benefit is smaller: 19 msec against 23 msec before.
Will let it run for several days before posting it to Github, need to be sure it's reliable.

[Egregius]

I wanted to test again the real speed difference between this and the usual blockly and lua scripts.
On my system Blockly seems to be the fastest with 12 msec response time and lua gives me 14 msec.
But, that's when there's only one single script active. As soon as there are some more the time rises to resp. 20 and 37 msec for 5 active scripts. In the past I already tested that even empty script_device files create some kind of bottleneck in the event system. Not so difficult to understand as each script needs to be opened and evaluated.
With pass2php I now have a consistant 19 msec response time, no mather what scripts are behind it. Below test is done with a fully filled php scripting behind it.
So I think I can conclude I reached the maximum speed for my system and for my script

Code: Select all

Blockly 12 msec (1 blockly)
If pir = on do set wasbak = on
2016-12-29 23:55:03.451  EventSystem: Event triggered: pirkeuken_1
2016-12-29 23:55:03.446  (ZWAVE) Light/Switch (pirkeuken)
2016-12-29 23:55:03.463  OpenZWave: Domoticz has send a Switch command! NodeID: 6 (0x06)

lua 14 msec (1 lua device script)
commandArray={}
if devicechanged['pirkeuken'] == 'On' then
	commandArray['wasbak']='On'
end
return commandArray
2016-12-30 00:01:59.428  EventSystem: Script event triggered: /usr/local/domoticz/var/scripts/lua/script_device_pirkeuken.lua
2016-12-30 00:01:59.424  (ZWAVE) Light/Switch (pirkeuken)
2016-12-30 00:01:59.438  OpenZWave: Domoticz has send a Switch command! NodeID: 6 (0x06)

Blockly 20 msec (5 blocklies)
2016-12-30 00:10:40.973  EventSystem: Event triggered: pirkeuken_1
2016-12-30 00:10:40.970  (ZWAVE) Light/Switch (pirkeuken)
2016-12-30 00:10:40.990  OpenZWave: Domoticz has send a Switch command! NodeID: 6 (0x06)

lua 37 msec (5 lua device scripts)
2016-12-30 00:05:21.635  EventSystem: Script event triggered: /usr/local/domoticz/var/scripts/lua/script_device_pirkeuken.lua
2016-12-30 00:05:21.628  (ZWAVE) Light/Switch (pirkeuken)
2016-12-30 00:05:21.665  OpenZWave: Domoticz has send a Switch command! NodeID: 6 (0x06)

pass2php (hundreds of scripts)
if($status=="On"){
	if(apcu_fetch('skeuken')=='Off'&&apcu_fetch('swasbak')=='Off'&&apcu_fetch('swerkblad')=='Off'&&apcu_fetch('skookplaat')=='Off'&&apcu_fetch('zon')<500){
		sw(apcu_fetch('iwasbak'),'On','wasbak');
		apcu_store('twasbak',time);
	}
	if((apcu_fetch('sweg')=='On'||apcu_fetch('sslapen')=='On')&&apcu_fetch('smeldingen')=='On'&&apcu_fetch('tweg')<time-178&&apcu_fetch('tslapen')<time-178){
		sw(apcu_fetch('isirene'),'On');
		telegram('Beweging keuken om '.strftime("%k:%M:%S",time),false,3);
	}
}

2016-12-30 00:16:20.361  (ZWAVE) Light/Switch (pirkeuken)
2016-12-30 00:16:20.380  User: Admin initiated a switch command (61/wasbak/On)
2016-12-30 00:16:20.380  OpenZWave: Domoticz has send a Switch command! NodeID: 6 (0x06)

[sincze]

Whoehoe that is a great performance. My numbers are Not nearly that close with a half converted system (rain and weather, my thermostat still in lua.) the rest already disabled. Will you wiki it a bit as well? the relationship between files. It will definitely help starters figuring out what goes where and why . Again excellent work and I will try to match the numbers using cget and cset.

Sent from my SM-G925F using Tapatalk

[Toulon7559]

@Egregius

Have done some digging related to the PHP-capabilities for my local and remote webservers.
The remote webserver allows at choice PHP versions 5.6 till 7.
For local experiments at a PC the version PHP version 5.6 seems easiest (and free) available.

Which version is most compatible with your script Pass2PHP?
Any differences in operation to be expected?

[sincze]

@Egregius

Have done some digging related to the PHP-capabilities for my local and remote webservers.
The remote webserver allows at choice PHP versions 5.6 till 7.
For local experiments at a PC the version PHP version 5.6 seems easiest (and free) available.

Which version is most compatible with your script Pass2PHP?
Any differences in operation to be expected?

5.6 is okay for speed 7

Sent from my SM-G925F using Tapatalk

[sincze]

Another nice thing about the new @include for your case: you don't need the underscores anymore, you can have spaces in the filenames, just tested with a device "test scha ke laar" and a "pass2php/test scha ke laar.php" file and worked

A yes spaces or nospaces it now indeed works excellent, modified all the php files in the pass2php subdirectory so it is now one happy .php family. Next step is store the $s and $i in memcache right so speed things up so we only need $c.

Code: Select all

foreach($c as $device=>$status)
{    
    if(@include '/var/www/php/pass2php/'.$device.'.php')
    {
        cset('t'.$device,time);
        $dev=$device;
        if($debug=="true") lg('Hi, I did find: '.$device);
    }
}
$split=microtime(true);
if(!isset($dev))die();
//include '/var/www/php/pass3php/__CRON.php';    

[Egregius]

@Toulon7559:
I don't think there are functions used that are only available in PHP7, so 5.6 should do just fine. There is a performance benefit in PHP7 tough.

@Sincze:
Storing the $i array isn't very difficult. See the cron604800 part of https://github.com/Egregius/LUA-Pass2PH ... __CRON.php (at the moment lines 180-199).

Anyway, updated Github with my latest version, in use for 24 hours without issues now.

[sincze]

@Sincze:
Storing the $i array isn't very difficult. See the cron604800 part of https://github.com/Egregius/LUA-Pass2PH ... __CRON.php (at the moment lines 180-199).

Anyway, updated Github with my latest version, in use for 24 hours without issues now.

Currently working on it,
So script is called.

Code: Select all

if(cget('cron604800')<time-604799){
    if($debug=="true") lg('Hi, I am updating the cache now');
    cset('cron604800',time);
    $cron.=' + 604800';
    $domoticz=json_decode(file_get_contents(domoticz.'json.htm?type=devices&used=true'),true);
    if($domoticz){
        foreach($domoticz['result'] as $dom){
            $name=$dom['Name'];
            if(strtotime($dom['LastUpdate'])!=cget('t'.$name))
            { 
                cset('t'.$name,strtotime($dom['LastUpdate']));
                if($debug=="true") lg('Hi, I am updating the cache time value now for: '.'t'.$name.' with value '.$dom['LastUpdate']);
            }
            if($dom['idx']!=cget('i'.$name))
            {
                cset('i'.$name,$dom['idx']);
                if($debug=="true") lg('Hi, I am updating the cache idx value now for: '.'i'.$name.' with value '.$dom['idx']);
            }
        }
    }
}

I cheated a bit by temporarily commenting out the if(cget(çron604800)
As a result a whole list of these appeared.

Code: Select all

2016-12-30 13:37:38.504 --->> Hi, I am updating the cache time value now for: tIPCAM (RF) with value 2016-11-12 12:54:58
2016-12-30 13:37:38.511 --->> Hi, I am updating the cache idx value now for: iIPCAM (RF) with value 556

However:

Code: Select all

if($debug=="true") lg('Hi, the idx for IPCAM is: '.cget('iIPCAM (RF)'));

Results in..

Code: Select all

2016-12-30 13:39:40.695 --->> Hi, the idx for IPCAM is: 0

Same goes for a non spaced device:

Code: Select all

if($debug=="true") lg('Hi, the idx for ZonWeringAuto is: '.cget('iZonweringAuto'));

Code: Select all

2016-12-30 13:42:47.151 --->> Hi, the idx for ZonWeringAuto is: 0

Memcache is running as this works:

Code: Select all

if($debug=="true") lg('Give me the answer to the light condition: '.cget('var_dark'));

Code: Select all

 --->> Give me the answer to the light condition: False

So I guess it has to do something with:
cset('t'.$name,strtotime($dom['LastUpdate']));

It does not seem to actually 'set' the variables.

[Egregius]

Don't know if spaces and brackets are allowed, shall test that tommorow.
Don't forget the i or t in front of the name...

[sincze]

Don't know if spaces and brackets are allowed, shall test that tommorow.
Don't forget the i or t in front of the name...

Hehe well here is the answer to the space issue
http://stackoverflow.com/questions/5826 ... ain-spaces
https://github.com/memcached/memcached/ ... otocol.txt[/url]
No. Memcached keys cannot contain spaces.

Code: Select all

Keys

Data stored by memcached is identified with the help of a key. A key is a text string which should uniquely identify the data for clients that are interested in storing and retrieving it. Currently the length limit of a key is set at 250 characters (of course, normally clients wouldn't need to use such long keys); the key must not include control characters or whitespace.

[sincze]

Don't know if spaces and brackets are allowed, shall test that tommorow.
Don't forget the i or t in front of the name...

Hehe well here is the answer to the space issue
http://stackoverflow.com/questions/5826 ... ain-spaces
https://github.com/memcached/memcached/ ... otocol.txt[/url]
No. Memcached keys cannot contain spaces.

Code: Select all

Keys

Data stored by memcached is identified with the help of a key. A key is a text string which should uniquely identify the data for clients that are interested in storing and retrieving it. Currently the length limit of a key is set at 250 characters (of course, normally clients wouldn't need to use such long keys); the key must not include control characters or whitespace.

Modified the code bit so cset and cget will work again without having to modify all the device names.(strpos($name, " ") !== false)

Code: Select all

if(cget('cron604800')<time-604799){
    cset('cron604800',time);
    $cron.=' + 604800';
    $domoticz=json_decode(file_get_contents(domoticz.'json.htm?type=devices&used=true'),true);
    if($debug=="true") lg('Hi, I rertieved values from domoticz now');
    if($domoticz){
        foreach($domoticz['result'] as $dom){
            $name=$dom['Name'];
            if (strpos($name, " ") !== false) $name=str_replace(" ","_",$name);
            if(strtotime($dom['LastUpdate'])!=cget('t'.$name)) cset('t'.$name,strtotime($dom['LastUpdate']));
            if($dom['idx']!=cget('i'.$name)) cset('i'.$name,$dom['idx']);            
        }
    }
}

[sincze]

Well I managed to get it working all via memcached, however my test switch is now inverted....

Switch ON result Light OFF (expected light ON):

Code: Select all

2016-12-30 18:44:15.195 User: Admin initiated a switch command (621/ZonweringAuto/On)
2016-12-30 18:44:15.197 (Virtual Device) Light/Switch (ZonweringAuto)
2016-12-30 18:44:15.334 --->> 1. In ZonweringAuto.php Finding Status of ZonWeringAuto the status is: Off
2016-12-30 18:44:15.340 --->> SWITCH Off CV Ruimte
2016-12-30 18:44:15.343 User: Admin initiated a switch command (531/Lamp (CV Ruimte) (RF)/Off)
2016-12-30 18:44:15.345 RFLink Sending: 10;NewKaku;ba1e09;a;OFF
2016-12-30 18:44:15.512 (RFXCom) Lighting 2 (Lamp (CV Ruimte))
2016-12-30 18:44:17.180 (RFLink Gateway) Light/Switch (Lamp (CV Ruimte) (RF))
2016-12-30 18:44:17.364 --->> 2. In ZonweringAuto.php Finding Status of ZonWeringAuto the status is: Off
2016-12-30 18:44:17.367 --->> In pass2php we see a status switch: ZonweringAuto status: On

So the status change ZonweringAuto to 'On' is shown later in the Domoticz log after the action has been executed. It is exactly as what happens. next time I switch ZonweringAuto to 'Off'...

Switch OFF result Light ON (expected light OFF)

Code: Select all

2016-12-30 18:48:03.204 --->> 1. In ZonweringAuto.php Finding Status of ZonWeringAuto the status is: On
2016-12-30 18:48:03.209 --->> SETLEVEL 100 CV Ruimte
2016-12-30 18:48:03.212 User: Admin initiated a switch command (531/Lamp (CV Ruimte) (RF)/Set Level)
2016-12-30 18:48:03.214 RFLink Sending: 10;NewKaku;ba1e09;a;15
2016-12-30 18:48:03.425 (RFXCom) Lighting 2 (Lamp (CV Ruimte))
2016-12-30 18:48:04.247 (RFLink Gateway) Light/Switch (Lamp (CV Ruimte) (RF))
2016-12-30 18:48:04.434 --->> 2. In ZonweringAuto.php Finding Status of ZonWeringAuto the status is: On
2016-12-30 18:48:04.437 --->> In pass2php we see a status switch: ZonweringAuto status: Off

And just for the check my ZonweringAuto,php

Code: Select all

<?php
    
    lg('1. In ZonweringAuto.php Finding Status of ZonWeringAuto the status is: '.(cget('sZonweringAuto')));

    if(cget('sZonweringAuto')=="On") sl((cget('iLamp_(CV_Ruimte)_(RF)')), 100,'CV Ruimte');

    elseif(cget('sZonweringAuto')=="Off") sw((cget('iLamp_(CV_Ruimte)_(RF)')), 'Off' ,'CV Ruimte');

    lg('2. In ZonweringAuto.php Finding Status of ZonWeringAuto the status is: '.(cget('sZonweringAuto')));
     

[Egregius]

Lg 1 and 2 will always print the same status as lg2 will be executed well before a new cset happens.

[sincze]

Lg 1 and 2 will always print the same status as lg2 will be executed well before a new cset happens.

A yes. That was just an extra debug line main question remains. Why is it inversed. No more hairs on my head.. I've been pulling them

Sent from my SM-G925F using Tapatalk