Domoticz

Open source Home Automation System
https://forum.domoticz.com/

Pass2PHP

https://forum.domoticz.com/viewtopic.php?t=12343

Page 71 of 76

Re: Pass2PHP

Posted: Friday 20 November 2020 10:16
by sincze
Tnx for the update.

Need to look at the github repository to see the changes and how to move to tmpfs. :)

Re: Pass2PHP

Posted: Friday 20 November 2020 10:36
by Egregius
Keep in mind dat tmpfs can cause data loss! Don't use it if you find your data of the day important!

First try the other stuff to improve and do that only if you want it at the maximum.

In short:
the secure/atreboot.sh script copies the files of domoticz and mysql to /temp.
/temp is created in /etc/fstab as tmpfs
the secure/atshutdown.sh syncs the changes from /temp to disk. I run it by cron once a day.
In /etc/init.d/domoticz.sh I added the same sync in the do_stop() function to sync changes to disk when domoticz stops.
I made a symlink from /var/lib/mysql to /temp/mysql
And probably some other things I don't think of right now...

Re: Pass2PHP

Posted: Friday 20 November 2020 14:15
by sincze
Ok so first start with :
wget instead of curl in the lua script (biggest change).

Got it. ;-)

Re: Pass2PHP

Posted: Friday 20 November 2020 14:40
by Egregius
Tell me, tell me... How big is the change on your system? Is it also 5 msec? I went form 15 to 10 with that.

Re: Pass2PHP

Posted: Friday 20 November 2020 17:41
by sincze
Current intermediate setup. (due to still in migration process)

Changed the LUA.

Code: Select all

os.execute('wget -O /dev/null -o /dev/null "http://192.168.1.200/secure/pass2php.php?d='..d..'&s='..s..'" &')
So.
- A Domoticz Pi3 B machine (192.168.1.100) executes the LUA as this machine is running Domoticz.

LUA script is calling pass2php.php that can be found on a different Raspberry Pi4 (192.168.1.200)
This PI4 has NGINX + PHP 7.4 and MariaDB running. That PI will talk to Domoticz (192.168.1.100) to actually switch on/off the light.

Code: Select all

2020-11-20 17:30:20.709 Status: --->> (Pass2PHP SWITCH) | Lamp (CV Ruimte) (RF) => Off
2020-11-20 17:30:20.722 Status: User: Admin initiated a switch command (531/Lamp (CV Ruimte) (RF)/Off)

Code: Select all

2020-11-20 17:31:56.039 Status: --->> (Pass2PHP SWITCH) | Function Licht_Badkamer On device: Deur Badkamer with status: Open
2020-11-20 17:31:56.054 Status: User: Admin initiated a switch command (1104/Lamp Badkamer/On)

Re: Pass2PHP

Posted: Saturday 21 November 2020 6:03
by Egregius
I don't see the triggering device line in your log. Here I see a delay of 15msec between pass2php and domoticz, that'll be caused by it installed on another Pi I guess. Those timings should be 0.

Re: Pass2PHP

Posted: Saturday 21 November 2020 6:40
by Egregius
Isn't this interesting? :D
Keep in mind that these aren't real life tests. In all tests other than pass2php there was only one small tiny script.

Code: Select all

domoticz_main
#!/bin/sh
wget -O /dev/null -o /dev/null "http://127.0.0.1/secure/pass2php.php?d=$6&s=$4" &

2020-11-21 07:12:17.379  (ZWAVE) Light/Switch (pirkeuken)
2020-11-21 07:12:18.387  Status: Executing script: /temp/domoticz/scripts/domoticz_main
2020-11-21 07:12:18.400  (STORE)                => pirkeuken    => On   (Pass2PHP)
2020-11-21 07:12:18.400  (SWITCH)               =>keuken=>On (functions.php:1027)
2020-11-21 07:12:18.400  Status: User: Admin initiated a switch command (11/keuken/On)
2020-11-21 07:12:18.400  OpenZWave: Domoticz has send a Switch command! NodeID: 4 (0x04)
1008 msec

Code: Select all

dzVents
return {
	on = {
		devices = {
			'pirkeuken'
		}
	},
	execute = function(domoticz, device)
		domoticz.devices('keuken').switchOn()
	end
}
2020-11-21 06:20:16.682  (ZWAVE) Light/Switch (pirkeuken)
2020-11-21 06:20:16.729  OpenZWave: Domoticz has send a Switch command! NodeID: 4 (0x04)
47 msec

Code: Select all

lua file
commandArray = {}
for deviceName,deviceValue in pairs(devicechanged) do
    if (deviceName=='pirkeuken') then
            commandArray['keuken'] = "On"
    end
end
return commandArray
2020-11-21 06:29:25.006  (ZWAVE) Light/Switch (pirkeuken)
2020-11-21 06:29:25.036  OpenZWave: Domoticz has send a Switch command! NodeID: 4 (0x04)
30 msec

Code: Select all

Blockly
2020-11-21 06:07:23.155  (ZWAVE) Light/Switch (pirkeuken)
2020-11-21 06:07:23.180  OpenZWave: Domoticz has send a Switch command! NodeID: 4 (0x04)
25 msec

Code: Select all

lua database
2020-11-21 06:27:17.113  (ZWAVE) Light/Switch (pirkeuken)
2020-11-21 06:27:17.132  OpenZWave: Domoticz has send a Switch command! NodeID: 4 (0x04)
19 msec

Code: Select all

pass2php from lua database
2020-11-21 06:35:05.848  (ZWAVE) Light/Switch (pirkeuken)
2020-11-21 06:35:05.864  (STORE)                => pirkeuken    => On   (Pass2PHP)
2020-11-21 06:35:05.864  (SWITCH)               =>keuken=>On (functions.php:1027)
2020-11-21 06:35:05.864  Status: User: Admin initiated a switch command (11/keuken/On)
2020-11-21 06:35:05.864  OpenZWave: Domoticz has send a Switch command! NodeID: 4 (0x04)
16 msec

Code: Select all

pass2php from lua file
2020-11-21 06:37:34.572  (ZWAVE) Light/Switch (pirkeuken)
2020-11-21 06:37:34.582  (STORE)                => pirkeuken    => On   (Pass2PHP)
2020-11-21 06:37:34.582  (SWITCH)               =>keuken=>On (functions.php:1027)
2020-11-21 06:37:34.582  Status: User: Admin initiated a switch command (11/keuken/On)
2020-11-21 06:37:34.582  OpenZWave: Domoticz has send a Switch command! NodeID: 4 (0x04)
10 msec

Re: Pass2PHP

Posted: Saturday 21 November 2020 13:47
by sincze
All running on the Pi4 now.

Xiaomi ZIgbee door sensor triggering a Tasmota (Sonoff Basic)

Code: Select all

2020-11-21 13:41:52.730 Status: {"cmd":"report","model":"magnet","sid":"158d000171ea5e","short_id":19206,"data":"{\"status\":\"open\"}"}
2020-11-21 13:41:52.745 Status: {"cmd":"report","model":"magnet","sid":"158d000171ea5e","short_id":19206,"data":"{\"status\":\"open\"}"}
2020-11-21 13:41:52.824 Status: --->> Deur Badkamer: Open
2020-11-21 13:41:52.825 Status: --->> (Pass2PHP SWITCH) | Function Licht_Badkamer On device: Deur Badkamer with status: Open
2020-11-21 13:41:52.826 Status: User: Admin initiated a switch command (1104/Lamp Badkamer/On)

Xiaomi ZIgbee door sensor / motion triggering a 433 MHZ Inwall kaku

Code: Select all

2020-11-21 14:30:40.544 (GW-Xiaomi-Boven) Light/Switch (Deur WC Boven)
2020-11-21 14:30:40.549 Status: {"cmd":"report","model":"magnet","sid":"158d0001e5be3a","short_id":11032,"data":"{\"status\":\"open\"}"}
2020-11-21 14:30:40.552 (GW-Xiaomi-Boven) Light/Switch (Deur WC Boven)
2020-11-21 14:30:40.555 Status: {"cmd":"report","model":"magnet","sid":"158d0001e5be3a","short_id":11032,"data":"{\"status\":\"open\"}"}
2020-11-21 14:30:40.575 Status: LUA: URL HAS BEEN CALLED for: Deur WC Boven
2020-11-21 14:30:40.596 Status: --->> (Pass2PHP SWITCH) | Function Licht_WC_Boven On device: Deur WC Boven with status: Open
2020-11-21 14:30:40.597 Status: User: Admin initiated a switch command (1134/Lamp WC Boven/On)
2020-11-21 14:30:40.770 (RFXCom) Lighting 2 (Unknown)
2020-11-21 14:30:41.517 (GW RFLink LAN) Light/Switch (Lamp WC Boven)
Xiaomi Motion Sensor trigger Tasmota lightswitch.

Code: Select all

2020-11-21 17:52:49.048 Status: {"cmd":"report","model":"motion","sid":"158d000222ef6e","short_id":2560,"data":"{\"status\":\"motion\"}"}
2020-11-21 17:52:49.085 Status: {"cmd":"report","model":"motion","sid":"158d000222ef6e","short_id":2560,"data":"{\"status\":\"motion\"}"}
2020-11-21 17:52:49.121 Status: --->> (Pass2PHP SWITCH) | Motion Logeerkamer
2020-11-21 17:52:49.125 Status: User: Admin initiated a switch command (1285/Lamp Logeerkamer/On)

Re: Pass2PHP

Posted: Sunday 22 November 2020 10:33
by ropske
Egregius wrote: ↑Friday 20 November 2020 7:20 My Ring doorbell is working with IFTTT but that sucks, sometimes the alerts arrive hours later, even with a premium IFTTT account. I've put a Python script to poll the Ring alerts every 3 seconds. That's running on a server with another ip and another ring account because I got locked for some reason. It's running good now, but still like to handle it in PHP just because I know that better.

The only problem I have at the moment with Zwave is for the Qubino ZMNHBA2 Flush 2 Relays that don't report their correct status. Everything else works as it should. I'm running Domoticz 2020.2 Build 12485 with openzwave 1.6-1392-gb0afd4c6. The Qubino's also have the right status in the OZW control panel, but not in Domoticz. I'm using a kind of refresh script to have the right status but it's not ideal.
Oh, wait. I disabled that again because it did to much on the zwave network and caused delays.
It was something like this: if ($d['pirkeuken']['s']=='On') RefreshZwave(4);


seems you are running a newer Domoticz with openzwave 1.6
mine is: Version: 1.6-1080-g5db357e3

is it possible to update the openzwave seperately?

Re: Pass2PHP

Posted: Sunday 22 November 2020 10:35
by ropske
btw, i'm running the stable version

Re: Pass2PHP

Posted: Sunday 22 November 2020 16:43
by Egregius
@Sincze: Do I see it correctly that there are great differences between calls? That's pretty strange. I always get the same respone time.
@Ropske: I'm running beta ;)

Re: Pass2PHP

Posted: Sunday 22 November 2020 18:52
by ropske
ok, maybe i better try to update to beta version then and see how the openzwave 1.6 is reacting (hopefully better than now lol)

Re: Pass2PHP

Posted: Sunday 22 November 2020 20:09
by sincze
Egregius wrote: ↑Sunday 22 November 2020 16:43 @Sincze: Do I see it correctly that there are great differences between calls? That's pretty strange. I always get the same respone time.
This is a Xiaomi ZIGBEE doorsensor triggering a Tasmota light.

Code: Select all

2020-11-22 19:50:54.309 Status: {"cmd":"report","model":"magnet","sid":"158d00029b0dfb","short_id":13505,"data":"{\"status\":\"open\"}"}
2020-11-22 19:50:54.318 Status: {"cmd":"report","model":"magnet","sid":"158d00029b0dfb","short_id":13505,"data":"{\"status\":\"open\"}"}
2020-11-22 19:50:54.372 Status: --->> (Pass2PHP SWITCH) | Deur Logeerkamer Open
2020-11-22 19:50:54.374 Status: User: Admin initiated a switch command (1285/Lamp Logeerkamer/On)
Have no Clue,

Domoticz log is pretty filling up with all the events that are happening

Code: Select all

lg('(STORE) Pass2php: Called '.$device.' = Received '.$status.' status');
Not enabling that function all the time ;-)

Also many interactions with the influxDB server on a remote machine to store all the updates from the temperature and energy sensors.
System load is 2.60 and memory in use only 1 GB of the 8 GB in total. PHP is consuming 365 MB.
monit.PNG
monit.PNG (70.57 KiB) Viewed 2918 times

Re: Pass2PHP

Posted: Monday 23 November 2020 12:34
by Egregius
I would start by saying dump monit...
Tried it this morning. When freshly started my system load was 0.6-0.7.
Configured monit to monitor domoticz, php-fpm, nginx, mysql, cron. Several hours later my system load was 1.5-1.6???

How much data is sent to influx db? Maybe on option to only have it by cron?
How much calls to pass2php do you have approx? If you have APCu enabled you could count this by adding apcu_inc('pass2php_raw'); to pass2php.php

Maybe you need to filter the number of calls. Either on the lua side, or die on the php side. Your system is a lot more stressed than mine.

Re: Pass2PHP

Posted: Monday 23 November 2020 12:37
by Egregius
If you change pass2php to this you'll see what's doing most calls:

Code: Select all

apcu_inc('pass2php_raw');
require '/var/www/html/secure/functions.php';
$device=$_REQUEST['d'];
$status=$_REQUEST['s'];

if (endswith($device, '_Temperature')) die('Ignore these doubles');
elseif (endswith($device, '_Utility')) die('Ignore these doubles');

apcu_inc('pass2php_net');
apcu_inc($device);

$d=fetchdata();
if ($d[$device]['dt']=='dimmer'||$d[$device]['dt']=='rollers'||$d[$device]['dt']=='luifel') {
	if ($status=='Off'||$status=='Open') {
		$status=0;
	} elseif ($status=='On'||$status=='Closed') {
		$status=100;
	} else {
		$status=filter_var($status, FILTER_SANITIZE_NUMBER_INT);
	}
} elseif (in_array($device, array('badkamer_temp'))) {
	$status=explode(';', $status);
	$status=$status[0];
} elseif ($device=='achterdeur') {
	if ($status=='Open') {
		$status='Closed';
	} else {
		$status='Open';
	}
} elseif ($device=='sirene') {
	if ($status=='Group On') {
		$status='On';
	} else {
		$status='Off';
	}
}
store($device, $status, 'Pass2PHP');
if(@include '/var/www/html/secure/pass2php/'.$device.'.php')apcu_inc('pass2php_effective');

Re: Pass2PHP

Posted: Monday 23 November 2020 18:18
by sincze
Egregius wrote: ↑Monday 23 November 2020 12:34 I would start by saying dump monit...
Tried it this morning. When freshly started my system load was 0.6-0.7.
Configured monit to monitor domoticz, php-fpm, nginx, mysql, cron. Several hours later my system load was 1.5-1.6???

How much data is sent to influx db? Maybe on option to only have it by cron?
How much calls to pass2php do you have approx? If you have APCu enabled you could count this by adding apcu_inc('pass2php_raw'); to pass2php.php

Maybe you need to filter the number of calls. Either on the lua side, or die on the php side. Your system is a lot more stressed than mine.
Forgot to install ACPU so will start with that first :D
Egregius-APCU.PNG
Egregius-APCU.PNG (33.46 KiB) Viewed 2891 times
Let me start with that. :lol:

All device updates that contain TEMP / Watt are pushed to influxdb.

Let me try your other solutions as well. This is what my system is running next to Domoticz.
Dockers.PNG
Dockers.PNG (51.32 KiB) Viewed 2891 times

Re: Pass2PHP

Posted: Monday 23 November 2020 18:50
by ropske
gonna update tonight to beta, fingers crossed :oops:

Re: Pass2PHP

Posted: Monday 23 November 2020 20:05
by sincze
Egregius wrote: ↑Monday 23 November 2020 12:37 If you change pass2php to this you'll see what's doing most calls:

Code: Select all

apcu_inc('pass2php_raw');
Modification Done (via Github) ;-) example.

Also downloaded acpu.php and acpu.css from your website.
Egregius-APCU2.PNG
Egregius-APCU2.PNG (54.44 KiB) Viewed 2887 times
Values time are updated , however hits stays 0.
Maybe I missed a command to activate it ?


for the sake of.. I added

Code: Select all

lg($device.' count '.apcu_fetch($device));
lg('pass2php_net '.apcu_fetch('pass2php_net'));
So at least we know something is working.

Code: Select all

2020-11-23 20:24:48.234 Status: --->> Airco_Temperature count 91
2020-11-23 20:24:48.235 Status: --->> pass2php_net 8124
2020-11-23 20:24:48.250 Status: --->> Airco count 91
2020-11-23 20:24:48.251 Status: --->> pass2php_net 8125
2020-11-23 20:24:48.331 Status: --->> Airco (kWh) count 91
2020-11-23 20:24:48.333 Status: --->> pass2php_net 8126
2020-11-23 20:24:48.347 Status: --->> Airco (kWh)_Utility count 91
2020-11-23 20:24:48.349 Status: --->> pass2php_net 8127
2020-11-23 20:24:48.441 Status: --->> Stroom Keuken TV (kWh) count 91
2020-11-23 20:24:48.442 Status: --->> pass2php_net 8128
2020-11-23 20:24:48.479 Status: --->> Airco (Watt) count 91
2020-11-23 20:24:48.481 Status: --->> pass2php_net 8129
2020-11-23 20:24:48.506 Status: --->> Airco (Watt)_Utility count 91
2020-11-23 20:24:48.514 Status: --->> pass2php_net 8130
2020-11-23 20:24:48.544 Status: --->> Stroom Keuken TV (Watt) count 91
2020-11-23 20:24:48.545 Status: --->> pass2php_net 8131
2020-11-23 20:24:48.589 Status: --->> Stroom Keuken TV (V)_Utility count 91
2020-11-23 20:24:48.591 Status: --->> pass2php_net 8132
2020-11-23 20:24:48.605 Status: --->> Stroom Keuken TV (V) count 91
2020-11-23 20:24:48.607 Status: --->> pass2php_net 8133
2020-11-23 20:24:48.656 Status: --->> Stroom Keuken TV (A) count 91
2020-11-23 20:24:48.657 Status: --->> pass2php_net 8134
OPcache.
Egregius-APCU2.PNG
Egregius-APCU2.PNG (54.44 KiB) Viewed 2887 times

Re: Pass2PHP

Posted: Tuesday 24 November 2020 6:48
by Egregius
How long was it running to reach pass2php_net 8134?

Mine has now been running 18 hours:
pass2php_raw 6489 request
pass2php_net 4551 after filtering _Temperature and _Utility devices
pass2php_effective 2983 executed because file exists in pass2php folder.

Re: Pass2PHP

Posted: Tuesday 24 November 2020 6:50
by Egregius
ropske wrote: ↑Monday 23 November 2020 18:50 gonna update tonight to beta, fingers crossed :oops:
If you have a good backup there should be a risk. I'm running beta since the beginning 5 years ago.
All times are UTC+02:00
Page 71 of 76

Powered by phpBB® Forum Software © phpBB Limited