Pass2PHP

[Egregius]

@Trigun: I think you have to do a POST in some way, similar like ropske does.

@ropske: there's a difference between your php and shell. In php you send text= and in shell name=
In wurst case: create some shell scripts with the commands you want to send and use shell_exec in php to execute them.

[Egregius]

@trigun:
Is it possible your tv has a built in webserver listening on port 8080?
Can you try this simple get request:

Code: Select all

http://ipofyoursamsungtv:8080/api/stv/poweroff

[Trigun]

@trigun:
Is it possible your tv has a built in webserver listening on port 8080?
Can you try this simple get request:

Code: Select all

http://ipofyoursamsungtv:8080/api/stv/poweroff

He guys, I learned another new thing yesterday. It’s probably me but didn’t know when installing mosquitto, domoticz is publishing every action your switches or sensors make. Therefore I can with node red pick this up when I subscribe to it. Because a lot of topics go by, I didn’t see my actions and assumed I had to create a script or something. But yesterday I got it to work and to be honest, it couldn’t be simpeler. Thank you all for your efforts and I will keep following this topic as I am very interested in connecting the google home to my domoticz .
Thanks guys!


Sent from my iPhone using Tapatalk

[ropske]

Hi Egregius and others, i need you help

since 21/09/2020 my _cron300.php and _cron3600.php are not working anymore.
But i don't know why.

this is what is in my cron.php:

Code: Select all

<?php
//require 'config.php';
require 'functions.php';
$d=fetchdata();
//if (isset($_REQUEST['cron10']))	include '_cron10.php';
if (isset($_REQUEST['verwarming'])) include '_verwarming.php';
if (isset($_REQUEST['rolluiken'])) include '_rolluiken.php';
if (isset($_REQUEST['cron60'])){
	lgmsg('cron60 wordt aangeropen!');
	include '_cron60.php';
}
//if (isset($_REQUEST['cron120'])) include '_cron120.php';
//if (isset($_REQUEST['cron180'])) include '_cron180.php';
//if (isset($_REQUEST['cron240'])) include '_cron240.php';
if (isset($_REQUEST['cron300'])){
	lgmsg('cron300 wordt aangeropen!');
	include '_cron300.php';
}
if (isset($_REQUEST['cron3600'])) include '_cron3600.php';

i can see the message 'cron60 wordt aangeropen' every 60seconds
but i don't see the 'cron300 wordt aangeropen' (even after more than 5minutes waiting)


this is what inside my cron.sh:

Code: Select all

#!/bin/bash

: '
crontab -e
# m h  dom mon dow   command
* * * * * /usr/bin/nice -n20 /var/www/html/secure/cron.sh >/dev/null 2>&1
* * * * * /usr/bin/nice -n20 /var/www/html/secure/cronsmappee.sh >/dev/null 2>&1
0 0 * * * /usr/bin/nice -n20 curl -s "http://127.0.0.1/secure/cleandomoticzdb.php" >/dev/null 2>&1
1 0 * * * /usr/bin/nice -n20 /var/www/_SQLBackup/sqldaily.sh >/dev/null 2>&1
*/5 * * * * /usr/bin/nice -n20 /var/www/html/secure/cleandisk.sh >/dev/null 2>&1
'

DOMOTICZ=`curl -s --connect-timeout 2 --max-time 5 "http://127.0.0.1:8080/json.htm?type=devices&rid=1"`
STATUS=`echo $DOMOTICZ | jq -r '.status'`
if [ "$STATUS" == "OK" ] ; then
    NOW=$(date +"%Y-%m-%d %H:%M:%S")
    MINUTE=$(date +"%M")
    CRON=""
    if [ $(($MINUTE%2)) -eq 0 ] ; then
        CRON="$CRON&cron120"
    fi
    if [ $(($MINUTE%3)) -eq 0 ] ; then
        CRON="$CRON&cron180"
    fi
    if [ $(($MINUTE%4)) -eq 0 ] ; then
        CRON="$CRON&cron240"
    fi
    if [ $(($MINUTE%5)) -eq 0 ] ; then
        CRON="$CRON&cron300"
    fi
    if [ $MINUTE -eq 0 ] ; then
        CRON="$CRON&cron3600"
    fi
	#0
	curl -s --connect-timeout 2 --max-time 30 "http://127.0.0.1/secure/cron.php?cron60&cron10&verwarming$CRON&$DEBUG" >/dev/null 2>&1 &
	sleep 8.859
	#10
	curl -s --connect-timeout 2 --max-time 30 "http://127.0.0.1/secure/cron.php?cron10&$DEBUG" >/dev/null 2>&1 &
	sleep 9.998
	#20
	curl -s --connect-timeout 2 --max-time 30 "http://127.0.0.1/secure/cron.php?cron10&$DEBUG" >/dev/null 2>&1 &
	sleep 9.998
	#30
	curl -s --connect-timeout 2 --max-time 30 "http://127.0.0.1/secure/cron.php?cron10&$DEBUG" >/dev/null 2>&1 &
	sleep 9.998
	#40
	curl -s --connect-timeout 2 --max-time 30 "http://127.0.0.1/secure/cron.php?cron10&$DEBUG" >/dev/null 2>&1 &
	sleep 9.998
	#50
	curl -s --connect-timeout 2 --max-time 30 "http://127.0.0.1/secure/cron.php?cron10&$DEBUG"
	if [ $? -gt 0 ] ; then
		/usr/sbin/service apache2 restart
	fi
else
	sleep 20
	DOMOTICZ=`curl -s --connect-timeout 2 --max-time 5 "http://127.0.0.1:8080/json.htm?type=devices&rid=1"`
	STATUS2=`echo $DOMOTICZ | jq -r '.status'`
	if [ "$STATUS2" == "OK" ] ; then
		exit
	else
		sleep 20
		DOMOTICZ=`curl -s --connect-timeout 2 --max-time 5 "http://127.0.0.1:8080/json.htm?type=devices&rid=1"`
		STATUS3=`echo $DOMOTICZ | jq -r '.status'`
		if [ "$STATUS3" == "OK" ] ; then
			exit
		else
			/usr/sbin/service domoticz stop
			/usr/sbin/service domoticz start
			#shutdown -r now
		fi
	fi
fi

ps cax | grep httpd
if [ $? -eq 0 ] ; then
	/usr/sbin/service apache2 stop
	/usr/sbin/service apache2 start
fi

[Egregius]

Does it work when you open the url in your browser?

Code: Select all

http://192.168.0.1/secure/cron.php?cron300&cron3600

I do think that you must remove all &$DEBUG as that variable is never defined in your script.

[Trigun]

Also, what does the error log say?


Sent from my iPhone using Tapatalk

[ropske]

@Egrerius:

that was indeed working, but i already found the problem.
I was the problem lol

you remember i was making a function to let my GoogleMini's speak?

Code: Select all

function speakToGoogleMini()
{
	$jsonString = json_encode(array("text" => "testje"));

	// You can directly replace your JSON string with $jsonString variable.
	$ch = curl_init('http://192.168.1.143/hello-form');
	curl_setopt($ch, CURLOPT_PORT, 1880);
    curl_setopt($ch, CURLOPT_POST, 1);
	//curl_setopt($ch, CURLOPT_POSTFIELDS, $jsonString);
	curl_setopt($ch, CURLOPT_POSTFIELDS, "text=testje");
	curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
	//curl_setopt($ch, CURLOPT_HTTPHEADER, array('Content-Type: application/json'));
	$result = curl_exec($ch);
	curl_close($ch);
}

but that code was not working?
Well, i called this function in my cron60.php
and it seems it skipped all the rest for some reason.

when i put this call (in cron60.php) in comment, everything works again

[Egregius]

There's no error in that function. I can execute it. There must be something in your php error log.

You could just add a timeout so the curl doesn't block the rest of the script.

Code: Select all

curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, 2);
curl_setopt($ch, CURLOPT_TIMEOUT, 2);

[Trigun]

I fetch a different site for sunrise/sunset.
Reason for this is that domoticz uses in my opinion the wrong type of sunrise/sunset.
For example today I see in domoticz ☀▲08:01 ▼18:03
While the civil twilight is 7:27 to 18:35.
At the sunset of domoticz it's already completely light outside. I rather use civil twilight so I can see the sunrise/sunset.

Anyway, the code...

from cron3600: Every hour I fetch the sunrise/sunset and store that.

Code: Select all

$sunrise=json_decode(
    file_get_contents(
        'https://api.sunrise-sunset.org/json?lat='
        .$lat.'&lng='
        .$lon.'&date=today&formatted=0'
    ),
    true
);
if (isset($sunrise['results']['civil_twilight_begin'])) {
    if (strtotime($sunrise['results']['civil_twilight_begin'])!=$d['civil_twilight']['s']) {
        store('civil_twilight', strtotime($sunrise['results']['civil_twilight_begin']), basename(__FILE__).':'.__LINE__);
    }
    if (strtotime($sunrise['results']['civil_twilight_end'])!=$d['civil_twilight']['m']) {
        storemode('civil_twilight', strtotime($sunrise['results']['civil_twilight_end']), basename(__FILE__).':'.__LINE__);
    }
}

from cron60: Every minute I check if current time is between sunrise and sunset. If that's the case I store that in the mode of auto.
This is an optional step. You could also just use the if statement everywhere. I just tought it would be easier to just have to if($d['auto']['m']==true) than each time the complete if civil...

Code: Select all

if (TIME>=$d['civil_twilight']['s']&&TIME<=$d['civil_twilight']['m']) {
    if ($d['auto']['m']!=true) {
        storemode('auto', true, basename(__FILE__).':'.__LINE__);
        $d['auto']['m']=true;
    }
} else {
    if ($d['auto']['m']!=false ) {
        storemode('auto', false, basename(__FILE__).':'.__LINE__);
        $d['auto']['m']=false;
    }
}

Hi Egregius,

if I use this code in my con60 and cron 3600, how do I use this in a script to switch a light at sunset?
also, something completely else, how do I add a different Cron, for example con7200?

thanks!

[Egregius]

I now have this in cron3600:

Code: Select all

$sunrise=json_decode(
    file_get_contents(
        'https://api.sunrise-sunset.org/json?lat='
        .$lat.'&lng='
        .$lon.'&date=today&formatted=0'
    ),
    true
);
if (isset($sunrise['results']['civil_twilight_begin'])) {
    if (strtotime($sunrise['results']['civil_twilight_begin'])!=$d['civil_twilight']['s']) {
        store('civil_twilight', strtotime($sunrise['results']['civil_twilight_begin']), basename(__FILE__).':'.__LINE__);
    }
    if (strtotime($sunrise['results']['civil_twilight_end'])!=$d['civil_twilight']['m']) {
        storemode('civil_twilight', strtotime($sunrise['results']['civil_twilight_end']), basename(__FILE__).':'.__LINE__);
    }
}

So every hour I store the civil twilight in s and m of civil_twilight.

Then, for example in the heating scripts:

Code: Select all

if (TIME>=$d['civil_twilight']['s']&&TIME<=$d['civil_twilight']['m']) $dag=true; else $dag=false;

Or in cron60:

Code: Select all

if (TIME>=$d['civil_twilight']['s']&&TIME<=$d['civil_twilight']['m']) {

Of course you can make any combination:

Code: Select all

if (TIME>=$d['civil_twilight']['s']&&TIME>strtotime('7:00')) {

This will only trigger after 7am and if the sun is up.

Why do you need a cron7200? Can't it just go in cron3600?

If really needed something like this in cron.sh:

Code: Select all

MINUTE=$(date +"%M")
HOUR=$(date +"%H") #Not sure about the H for hours, must be googled.


if [ $(($HOUR%2)) -eq 0 ] ; then
        CRON="$CRON&cron7200"
fi

Another option could be to do it in cron3600:

Code: Select all

$hour=date("H");
if ($hour%2==0) {
	echo 'Even hour';
} else {
	echo 'Odd hour';
}

[Trigun]

Great! thanks a lot! Do I have fill in the longitude and latitude for my own place somewhere?

perhaps another dumb question but what do the S and M stand for?

[Egregius]

Yes, you need $lat en $lon in your config file.
Example:

Code: Select all

$lat=50.123456;
$lon=3.987654;

's' useally holds the status of the device. But because pass2php auto generates a dummy switch you can store whatever you want in it with sw('device', value') or store('device', 'value')
'm' stand for 'mode'. I often use that to set things to auto, manual or whatever. You can set it with storemode('device', 'mode')

I'm curious what you want to do in the cron7200

[Trigun]

Great! I’m going to look into it

The cron7200 is needed for the following.
Sometimes my girlfriend is switching “auto” off because with auto on the light goes on automatically and because the door of the baby room is always open(a bit) the baby sometimes wake up because of the light(even when it’s dimmed. With the cron7200 I want “auto” to turn back on after 2 hours of being off so the system keeps doing what it does best.


Sent from my iPhone using Tapatalk

[Egregius]

Something like this you mean?

Code: Select all

if ($d['auto']['s']!='On') {
	if (past('auto')>10795) {
		sw('auto', 'On', basename(__FILE__).':'.__LINE__);
	}
}

Mine is in cron300.

[Trigun]

Something like this you mean?

Code: Select all

if ($d['auto']['s']!='On') {
	if (past('auto')>10795) {
		sw('auto', 'On', basename(__FILE__).':'.__LINE__);
	}
}

Mine is in cron300.

Haha yeah I guess so...


Sent from my iPhone using Tapatalk

[kispalsz]

Doesn’t look bad. 16 msec for domoticz, 1 msec for pass2php to respond.

Code: Select all

 2020-01-30 22:22:20.771  (Xiaomi) Light/Switch (deurgarage)
2020-01-30 22:22:20.787  (STORE)		=> deurgarage	=> Open	(Pass2PHP)
2020-01-30 22:22:20.787  (SWITCH)		=>keuken=>On (functions.php:939)
2020-01-30 22:22:20.787  Status: User: Admin initiated a switch command (11/keuken/On)
2020-01-30 22:22:20.787  OpenZWave: Domoticz has send a Switch command! NodeID: 4 (0x04)
2020-01-30 22:22:20.788  (ZWAVE) Light/Switch (keuken)

Hi,
i know you are using an Intel Nuc 7 i3-based proxmox system. Previously 8GB then 24GB then 32GB ram. You also had an Sata SSD -> NVME switch.
My question would be, how did you configure your proxmox system, what did you use to achieve a response time of 15ms (pass2php 1ms)

1) Do you use a CT container or a separate VM?

2) Is Domoticz natively in the virtual system above or are you using a separate Docker?

3) Do you use log in Domoticz?

4) How much space did you give the Domoticz system? How much memory did you allocate to him, how many CPU cores?
You could put a picture of the hardwares, cpu settings in the proxmox.


5) You are using a Debian 10 buster. Is there another system that is even more minimal ( apt/deb based)?


I want to switch from Pi 3b+ to Proxmox and I want to maximize my system. I also have 1 VM / CT in the system. Have you written anything about memory usage so that if the size of the disk is the same as the size of the allocated ram, it will run on ram, what is that to be understood?

Thanks

[Egregius]

Hi,

You've read the topic quite good

I'm indeed running a i3 based Nuc with Proxmox as hypervisor. It has 24GB memory. That's mainly to let it run other VM's in test, like a Windows 10, Ubuntu, Elemantary OS etc. For Domoticz only you don't need that much.


For Domoticz I use a VM with Debian Buster. I choosed Debian because I also use that on several webservers I manage. Makes it easier to learn and handle. Mine is set to use 3.5GB RAM and all 4 cores. I mostly let the VM's use all cores because Proxmox does a great job in deviding the power when needed.

In top you see that 3.5 GB RAM is enough because Debian can keep more than 2GB of files in cache. All linux based systems like to fill the available RAM with cached files. For example my iMac often keeps more than 30GB in cache, just because he can.


On the hardware details I raised the CPU cycles to 8192. The default in Proxmox is 1000. Another VM wich runs pfSense is set to 16384.
That's also not needed when you only run domoticz on it. I mainly did it to avoid slowness when running other test VM's.


Besides that I didn't do much to achieve the 15 msec response time. It just came faster and faster with every change I made. I must admit that I didn't change much in the last several months. Everything runs as it should.

For the rest of the questions: Domoticz runs native in Debian, not in a docker. The logging in domoticz is disabled because I can see enough in the Pass2php logging. Even that is mostly disabled, I only enable it when I program something new.
There must be other distro's, don't know, never looked at it. I install Debian from the net install, that makes it pretty small to install. Don't use a full blown DVD to install.

[kispalsz]

Hi,

You've read the topic quite good

I'm indeed running a i3 based Nuc with Proxmox as hypervisor. It has 24GB memory. That's mainly to let it run other VM's in test, like a Windows 10, Ubuntu, Elemantary OS etc. For Domoticz only you don't need that much.


For Domoticz I use a VM with Debian Buster. I choosed Debian because I also use that on several webservers I manage. Makes it easier to learn and handle. Mine is set to use 3.5GB RAM and all 4 cores. I mostly let the VM's use all cores because Proxmox does a great job in deviding the power when needed.

In top you see that 3.5 GB RAM is enough because Debian can keep more than 2GB of files in cache. All linux based systems like to fill the available RAM with cached files. For example my iMac often keeps more than 30GB in cache, just because he can.


On the hardware details I raised the CPU cycles to 8192. The default in Proxmox is 1000. Another VM wich runs pfSense is set to 16384.
That's also not needed when you only run domoticz on it. I mainly did it to avoid slowness when running other test VM's.


Besides that I didn't do much to achieve the 15 msec response time. It just came faster and faster with every change I made. I must admit that I didn't change much in the last several months. Everything runs as it should.

For the rest of the questions: Domoticz runs native in Debian, not in a docker. The logging in domoticz is disabled because I can see enough in the Pass2php logging. Even that is mostly disabled, I only enable it when I program something new.
There must be other distro's, don't know, never looked at it. I install Debian from the net install, that makes it pretty small to install. Don't use a full blown DVD to install.

Wow!
Thanks for the detailed description. In the meantime, I still had a few questions.

1) Do you not use any swap partitions on proxmox?

2) You have turned on NUMA. Do I need to set something in Debian to work well, or is it automatic?

3) In TOP, I see that you only have 110 processes. Have you truncated the base debian buster yet, or does it only use that many tasks by default? (the newly installed minimal lubuntu 20.04.01 (graphical interface) runs 160 tasks by default) The proxmoxon I want either ubuntu server minimal or debian buster, or I'm still looking for an even smaller distro based on deb.

4)You may be able to copy your proxmox cpus info:

cat /sys/bus/cpu/devices/cpu0/cpufreq/scaling_cur_freq
cat /sys/bus/cpu/devices/cpu0/cpufreq/scaling_governor
cat /sys/bus/cpu/devices/cpu0/cpufreq/scaling_available_governors

5) If only domoticz and pfsense went, there was the noise of the series 7 i3 NUC. Would it be acceptable in the living room?


+1) one more "thing" question. Did you measure your proxmox consumption by how much your domoticz system ( or domoticz with pfsense) bites? How many watts. (without ubuntu, windows 10, etc.)

Thanks for your help.

[Egregius]

1) I disabled all swap in proxmox to save SSD writes. I found that my Kingston drive was already over half of it's lifetime. With less than 50% RAM used it shouldn't be needed after all.
2) NUMA makes it possible to add RAM without reboot. And should be better according to several sites when more than 1 core is used.
3) It's just the net install of Debian Buster, domoticz, Apache2 with php-fpm, mariaDB and fail2ban (I think )
4)

Code: Select all

root@proxmox:~# cat /sys/bus/cpu/devices/cpu0/cpufreq/scaling_cur_freq
2400265
root@proxmox:~# cat /sys/bus/cpu/devices/cpu0/cpufreq/scaling_governor
performance
root@proxmox:~# cat /sys/bus/cpu/devices/cpu0/cpufreq/scaling_available_governors
performance powersave

5) Perfectly silent. There's a small fan but I don't here it. My NUC is installed in the garage. I believe there are also fanless nuc's available.
6) Measured with a Fibaro Wall plug between 8W and 18W. 8W idle, 18W with heavy use of ipsec vpn tunnels in pfSense. That's with the Kingston SATA SSD and the Samsung NVME SSD together.

[kispalsz]

Please help me.
I’ve been struggling with this for days and I don’t understand why there’s no improvement.
I used RPI3 B + before. There mysql/mariadb version did not work ,the basic version only worked.

But I have not achieved the good times as you have achieved.

I switched systems now.

My current system is an Asrock Q1900DC-ITX motherboard ( j1900 CPU 4 core, 2 GHz), 2 x8 GB Dual Channel 1333 MHz, DDR3L ram. 1pc sata3 30 GB SSD. A Crucial mx500 250 GB SSD (sata) will arrive tomorrow.
I use now a Proxmox system, there is 1 Virtual Machine only in the system. The storage space of the VM (no swap) is 4 GB, the size of the allocated ram is 10 GB.
What were in the description were things I installed.
Debian 10 Buster minimal system
Domoticz
Apache 2
PHP 7.3
Mariadb 10.3.23

In Domoticz, I turned off Dzvents. I deleted all Dzvent rules. In the / home/user/domoticz/scripts/lua folder, only the there is a script_device_pass2php.lua file.
Contents:

Code: Select all

for d,s in pairs(devicechanged)
do
os.execute('curl -X POST -d "d='..d.."&s="..s..'" http://127.0.0.1/secure/pass2php.php &')
end

What I'm trying to do is turn on the electricity as fast as possible when there's movement.
My php file looks like this:

Code: Select all

<?php                                                                                                                                                                        
if ($status=='On'){
    sw('E-I-Kapcsolo-Jobb','On');

}

The problem is that even with the slow RPI 3B + device I achieved close to these values: .
Now with Proxmox:
549 -> 701 = 152 ms ( it's even "much" slower than the zwave association.)

Code: Select all

2020-10-12 15:48:57.549 (AEON) Light/Switch (E-I-Mozgas2-Home Security)
2020-10-12 15:48:57.562 (AEON) Light/Switch (E-I-Mozgas2-Sensor)
2020-10-12 15:48:57.701 OpenZWave: Domoticz has send a Switch command! NodeID: 10 (0x0a)

my top (VM):

Code: Select all

top - 16:21:53 up 17:27,  1 user,  load average: 0.25, 0.13, 0.04
Tasks: 102 total,   1 running, 101 sleeping,   0 stopped,   0 zombie
%Cpu(s):  1.5 us,  1.5 sy,  0.0 ni, 97.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
MiB Mem :   9994.3 total,   8578.6 free,    295.4 used,   1120.3 buff/cache
MiB Swap:      0.0 total,      0.0 free,      0.0 used.   9410.4 avail Mem 

How can i find out what the bottleneck is, what is misconfigured?
Proxmox configuration

vm_domoticz.jpg (17.26 KiB) Viewed 2640 times

I want to achieve a total response time of 15-30 ms


Thanks.