Page 1 of 2
Maximum of lua scripts?
Posted: Monday 29 March 2021 20:34
by JohnnySK
Raspberrypi 3,
latest stable release of Domoticz
I have 14 devices scripts, 6 time scripts and 3pcs of variable scripts and one of devices scripts work calculation with 10 devices (each 5seconds) .. and I'm getting delay in reactions of other lua scripts.
Is it any maximum of lua scripts?
Is it better write scripts in some other languages ? (because of performance)
Maybe python scripts running out of domoticz core and have better performance ??
thanks for help.
Re: Maximum of lua scripts?
Posted: Monday 29 March 2021 21:06
by waaren
JohnnySK wrote: ↑Monday 29 March 2021 20:34
I have 14 devices scripts, 6 time scripts and 3pcs of variable scripts and one of devices scripts work calculation with 10 devices (each 5seconds) .. and I'm getting delay in reactions of other lua scripts.
The Lua language interpreter by itself is very close to C and switching to another (script) language will not improve performance as such.
The way classic Lua scripts are implemented in domoticz might cause your issue.
Every device script is executed on every device update in the system and every time script is executed every minute and every variable script is executed every variable update, etc.
Every script trigger will receive a copy of the domoticz object with many sub-tables.
This is the reason why @Dannybloe and @jvandenbroek designed and integrated dzVents in domoticz. In dzVents only 1 Lua script is triggered for these events (device, time, variable, etc) and that single Lua scripts only loads the user scripts when needed.
This improves response times significantly, especially when the number of scripts rises.
You might want to have a look at
the dzVents wiki
Re: Maximum of lua scripts?
Posted: Monday 29 March 2021 23:31
by JohnnySK
I have onchange condition in each script.
When I last time enabled dzVents, Domoticz freezes few times in hour
I will try it again.
Thanks for now.
Re: Maximum of lua scripts?
Posted: Tuesday 30 March 2021 9:06
by waaren
JohnnySK wrote: ↑Monday 29 March 2021 23:31
I have onchange condition in each script.
That will not prevent the script from being triggered.
In my experience the script itself is hardly ever causing a response issue if you don't use blocking commands like os.execute or io.popen.
For these type of commands dzVents have async alternatives.
Re: Maximum of lua scripts?
Posted: Tuesday 30 March 2021 9:43
by JohnnySK
Yes,
maybe this is problem.. I have os.execute in one script.. I will move this one to the dzVents.
Re: Maximum of lua scripts?
Posted: Tuesday 30 March 2021 14:50
by jannl
Propably all the commandArray commands are causing the delay.
Do as suggested above, use dzVents, or create your own containerscript like I did, partly because dzVents did not exist at the time and partly because I better like the straight forward programming.
Jan
Re: Maximum of lua scripts?
Posted: Tuesday 30 March 2021 21:20
by manjh
I have a script that runs for about 30 seconds, and I fully understand why (this is OK, not an error).
In LUA it throws a warning that the script runs for more than 10 seconds.
So I decided to move the function to a dzVents script, hoping to lose the error. Not so, it gives me the same error message...
Is there a way around this, or should I learn to live with it?
Re: Maximum of lua scripts?
Posted: Tuesday 30 March 2021 21:39
by waaren
manjh wrote: ↑Tuesday 30 March 2021 21:20
Is there a way around this, or should I learn to live with it?
Best to share the script. That might help in understand why the 30 seconds are needed and to advise a possible work around (if any)
Re: Maximum of lua scripts?
Posted: Tuesday 30 March 2021 22:16
by jvdz
manjh wrote: ↑Tuesday 30 March 2021 21:20
I have a script that runs for about 30 seconds, and I fully understand why (this is OK, not an error).
In LUA it throws a warning that the script runs for more than 10 seconds.
Well, it isn't really OK as you are "hanging" the whole event system due to it being single threaded.
One should really run these script in an async way by either shelling it with os.execute("command &") or running it by crontab, but it can't obviously not have an commandArray returning any updates, so those would have to be done with JSON calls in the script.
Jos
Re: Maximum of lua scripts?
Posted: Tuesday 30 March 2021 22:25
by waaren
jvdz wrote: ↑Tuesday 30 March 2021 22:16
Well, it isn't really OK as you are "hanging" the whole event system due to it being single threaded.
One should really run these script in an async way by either shelling it with os.execute("command &") or running it by crontab
Or even better use dzVents async executeShellCommand() in combination with shellCommandResponses (available in domoticz >= build 12771)
Re: Maximum of lua scripts?
Posted: Wednesday 31 March 2021 0:32
by JohnnySK
waaren wrote: ↑Tuesday 30 March 2021 22:25
jvdz wrote: ↑Tuesday 30 March 2021 22:16
Well, it isn't really OK as you are "hanging" the whole event system due to it being single threaded.
One should really run these script in an async way by either shelling it with os.execute("command &") or running it by crontab
Or even better use dzVents async executeShellCommand() in combination with shellCommandResponses (available in domoticz >= build 12771)
build 12771 ... I must update to latest beta.. aaand when I last time tried it.. lot of energy devices doesn't have any value and part of house not worked anymore
so I reverted back to backup of release..
When I see issue list in latest beta.. it's little bit risky.. or not ??
Re: Maximum of lua scripts?
Posted: Wednesday 31 March 2021 10:35
by waaren
JohnnySK wrote: ↑Wednesday 31 March 2021 0:32
it's little bit risky.. or not ??
Risk can be mitigated by ensuring you have a recent and tested backup of your domoticz directory and subdirectory's (exclude the backups subdirectory)
Re: Maximum of lua scripts?
Posted: Wednesday 31 March 2021 21:36
by JohnnySK
Update looks good but I have this in log and I'm trying to search at forum but without success..
2021-03-31 21:17:40.579 Error: unable to remove file /home/pi/domoticz/scripts/dzVents/data/domscript1017.out
2021-03-31 21:17:40.580 Error: unable to remove file /home/pi/domoticz/scripts/dzVents/data/domscript1017.err
2021-03-31 21:19:28.442 Error: unable to remove file /home/pi/domoticz/scripts/dzVents/data/domscript1062.out
2021-03-31 21:19:28.442 Error: unable to remove file /home/pi/domoticz/scripts/dzVents/data/domscript1062.err
2021-03-31 21:23:40.904 Error: unable to remove file /home/pi/domoticz/scripts/dzVents/data/domscript1163.out
2021-03-31 21:23:40.904 Error: unable to remove file /home/pi/domoticz/scripts/dzVents/data/domscript1163.err
2021-03-31 21:24:17.896 Error: unable to remove file /home/pi/domoticz/scripts/dzVents/data/domscript1177.out
2021-03-31 21:24:17.896 Error: unable to remove file /home/pi/domoticz/scripts/dzVents/data/domscript1177.err
2021-03-31 21:28:18.573 Error: unable to remove file /home/pi/domoticz/scripts/dzVents/data/domscript1274.out
2021-03-31 21:28:18.573 Error: unable to remove file /home/pi/domoticz/scripts/dzVents/data/domscript1274.err
2021-03-31 21:30:13.902 Error: unable to remove file /home/pi/domoticz/scripts/dzVents/data/domscript1321.out
2021-03-31 21:30:13.902 Error: unable to remove file /home/pi/domoticz/scripts/dzVents/data/domscript1321.err
2021-03-31 21:32:11.882 Error: unable to remove file /home/pi/domoticz/scripts/dzVents/data/domscript1368.out
2021-03-31 21:32:11.882 Error: unable to remove file /home/pi/domoticz/scripts/dzVents/data/domscript1368.err
Re: Maximum of lua scripts?
Posted: Wednesday 31 March 2021 21:52
by waaren
JohnnySK wrote: ↑Wednesday 31 March 2021 21:36
Update looks good but I have this in log and I'm trying to search at forum but without success..
2021-03-31 21:17:40.579 Error: unable to remove file /home/pi/domoticz/scripts/dzVents/data/domscript1017.out
2021-03-31 21:17:40.580 Error: unable to remove file /home/pi/domoticz/scripts/dzVents/data/domscript1017.err
...
2021-03-31 21:32:11.882 Error: unable to remove file /home/pi/domoticz/scripts/dzVents/data/domscript1368.err
Seems to be an authorisation issue. These files should not be in /home/pi/domoticz/scripts/dzVents/data/ and can be removed from the command line using
Code: Select all
sudo rm /home/pi/domoticz/scripts/dzVents/data/*.out /home/pi/domoticz/scripts/dzVents/data/*.err
Re: Maximum of lua scripts?
Posted: Wednesday 31 March 2021 21:58
by JohnnySK
I tried change permissions on whole folder.
Files create domoticz and removing them too.. They are connected to my first dzVents skript where I'm using domoticz.executeShellCommand().
But I don't know why domoticz wont again remove already removed files.
Re: Maximum of lua scripts?
Posted: Wednesday 31 March 2021 22:05
by manjh
waaren wrote: ↑Tuesday 30 March 2021 21:39
manjh wrote: ↑Tuesday 30 March 2021 21:20
Is there a way around this, or should I learn to live with it?
Best to share the script. That might help in understand why the 30 seconds are needed and to advise a possible work around (if any)
Script is very simple. It starts a speedtest to measure line speeds.
Code: Select all
return {
on = {
devices = {
'Utrecht speedtest trigger'
}
},
execute = function(domoticz, device)
if (device.state =='On') then
print ("speedtest_device_dzvents starting speedtest measurement")
os.execute ("sudo bash /home/pi/domoticz/scripts/speedtest2.sh")
end
end
}
As you can see, there is no result flowing back from this script. The speedtest2.sh bash file stores results straight into virtual devices.
I tried to use this:
Code: Select all
domoticz.executeShellCommand('/home/pi/domoticz/scripts/speedtest2.sh')
but am getting an error:
Code: Select all
2021-03-31 22:28:41.852 Error: dzVents: Error: (3.0.2) An error occurred when calling event handler Speedtest_device
2021-03-31 22:28:41.852 Error: dzVents: Error: (3.0.2) ...z/scripts/dzVents/generated_scripts/Speedtest_device.lua:11: attempt to call a nil value (field 'executeShellCommand')
Re: Maximum of lua scripts?
Posted: Wednesday 31 March 2021 22:23
by JohnnySK
I have it similar
Code: Select all
return {
on = {
devices = {
'Batt Napatie'
}
},
execute = function(domoticz, device)
domoticz.executeShellCommand('/home/pi/domoticz/scripts/pv_fpv1.sh &')
domoticz.executeShellCommand('/home/pi/domoticz/scripts/pv_fpv2.sh &')
end
}
scripts have --silent parameter in each call.. so doesn't have output.. but files appear and disappear in data folder and errors still comming
Re: Maximum of lua scripts?
Posted: Wednesday 31 March 2021 22:41
by JohnnySK
I tried to use this:
Code: Select all
domoticz.executeShellCommand('/home/pi/domoticz/scripts/speedtest2.sh')
but am getting an error:
Code: Select all
2021-03-31 22:28:41.852 Error: dzVents: Error: (3.0.2) An error occurred when calling event handler Speedtest_device
2021-03-31 22:28:41.852 Error: dzVents: Error: (3.0.2) ...z/scripts/dzVents/generated_scripts/Speedtest_device.lua:11: attempt to call a nil value (field 'executeShellCommand')
Update domticz.. dzVents have executeShellCommand from 12771 build
Re: Maximum of lua scripts?
Posted: Wednesday 31 March 2021 22:56
by waaren
JohnnySK wrote: ↑Wednesday 31 March 2021 22:23
scripts have --silent parameter in each call.. so doesn't have output.. but files appear and disappear in data folder and errors still comming
You should not use the background operator (&) with domoticz.executeShellCommand()
Re: Maximum of lua scripts?
Posted: Wednesday 31 March 2021 23:01
by waaren
manjh wrote: ↑Wednesday 31 March 2021 22:05
As you can see, there is no result flowing back from this script. The speedtest2.sh bash file stores results straight into virtual devices.
when using os.execute you should use the background operator
Code: Select all
os.execute('/home/pi/domoticz/scripts/speedtest2.sh &')
I tried to use this:
Code: Select all
domoticz.executeShellCommand('/home/pi/domoticz/scripts/speedtest2.sh')
but am getting an error:
domoticz.executeShellCommand is available in dzVents >= 3.1