Introducing dzVents - Domoticz Lua file based event scripting made e-z!

[dannybloe]

Hello folks,

After spending a lot of hours in the past two weeks trying to enhance the Lua scripting experience for Domoticz I am happy to announce dzVents on GitHub. I have been truly enjoying making this enhancement and I am glad to share it to you peeps. Maybe it makes your scripting live a lot easier as well. (Note that it is still beta, so please be patient a little here, and help out perhaps).

This is the excerpt from GitHub:

About

dzVents (|diː ziː vɛnts| short for Domotiz Easy Events) brings Lua scripting in Domoticz to whole new level. Writing scripts for Domoticz has never been so easy. Not only can you define triggers more easily, and have full control over timer-based scripts with extensive scheduling support, dzVents presents you with an easy to use API to all necessary information in Domoticz. No longer do you have to combine all kinds of information given to you by Domoticzs in many different data tables. You don't have to construct complex commandArrays anymore. dzVents encapsulates all the Domoticz peculiarities regarding controlling and querying your devices. And on top of that, script performance has increased a lot if you have many scripts because Domoticz will fetch all device information only once for all your device scripts and timer scripts.

Let's start with an example. Let's say you have a switch that when activated, it should activate another switch but only if the room temperature is above a certain level. And when done, it should send a notification. This is how it looks like in dzVents:

Code: Select all

return {
    active = true,  
    on = {
        'Room switch'
    },

    execute = function(domoticz, roomSwitch)

        if (roomSwitch.state == 'On' and domoticz.devices['Living room'].temperature > 18) then
            domoticz.devices['Another switch'].switchOn()
            domoticz.notify('This rocks!', 
                            'Turns out that it is getting warm here', 
                            domoticz.PRIORITY_LOW)
        end

    end
}

Or you have a timer script that should be executed every 10 minutes but only on weekdays and have it do something with some user variables and only during daytime:

Code: Select all

return {
    active = true, 
    on = {
        ['timer'] = {'Every 10 minutes on mon,tue,wed,thu,fri'}
    },

    execute = function(domoticz)

        -- check time of the day
        if (domoticz.time.isDayTime and domoticz.variables['myVar'].nValue == 10) then
            domoticz.variables['anotherVar'].set(15)
            --activate my scene
            domoticz.setScene('Evening lights', 'On')
            if (domoticz.devices['My PIR'].lastUpdate.minutesAgo > 5) then
                domoticz.devices['Bathroom lights'].switchOff()
            end
        end         

    end
}

Just to give you an idea! Everything that was previously scattered around in a dozen Lua tables is now logically available in the domoticz object structure. From there you can get to all the information and you can control the devices.
....

This is just a sneek preview. And there is a lot more to tell so please read along on GitHub and let me know what you think. I have this running for a couple of days now without any problems so far and my event scripts have become almost trivial now and a lot smaller.

Oh, btw, people who tried my earlier versions may have to change their execute function a bit now that the domoticz object and device objects are passed along. Sorry for that.

Cheers,

Danny

[jmleglise]

whaouu. Looks awsome

[remb0]

Can we push this into the domoticz branch or has it other effects?

[dannybloe]

Not that I know of. Other than some more testing here and there I don't think it will bite whatever Domoticz already ships.

[simonrg]

Great work, then assuming I haven't misunderstood how things work (and I may be ), could I suggest a slightly more pragmatic, less consistent but easier approach.

Basically to keep all the trigger logic in script_device_main and script_device_time, stick to the current file naming and use your helper functions to give the added flexibility. The individual scripts could either duplicate the trigger logic, which would have a minimal performance impact, but allow backward compatibility or could just not bother if it was a new script.

If I have got the usage right then basically I don't alter script_device_main.lua and script_device_time.lua, but just add my altered scripts into the ~/domoticz/scripts/lua/scripts sub-directory, which have the trigger logic.

So am I right to say that all the logic for when the scripts will execute is contained in the individual scripts and so to understand which script is executed when, then I have to open each script.

The great advantage I have found from calling all my scripts from a single script is that I have list of all the script triggers in one place.

By sticking to the standard script naming, it is very easy to test an individual script by moving it from the sub-directory to the main Lua directory and then Domoticz stops it again.

Computationally with the logic in script_?????_main.lua, this would mean that script_?????_main.lua would only load the scripts which are active and not all of them. I appreciate the major time saving is by not recreating the Domoticz specific tables rather than in file loading, but every little helps.

If you move your logic from the individual scripts to the main scripts then you will have all the advantages plus a single place to understand when scripts are called and don't need to modify existing functioning scripts.

P.S. Looking at script_device_main.lua

Code: Select all

devicechanged['*'] = ''  -- trigger for * events

if (devicechanged~=nil) then

by setting devicechanged['*']='' doesn't this mean that devicechanged will never be nil and so all the scripts will be read in, even if devicechanged was previously nil.

[four2six]

this all sounds really good. thank you very much!

Wouldn't there be a way to add this into a section like the blockly scripts, just with showing the script text in the middle instead of the blocks? I personally don't mind nano-ing scripts, but i think it would make a great feature, increase the overview which scripts are activated and which are not, and increase the overall acceptance of using lua scripts.

[dannybloe]

Well, I think that with dzVents you can easily have full and foremost easy control about which script should be active or not. Of course, right out of the box you have to set the 'active' key in each script file so yeah, that means that you have to open each file to see whether it is active or not.

However, and maybe you didn't realize this enough, the active key can be a function. So, one of the ways to make this a lot easier than juggling with files in and out of a folder or renaming them by added _demo to the file is this:

• In Domoticz, create a Room and call it something like Scripts.
• For each script you have, create a Dummy switch and give it the name of the script
• Place each of these switches on the new Room (see docs for making rooms and room plans)
• Then, in each script, change the active to something like this:

Code: Select all

return {
	['active'] = function(domoticz) return (domoticz.devices['this_scriptname'].state=='On') end,
	...
}

That's all. Now you can just press the switch in Domoticz and control whether it is active or not. Or.. you could use uservariables for this and change it like this:

Code: Select all

return {
	['active'] = function(domoticz) return (domoticz.variables['this_scriptname'].nValue==1) end,
	...
}

Then you don't have a switch but a bunch of variables controlling the script. You could even have a variable or switch called something like 'Timer events active' or 'Device events active' and inside the script_time_main.lua and script_device_main.lua, just after the initialization of the domoticz variable put this test:

Code: Select all

local Domoticz = require('Domoticz')
local domoticz = Domoticz()
if (domoticz.devices['Run timer scripts'].state == 'Off') then
	return {} -- empty commandArray
end	

So, with dummy switches you can create a nice admin panel with buttons for all your event scripts. Pretty nice and hardly any work I'd say .

Computationally with the logic in script_?????_main.lua, this would mean that script_?????_main.lua would only load the scripts which are active and not all of them. I appreciate the major time saving is by not recreating the Domoticz specific tables rather than in file loading, but every little helps.

If you move your logic from the individual scripts to the main scripts then you will have all the advantages plus a single place to understand when scripts are called and don't need to modify existing functioning scripts.

It is not only about which script is active and which is not. It is also about the trigger definitions. A script can be triggered by more than one device event. I have several of them that respond to many different switches. If you name you scripts right then it is easy to see which script is responsible for what.

In an earlier implementation I had a binding file that controlled which script is active and what the triggers are. For some use cases that is easier maybe but I think that it is more logical all the logic remains within the script file itself. But maybe that's just a matter of taste. If there is enough need for it I could easily make a combination where perhaps there is a file that overrides the active states from the script files. But then again, I think is is much easier with the approach that I described above.

And the loading of the modules is really fast. The only thing that is always executed for every module is the 'active' part. That is why you shouldn't do much computation in the active function (but checking for a device is pretty fast so that's not a problem).

And yeah, you are correct about:

Code: Select all

devicechanged['*'] = ''  -- trigger for * events
if (devicechanged~=nil) then

The if statement doesn't make any sense. However,

and so all the scripts will be read in, even if devicechanged was previously nil.

Is not true. All the scripts are loaded earlier, in here:

Code: Select all

local eventBindings = helpers.getEventBindings(domoticz)

Only the scripts are executed when there is a device event for it. A significant difference. Loading is fast, executing is slow. That's why dzVents only executes the scripts that are relevant.

And of course it's not all about performance anymore with dzVents. The ease of defining triggers (declarative) and manipulating and querying the domoticz state makes the scripts smaller, much easier to read and better maintainable. I believe

Oh, and another thing: copy these lines to any other script you already have:

Code: Select all

local scriptPath = debug.getinfo(1).source:match("@?(.*/)")
package.path    = package.path .. ';' .. scriptPath .. '?.lua'
local Domoticz = require('Domoticz')
local domoticz = Domoticz()

-- your script stuff .....


commandArray = domoticz.commandArray
return commandArray

And you have your domoticz object in your traditional scripts. No need to use the trigger/binding/dispatching stuff of dzVents if you don't like it. But you can still do everything with the domoticz object!

[simonrg]

Thanks for the very detailed response to my "not invented here" post

I guess the key point I hadn't really got was that dzVents is not really about taking your existing scripts without modification,

And of course it's not all about performance anymore with dzVents. The ease of defining triggers (declarative) and manipulating and querying the domoticz state makes the scripts smaller, much easier to read and better maintainable. I believe

but simplifying them and incorporating dvVents features into the individual scripts.

Your scripting arrangement sounds complex to take full advantage of dzVents. I guess until I try dzVents with my scripts, I won't get a proper picture as to how it all fits together.

[simonrg]

Any idea why this is all that happens when I implement dzVents?

Code: Select all

 2016-03-04 00:20:00.205 LUA: Handle timer events
2016-03-04 00:20:15.631 LUA: Cannot find a device by the event name CPU_Usage
2016-03-04 00:20:15.632 LUA: Cannot find a device by the event name CPU_Usage_Utility 

I have only one very simple dzVents script in the scripts sub-directory:

Code: Select all

-- ~/domoticz/scripts/lua/scripts/first.lua

return {
        active = false,                  -- set to true to activate this script
        on = {
                'Owl',                 -- name of the device
--              'My sensor_Temperature',     -- better not use but check device.attributeIsChanged('temperature')
--              'My sensor',
--              258,                         -- index of the device
--              ['timer'] = 'every minute',  -- see readme for more options and schedules
--              '*',                         -- script is always executed no matter which device is updated
        },

        execute = function(domoticz, device) -- see readme for what you get
        -- see readme for the entire domoticz object tree
        -- mySwitch is a Device object with all the properties of the device that was updated
        -- unless this is a timer script, then there is not second parameter to this execute function
print('Inside first.lua')
        if (device.state == 'On') then
                domoticz.notify('Hey!', 'I am on!', domoticz.PRIORITY_NORMAL)
        end
        end
}

[dannybloe]

Any idea why this is all that happens when I implement dzVents?

Code: Select all

 2016-03-04 00:20:00.205 LUA: Handle timer events
2016-03-04 00:20:15.631 LUA: Cannot find a device by the event name CPU_Usage
2016-03-04 00:20:15.632 LUA: Cannot find a device by the event name CPU_Usage_Utility 

I have only one very simple dzVents script in the scripts sub-directory:

Code: Select all

-- ~/domoticz/scripts/lua/scripts/first.lua
[/quote]

I guess it is because I assumed that device names by themselves don't have an _ (underscore) in their names  :D 
Whenever an event occurs on a sensor value then Domoticz creates an entry in devicechanged table like 'My sensor_Temperature'. What I do is split the name to get to the device name by splitting around the underscore. I will have to change that and start from the end and then search for the underscore. Sorry about that ;-)

You could try for now by removing the _ from the device name until I have fixed it.

[dannybloe]

Thanks for the very detailed response to my "not invented here" post

Your scripting arrangement sounds complex to take full advantage of dzVents. I guess until I try dzVents with my scripts, I won't get a proper picture as to how it all fits together.

Nah.. it's not that complex. I have two complex scripts, one for controlling humidity in the bathroom based on humidity changes and one for controlling a room temperature in combi with a boiler. The rest is pretty straight forward. (Don't tell anyone )

I just like neat code that can be read like sentences in my event scripts.

[thorbj]

Hi, this looks very interesting.
I'm trying to figure this out, and have startet to move the following script:

Code: Select all

-- ~/domoticz/scripts/lua/script_device_watchestv.lua

commandArray = {}

-- SETTINGS --
local mode_var = 'LastMode'
local low_lux = 'Low Lux'
local motion_switch = 'Bevegelse Stue' -- name of the motion switch
local lamp_switch = 'Dimmer stue sør' -- name of a lamp that this script should depend on
local media_device = 'Chromecast' -- name of a media device like Chromecast or TV
local override_switch = 'Overstyr' -- Switch for activating system override
local night_switch = 'Nattmodus'
-- END SETTINGS --

-- Define hours for day and evening lights
h = tonumber((os.date('%H')))
if     (h >= 8 and h < 20)
   then
   x = 'Scene:Stue PÅ dag'
   y = '<font color="green">Stue P&Aring; <b>dag</b> aktivert</font>'
   else
   x = 'Scene:Filmmodus'
   y = '<font color="green">Filmmodus (belysning)</font>'
end

-- Issue command "x" if lux is below max_lumen and media_device is switched on.
if    (otherdevices[low_lux] == 'On' and devicechanged[media_device] == 'On' and otherdevices[override_switch] == 'Off' and otherdevices[night_switch] == 'Off') then   
   print('<font color="#CC6600">Last mode: ' ..uservariables[mode_var].. ' ...</font>')
   commandArray[x]='On'
   commandArray['Variable:LastMode'] = x
   print(y)
end
return commandArray

This is what I have so far:

Code: Select all

return {
	active = false,                  -- set to true to activate this script
	on = {
		'Low Lux',
		'Chromecast',
	},
	off = {
		'Overstyr',
		'Nattmodus',
	},

	execute = function(domoticz, mySwitch)


	if (mySwitch.state == 'On') then
		domoticz.setScene('Stue PÅ dag', 'On')
	end
	end
}

It this the right approach when I want to issue a command based on some switches that si on and some that is off?
How can I define the times for the appropriate scene - between 8am and 8pm, then day scene, else night scene?

Thanks!

[dannybloe]

Hi, this looks very interesting.

This is what I have so far:

Code: Select all

return {
	active = false,                  -- set to true to activate this script
	on = {
		'Low Lux',
		'Chromecast',
	},
	off = {
		'Overstyr',
		'Nattmodus',
	},

	execute = function(domoticz, mySwitch)


	if (mySwitch.state == 'On') then
		domoticz.setScene('Stue PÅ dag', 'On')
	end
	end
}

It this the right approach when I want to issue a command based on some switches that si on and some that is off?
Thanks!

Almost. Maybe it is a bit confusing in this domain of devices that can be on and off but the on in the return object/module means something like: 'on a change execute blah' and not so much the state of a device. Get it?

So you should do it like this:

Code: Select all

return {
	active = false,                  -- set to true to activate this script
	on = {
		'Low Lux',
		'Chromecast',
		'Overstyr',
		'Nattmodus',
	},

	execute = function(domoticz, mySwitch)


		if (mySwitch.state == 'On') then
			domoticz.setScene('Stue PÅ dag', 'On')
		end
		
	end
}

So the 'on' key should be a list of devices that, when changed ('on a change') will trigger the execute method. And in this example, when the execute method is called, mySwitch is the device that was actually changed. In this example I called it mySwitch but it can be any name that has a meaning to you. Could be just 'device' or 'changedDevice' or 'doorSensor' if the script is about door sensors that are changed.

So, just to make things clear: my advise is to have one script deal with only one kind of (related) event(s). So for instance, I have a couple of floating sensors in my roof gutters that switch on if the water level is too high (dirt blocking the pipes). All these sensors have a dummy switch in Domoticz and I have on script that deals with them:

Code: Select all

return {
	on = {
		250, 251, 252, 253 -- gutter sensors
	},
	active = true,
	execute = function(domoticz, sensor)
		if (sensor.state == 'On') then
			domoticz.notify('Gutter is clogged', sensor.name,  domoticz.PRIORITY_EMERGENCY)
		else
			print('Nothing to take care off, all clear again')
		end
	end
}

That's all there is in this script file (called gutter_sensors.lua).

How can I define the times for the appropriate scene - between 8am and 8pm, then day scene, else night scene?

What you can do is this:

Code: Select all

		if (domoticz.time.isDayTime) then
			domoticz.setScene('my scene', 'On')
		else
			domoticz.setScene('my scene', 'Off')
		end

[tonadam]

Thanks, this looks very cool. I just started scripting, so experimented a bit.

2 observations so far.

Simple script to make device 'ballen' a slave of device 'ster', with a delay:

return {
active = true,
on = {
'Ster'
},

execute = function(domoticz, device)
if (device.state == 'On') then
domoticz.devices['Ballen'].switchOn('AFTER 600')
end
if (device.state == 'Off') then
domoticz.devices['Ballen'].switchOff('AFTER 600')
end
end
}

This works, however the delay (after 600) doesn't seems to be honored, 'ballen' is switched on/of immediately when 'ster' changes state.

I also experimented with a HUE lamp. For the HUE lamp an event is triggered when the light is switched on/off via Domotics. However, when I switch from the HUE App, thus talking directly to the HUE controller, no event is generated. Domotics does know the light state has changed, since it polls the HUE controller regularly, this is also visible in the dashboard, the state is updated there fairly quickly.

Thanks:)

~

[dannybloe]

This works, however the delay (after 600) doesn't seems to be honored, 'ballen' is switched on/of immediately when 'ster' changes state.

I also experimented with a HUE lamp. For the HUE lamp an event is triggered when the light is switched on/off via Domotics. However, when I switch from the HUE App, thus talking directly to the HUE controller, no event is generated. Domotics does know the light state has changed, since it polls the HUE controller regularly, this is also visible in the dashboard, the state is updated there fairly quickly.

Thanks:)

~

Hiya tonadam. I must be honest that I haven't tested the AFTER options fully. I will and see what's wrong. Probably an easy fix.

Regarding the HUE lamp problems: in the script script_device_main.lua there is a commented line at line 23: --print('event ' .. event). Could you remove the comment and look in the log what is printed when you change the HUE lamp? Maybe domotics doesn't fire the script or it is with a name that is different than expected.

[tonadam]

Hiya tonadam. I must be honest that I haven't tested the AFTER options fully. I will and see what's wrong. Probably an easy fix.

Regarding the HUE lamp problems: in the script script_device_main.lua there is a commented line at line 23: --print('event ' .. event). Could you remove the comment and look in the log what is printed when you change the HUE lamp? Maybe domotics doesn't fire the script or it is with a name that is different than expected.
~

Hmm and then it starts working...

2016-03-05 12:44:45.094 LUA: event *
2016-03-05 12:44:45.615 LUA: event Keuken
2016-03-05 12:44:45.615 LUA: event *
2016-03-05 12:44:45.608 (Hue) Lighting Limitless/Applamp (Keuken)
2016-03-05 12:44:45.620 LUA: event Woonkamer
2016-03-05 12:44:45.620 LUA: Handling events for: "Woonkamer", value: "On"
2016-03-05 12:44:45.620 LUA: =====================================================
2016-03-05 12:44:45.620 LUA: >>> Handler: ster
2016-03-05 12:44:45.620 LUA: >>> Device: "Woonkamer" Index: 3
2016-03-05 12:44:45.620 LUA: .....................................................
2016-03-05 12:44:45.620 LUA: .....................................................
2016-03-05 12:44:45.620 LUA: <<< Done
2016-03-05 12:44:45.620 LUA: -----------------------------------------------------
2016-03-05 12:44:45.620 LUA: event *
2016-03-05 12:44:45.620 LUA: [1] = Ster: On
2016-03-05 12:44:45.620 LUA: =====================================================
2016-03-05 12:44:45.620 EventSystem: Script event triggered: /thuis/domotica/domoticz/scripts/lua/script_device_main.lua
2016-03-05 12:44:45.616 (Hue) Lighting Limitless/Applamp (Woonkamer)
2016-03-05 12:44:45.620 (Hue) Lighting Limitless/Applamp (Group Untitled Group)
2016-03-05 12:44:45.621 (Hue) Lighting Limitless/Applamp (Group Dimmer 2)
2016-03-05 12:44:46.401 RFLink Sending: 10;FA500;008fc3;1;ON


I did upgrade to Domiticz v3.4858 before trying this. Not sure what fixed it. But it's working now Thanks!

[dannybloe]

Oh, ok that's cool.
I'll look into that other problem a bit later.

[commodore white]

Still struggling to get something meaningfull for my alarm system but alread stumbled into an issue I raised a cuppla years ago. It a shame domoticz doesnt expose device types then I could have a script that simply filters on devicetype = "Motion Sensor", "Doorbell", or somesuch.

After all, your recommendation is that its best that scripts handle events from similar classes of device but offers no way of codifying that. Not your fault though.

Peter (too tired to be making any sence, sorry)

[Westcott]

HI Commodore White,
There is a sort of way to do that in Lua, using a JSON call.
It's basically a dump of the devices table, so you can do your own filtering.

[simonrg]

The thing to do would be add this information to the device object in the table that dzVents creates. This would allow filtering in the spirit of dzVents.

To reduce the number of device dumps which are needed from Domoticz, it would make sense to have a standard timer script that runs every minute, which would do the json call and save the complete device object table to a file, which would be read in each time a script is executed to create the device object, with some items being updated from otherdevices / devicechanged tables to keep it current.