Iterate within a group Topic is solved

[MenzZana]

Hi all

This is my first post and I have currently made many scripts in LUA in domoticz for better automate my home.

One thing that I have not find any information is to iterate within a group in LUA.
Perhaps it is not possible?

I have a group called Group:Mysbelysning
which contains a number of switches.

In general I would like that if you light one of the switches included, all of them light up, whereas if you turn off one of the switches included, all
of them turn off.

The scripts works fine, but I would like to not make an array of switches in LUA, and then have that information in duplicate within the group.

So is it possible in LUA to read the name of the switches and add them to an array so I can read them using...

Code: Select all

    ncount=0
    for _,name in ipairs(sensorlist) do
        if (otherdevices[name]==status) then
            ncount=ncount+1
            end
        end
    return ncount

sensorlist={'switch1','switch2','switch3'}
which should be read from the Group

Cheers
MenzZana

[waaren]

One thing that I have not find any information is to iterate within a group in LUA.

Don't know if this is possible in classic Lua but can be done in dzVents. check this chapter in the wiki

[MenzZana]

Thanks!

I have not used dzVents that much but it seems I have to get better acquainted with it.

[rrozema]

I see that I never posted this script that does exactly what you need. It's one script that you can use to sync any number of groups of any number of switches. And you can simply configure which switches to group from the domoticz user interface. All you need to do is add a little json into the switches' descriptions field. All the switches that have the same group name will from that time on follow each other's lead: switch one in the group and the entire group follows.

I still have to put this script in github and start a separate thread to explain it, but here it is for you to use it already.

Code: Select all

-- This script will keep any number of groups of switches in sync. This is usefull in many 
-- scenarios.
-- For example: you have a switch at the bottom of the stairs that switches a light, but there is
-- no switch at the top of the stairs. Now if you add some wall switch with no wires to the light,
-- in the 2nd floor, you can use this script to make both switches control the light. You can even 
-- add more lights and another switch on for example the 3rd floor.
-- Another example: you have a wall switch next to the door, and you want to have all the lights
-- in the room to go on and off with this one wall switch and you don't feel like having wires to
-- all of these lights. Simply put a smart plug between the lights you want to switch off and on.
-- Then usew this script to have the wall switch control all of the smart plugs.

-- It is very easy to use: In the description for all switches that you want to form a "group",
-- add the following following text:
-- {
-- "sync_group": "group-name"
-- }
-- "group-name" must be the same for all switches that are in the same "group", the actual text can
-- be any valid json text value. So choose any name you like, but be careful with putting " in 
-- the group name. You can have multiple groups in your house, simply give each group a unique 
-- name and only the switches with that name will follow the group.

-- How to do it: 
-- Put below json text in the description field for 2 or more of your switches and all 
-- of these switches will switch on if one switch in the group switches on and off when some other 
-- switch switches off. A json is always enclosed by { and }, so these must be included and if you
-- need to add more json properties for a device, all but the last property must be followed by 
-- a ,.
-- {
-- "sync_group": "stair lights"
-- }
--
-- To show you can have more than one group of switches, now assume in the kitchen you create 
-- another group, for a cabinet light that you want to switch using a wall switch near the entrance
-- of the kitchen. Both switches get a json like below in their description. Note that the name 
-- after "sync_group" is different from that of the above example's. This makes that the kitchen
-- light won't go on and off with the stair light(s), even though they use this same script. Also 
-- note that this json has an additional property "some other property", which is ignored by this
-- script. This property could be used by some other script. It is in this example only to 
-- demonstrate the use of the , after the 1st property.
-- {
-- "sync_group": "kitchen cabinet",
-- "some other property": 10
-- }

return {
	on = {
		devices = {
			'*'
		}
	},
    execute = function(domoticz, device, triggerInfo)
		if (domoticz.EVENT_TYPE_DEVICE == triggerInfo.type) then
			if device.deviceType == "Light/Switch"
					and device.description ~= nil
					and device.description ~= ""
			then
				local description = device.description
				local state = device.state
				local ok, settings = pcall( domoticz.utils.fromJSON, device.description)
    	        if ok and settings ~= nil then
					local sync_group = settings.sync_group
					if sync_group ~= nil and sync_group ~= "" then
						domoticz.devices().forEach(
							function(otherdevice)
								if otherdevice.deviceType == "Light/Switch"
									and otherdevice.description ~= nil
									and otherdevice.state ~= state
								then
									local ok, othersettings = pcall( domoticz.utils.fromJSON, otherdevice.description)
									if ok and othersettings ~= nil then
										if othersettings.sync_group == sync_group then
											otherdevice.setState(state).silent()
										end
									end
								end
							end
						)
					end
			    end
		    end
        end
    end
}

[waaren]

Code: Select all

	on = { devices = { '*' } },

Please reconsider this part. The script will trigger on every device update in the system ! On my production system this averages to 3800 starts / hour.

[rrozema]

Code: Select all

	on = { devices = { '*' } },

Please reconsider this part. The script will trigger on every device update in the system ! On my production system this averages to 3800 starts / hour.

Triggering for every device is exactly what is intended here and what makes this script so powerful. I've got 426 devices at the moment and a rather chatty system. I'm absolutely sure that it fires many more times than 3800 times /hour in my system: 3800 is only a little bit more than 1 trigger per second. I've got many more than that going through with no problem at all: my rpi 3b only has max 10 - 15% cpu usage; including mqtt.

I have been thinking of optimising this script by splitting it into 2 parts: 1 reading the descriptions periodically on a timer and store the trigger devices in a data variable. The 2nd part with a function in the on-section that looks for the triggered device's name to be in the variable. My guess is that it will not be much faster and I didn't bother verifying this in the end because I've got no problem at all with performance. There is one advantage by using generic scripts: the number of scripts to check for at each device triggers stays low, so dzvents doesn't have to check so many on-sections. In my house I've got at least 8 groups defined, so instead of having 8 scripts doing pretty much the same, i've got only 1. Plus, having generic scripts have other advantages that I think are often underestimated. f.e. if I later on decide to optimise the script; that effort will then be beneficial for all my devices at once. Plus, I only need to test/debug this script once, I don't have another copy-paste including the incidental error for all of my groups.

Then again: If you don't like my script, don't use it. But if you've got a suggestion how to do something similar generic like this without triggering for every device change, I'm of course happy to learn!

[waaren]

Triggering for every device is exactly what is intended here and what makes this script so powerful.
I've got many more than that going through with no problem at all: my rpi 3b only has max 10 - 15% cpu usage; including mqtt.

It is not any computer hardware resource I am worried about. I asked for reconsideration of this part because of the way the event system works. As you know it is single threaded and if one script is executing other scripts are queued and need to wait. This will cause other scripts to become less responsive.

Then again: If you don't like my script, don't use it. But if you've got a suggestion how to do something similar generic like this without triggering for every device change, I'm of course happy to learn!

I like the concept of the script. My suggestion is to use an array of device names for the " on = " part so dzVents will only execute the script if one of the included devices is updated. It's a bit more maintenance for users when adding a device but in my opinion worthwhile.
The same array can also be used to iterate over the 'otherdevices' .

[rrozema]

My suggestion is to use an array of device names for the " on = " part so dzVents will only execute the script if one of the included devices is updated. It's a bit more maintenance for users when adding a device but in my opinion worthwhile.
The same array can also be used to iterate over the 'otherdevices' .

Do you known if it is possible to put a function in the devices = {} section, jsut like in the active = {} and in the timer = {} ? And if so, do we have access to the triggering device's properties in that function? If I can put a function in that devices = {} section that checks for the device to be a switch, that would narrow down the number of devices the script executes for (in my system thats 200 instead of 426 devices). All other devices, like sensor updates for example, will no longer trigger the script's execution.

Code: Select all

return {
	active = {

		function(domoticz)
			return true/false
		end
	}
	on = {
		-- device triggers
		devices = {
			-- scripts is executed if the device that was updated matches with one of these triggers
			'device name', -- device name
			'abc*', -- triggers for all devices which name begins with abc
			258, -- id
		},
		-- timer riggers
		timer = {
			function(domoticz)
				return true or false
			end
		},
	....
}

This partial (and modified) snippet from dzvents' "All (commented)" example shows the functions I know are available for active = {} and time= {}, the example however doesn't have such function in devices = { }. I'm wondering is it possible to put one in there too. And if so, what are the parameters: just the domoticz object like for these functions, or can I access the triggering device from inside this function too (I would suggest to make it read-only access only though)?

The optimal alternative approach would be to have a meta-data trigger in dzVents: This meta-data trigger gets executed when in Domoticz a device gets added, a device gets removed or a non-value-property on an existing device gets altered. Non-value properties are in this context all of the devices properties that do not trigger a current device trigger to fire. Using this meta-data trigger I can have the script monitor for changes in any device's meta data that causes a change in the list of devices that need to be (value-)triggered. This way I can keep the executions on value changes down to exactly those devices that have the configuration.

A poor man's version of this same approach is what I was tinkering about -but at the time didn't find it worth building-: include a time trigger that periodically executes to read all device's configurations and build a list of devices that need the script to execute. This however still requires the device = { function( domoticz, device) } or how else can I dynamically decide to execute the script for a device that is in the list and not for one that isn't in the list?

[rrozema]

Something like this:

Code: Select all

return {

	on = {
		devices = {
		    domoticz.data.deviceList    -- <-- of course this won't work, because domoticz is undefined here.
		                                -- but would there be an alternative way to achieve something similar?
		},
		timer = {
			'at 13:43'
		}
	},

	data = {
	    deviceList = { intial = {} }
	},

	execute = function(domoticz, triggeredItem, info)
	    if triggeredItem.isDevice then
		    domoticz.log('device ' .. triggeredItem.name .. ' was changed.', domoticz.LOG_FORCE)
		    -- todo: add whatever you want to do here.
		elseif trigeredItem.isTimer then
		    -- todo: replace this by whatever code you need to identify which devices should 
		    -- be in the list and put them into domoticz.data.deviceList.
		    domoticz.data.deviceList = { someDevice.name }
        end		  
	end
}

[rrozema]

Using device = { function( domoticz, device) } it would work like this:

Code: Select all

return {
	on = {
		devices = {
		    function ( domoticz, device) 
		        for _, v in pairs( domoticz.data.deviceList) do
		            if v == device.name then
		                return true
		            end
	            end
	            return false
		    end
		},
	},

	data = {
	    deviceList = { intial = {} }
	},

	execute = function(domoticz, triggeredItem, info)
	    if triggeredItem.isDevice then
		    domoticz.log('device ' .. triggeredItem.name .. ' was changed.', domoticz.LOG_FORCE)
		    -- todo: add whatever you want to do here.
		elseif trigeredItem.isTimer then
		    -- todo: replace this by whatever code you need to identify which devices should 
		    -- be in the list and put them into domoticz.data.deviceList.
		    domoticz.data.deviceList = { someDevice.name }
        end		  
	end
}

[waaren]

Do you known if it is possible to put a function in the devices = {} section, jsut like in the active = {} and in the timer = {} ? And if so, do we have access to the triggering device's properties in that function? If I can put a function in that devices = {} section that checks for the device to be a switch, that would narrow down the number of devices the script executes for (in my system thats 200 instead of 426 devices). All other devices, like sensor updates for example, will no longer trigger the script's execution.

No that is not possible.

The optimal alternative approach would be to have a meta-data trigger in dzVents: This meta-data trigger gets executed when in Domoticz a device gets added, a device gets removed or a non-value-property on an existing device gets altered. Non-value properties are in this context all of the devices properties that do not trigger a current device trigger to fire. Using this meta-data trigger I can have the script monitor for changes in any device's meta data that causes a change in the list of devices that need to be (value-)triggered. This way I can keep the executions on value changes down to exactly those devices that have the configuration.

That might be possible and is certainly worth to investigate further. dzVents already have System events for stop / start and backup. Maybe this can be extended to the meta data trigger you suggest. I am not very C++ savvy so will have to ask @MrHobbes74 and @dannybloe

A poor man's version of this same approach is what I was tinkering about -but at the time didn't find it worth building-: include a time trigger that periodically executes to read all device's configurations and build a list of devices that need the script to execute. This however still requires the device = { function( domoticz, device) } or how else can I dynamically decide to execute the script for a device that is in the list and not for one that isn't in the list?

You could use this mechanism to modify the current dzVents script in <domoticz>/scripts/dzVents/scripts.
replace line 1 of the script (before the return) with a table containing all devices to monitor

Code: Select all

deviceList = { 'deviceA', ' deviceB', 'deviceC' }

your devices = line should then be

Code: Select all

devices = deviceList,

[rrozema]

your devices = line should then be

Code: Select all

devices = deviceList,

Yes, I know that. It's just that I don't want to have to edit a script ever to indicate which devices should be controlled by this script. Someone can change the name of the device in the gui, risk of misspelling a device name when editing the script, risk of accidentally modifying the script code when I/someone adds or removes a device name and I can think of many more reasons to keep code and data separate. I just don't think names (or even worse: idx values) of devices should be in a script. I want my scripts to be this simple: copy the script code, paste it into the script window and run it. Set and forget: configuration is done in the gui.

I would love to get a meta data trigger. This will make creating many things more flexible. For example: it enables us to build logging of who made changes into any settings in Domoticz.

But what about adding the option to write on { device = { function( domoticz, device) ... end } } ? There of course have to be restrictions on what you can do inside this function body. Something like: the function should return true to have the execute function called or false to skip calling said execute() function. The function can have no side effects: You can not modify anything from within the function body. Not the device, no variables, no persistent data nor the domoticz environment itself can be modified from within the function body, nor can you issue any commands, write into files or make url calls. If the function does have a side effect, the results will be unpredictable and may be subject to change in any next version of dzvents.

[waaren]

But what about adding the option to write on { device = { function( domoticz, device) ... end } } ? There of course have to be restrictions on what you can do inside this function body. Something like: the function should return true to have the execute function called or false to skip calling said execute() function. The function can have no side effects: You can not modify anything from within the function body. Not the device, no variables, no persistent data nor the domoticz environment itself can be modified from within the function body, nor can you issue any commands, write into files or make url calls. If the function does have a side effect, the results will be unpredictable and may be subject to change in any next version of dzvents.

For now a function cannot be used in the devices = section.
You could use the active = section for that but I would not know how to restrict functionality in that function. Probably possible with Lua metatables but that is (far) beyond my Lua level

[dannybloe]

All these suggestions are far from trivial mostly due to performance issues. Doing a devices={ '*' } is a true performance killer as for each event (can be dozens per event-tick) all device objects get instantiated in dzVents and the execute function gets executed. So I truly don't recommend this at all. If it works for rrozema then fine but it's your call .

Also, why don't you just define some groups in Domoticz and use groups-triggers in your script. Perhaps with some metadata in your group description you can trigger connected groups. This way you can manage the groups entirely using the GUI.

And regarding the meta-data trigger that's being proposed... I don't think that is trivial to make. For your use-case you also need pre-change values in order to manage your list (how else would you update the list if a device gets a new index or id/title)? I wonder if that's possible at all as it may be managed by hardware plugins. But that I'm not sure of.

[rrozema]

All these suggestions are far from trivial mostly due to performance issues. Doing a devices={ '*' } is a true performance killer as for each event (can be dozens per event-tick) all device objects get instantiated in dzVents and the execute function gets executed. So I truly don't recommend this at all. If it works for rrozema then fine but it's your call .

Also, why don't you just define some groups in Domoticz and use groups-triggers in your script. Perhaps with some metadata in your group description you can trigger connected groups. This way you can manage the groups entirely using the GUI.

And regarding the meta-data trigger that's being proposed... I don't think that is trivial to make. For your use-case you also need pre-change values in order to manage your list (how else would you update the list if a device gets a new index or id/title)? I wonder if that's possible at all as it may be managed by hardware plugins. But that I'm not sure of.

I never said any of these things would be easy to implement, did I? And to be honest, I did at some time ask for some of the topics mentioned here, but I wasn't thinking I'd get any of them any time soon. It was only because of waaren's (rightfull!) warnings that we started pondering on what could be possible

I think you're right and we've gone far enough with this 'what-if' thinking. I think I'll leave it at this with just a small teaser to end with: having a meta-data-changed event could be of use in optimising this very performance issue in dzvents too: You could instantiate an object for every device in a 'domoticz-start'-event trigger, keeping all device objects in memory all the time. Then in a meta-data-changed-event trigger you'd only have to instantiate newly added device objects, update any that have been modified and remove any that have been deleted. Now when a value-changed event comes around, all device objects are ready to be used in memory and the on-checks plus any resulting execute() calls can be even more responsive.

[rrozema]

Out of curiosity I have rewritten the script such that it does as suggested: only trigger on devices that are actually in the list: I wanted to see if there is any difference in the response times for the switches themselves and/or my system in general.

So here is an alternative version of the script. Less user friendly, because you have to type the names for the devices exactly right, and much more error prone, because you are constantly changing the script when you want to make a configuration change. I've got it running now in my system, disabled the original script and I can't say that I notice any difference...

Code: Select all

-- This script will keep any number of groups of switches in sync. This is usefull in many 
-- scenarios.
-- For example: you have a switch at the bottom of the stairs that switches a light, but there is
-- no switch at the top of the stairs. Now if you add some wall switch with no wires to the light,
-- in the 2nd floor, you can use this script to make both switches control the light. You can even 
-- add more lights and another switch on for example the 3rd floor.
-- Another example: you have a wall switch next to the door, and you want to have all the lights
-- in the room to go on and off with this one wall switch and you don't feel like having wires to
-- all of these lights. Simply put a smart plug between the lights you want to switch off and on.
-- Then use this script to have the wall switch control all of the smart plugs.


local SETTINGS = {
    { "Angelique: Bedlamp", "Angelique: S2" },
    { "Joeri: S1", "Joeri: Ventilator 1", "Joeri: Ventilator 2" },
    { "Mischa: Ledstrip", "Mischa: S2" },
    { "Dimitri: Wandlamp", "Dimitri: S2"},
    { "Keuken: Aanrecht", "Keuken: S2" },
    { "Berging: Plafond", "Berging: Deur" }
}



-- A list of all unique devices mentioned in the settings (exlcuding any that are alone in a group).
local triggers = {}

-- A list of lists, containing all groups any unique device is in.
local groups = {}

for grpidx, grp in ipairs(SETTINGS) do
    if #grp > 1 then    -- If a group has only one device in it
                        -- then there are no other devices that
                        -- we need to switch.
        for _, name in ipairs(grp) do
    
            -- Build a table with all unique device names from
            -- any group. This is the list of triggers that we
            -- wait for.
            local j = 1
            while j <= #triggers and name > triggers[j] do
                j = j + 1
            end
            if j > #triggers then
                table.insert( triggers, j, name )
            elseif name ~= triggers[j] then     -- no duplicates
                table.insert( triggers, j, name )
            end
            
            -- For each unique device we also keep a list of the
            -- groups it is in. This way we can find what other 
            -- devices to switch. We don't want to check the enitre
            -- group twice if someone accidentaly enters the same 
            -- name twice in a group, so we filter out any 
            -- duplicates here.
            if nil == groups[name] then
                groups[name] = { grpidx }
            else
                local j = 1
                while j <= #groups[name] and grpidx > groups[name][j] do
                    j = j + 1
                end
                if j > #groups[name] then
                    table.insert( groups[name], j, grpidx )
                elseif grpidx ~= groups[name][j] then
                    table.insert( groups[name], j, grpidx )
                end
            end
        end
    end
end

return {
	on = {
		devices = triggers
	},
    execute = function(domoticz, device, triggerInfo)
		if domoticz.EVENT_TYPE_DEVICE == triggerInfo.type then
		    if nil ~= groups[device.name] then
		        for _, grpidx in ipairs(groups[device.name]) do
		            domoticz.devices().filter(SETTINGS[grpidx]).forEach(
						function(otherdevice)
						    if otherdevice.name ~= device.name and 
						       otherdevice.bState ~= device.bState then
							    if device.bState then
							        otherdevice.switchOn().silent()
							    else
							        otherdevice.switchOff().silent()
							    end
							end
						end
					)		            
		        end
		    end
        end
    end
}

[rrozema]

hmm, somehow the script can now get into an oscillating state, where all of the lights in a group keep going on and off. I can't explain that yet. other than than I'm now using .bState instead of .state I don't think I made any significant changes that could affect this. In fact, both the .silent() on the switchOn() and switchOff() and the check for different states should have made it impossible to get any oscillation.. And neither works apparently.