DzVents script suggestions please [Solved]

[HvdW]

Hi,
It's getting cold outside and we're doing our utmost to reduce gas consumption.
We're using P1 Smart Meter USB which shows daily usage.
In the current situation I can read our daily usage from today up till the current hour, but yesterdays'usage only displays teh end resut at 23:59.

I want to write/copy a script that can display the usage at a given time, or at the current time yesterday and the days before.
In other words: Is my daily gas consumtion at noon more or less than yesterday at noon.

Because I don't know where to start I'm asking for help.
Must be something with a Virtual Sensor displaying yesterday's data I guess.

[habahabahaba]

1. Dummy hardware - Virtual sensor - Custom sensor
2. making Script that running every hour (or how often you nedd) and taking data from Smart Meter and putting them to new sensor

[waltervl]

If you check your log page on your gasmeter device and you see the data of the last week per day including today.
I know exactly If we are above or below our normal figures if I look at the current usage of today.
Energy usage is only fair to compare with other days if you have similar temperatures. Therefore the degree days have been introduced. Also visible on the report page of a temperature device.

[waltervl]

Else you can look at this discussion for inspiration to get the data from yesterday. viewtopic.php?t=26915

[willemd]

By far the easiest would be:
1) extend the history on your device to 7 days
2) run below sql statement

Code: Select all

 select substring(date,1,10) dateonly,max(value)-min(value) from meter where devicerowid=2 and substr(date,12,2)<="01" group by dateonly;

whereby 2 is replaced by the idx of your device and "01" by the hour up to which you want to have the values included.

This will show for each available day the consumption up to and including the hour selected.

You can run the sql statement from sqlite3, or from a python program, or from dzVents.

[willemd]

Here is the complete DzVents script. You just need to put in the correct idx of your gas device on line 1.

Code: Select all

local gasDeviceIdx=2
return {
	on = {
		timer = {
			'every hour'  
		},
		shellCommandResponses =
        {
            'GASconsumption',
        },
	},
	logging = {
		level = domoticz.LOG_INFO,
		marker = 'GASconsumption',
	},
	execute = function(domoticz, item)
	    if item.isTimer then
    		local currentHour=os.date("%H",os.time())
    		local shellCommand='sqlite3 /home/pi/domoticz/domoticz.db \"select substring(date,1,10) dateonly,max(value)-min(value) from meter where devicerowid='..gasDeviceIdx..' and substr(date,12,2)<\\\"'..currentHour..'\\\" group by dateonly;\"'
    		domoticz.log('running shellCommand '.. shellCommand,domoticz.LOG_INFO)
    		domoticz.executeShellCommand({
                command = shellCommand,
                callback = 'GASconsumption',
                timeout = 5,
            })
        else 
        
            domoticz.log('******  cumulative consumption ***********', domoticz.LOG_INFO)
            domoticz.log("\n" .. item.data,domoticz.LOG_INFO)
            domoticz.log('*****************\n', domoticz.LOG_INFO)
        end
	end
}

[HvdW]

Thanks willemd.
I'll test it tomorrow.

[waltervl]

Using SQL is not the best idea to get this.
There is a historic function to get yesterday's value so no reason to do this ugly workaround.

Also sqlite3 needs to be installed on your system.

[HvdW]

OK thanks. @waltervl your warning of interfering in the domoticz.db sounds reasonable.

So I have read the link you gave and tested the URL string @waaren suggested over there as a start.
This string doesn't show a result.
So I started with this one:

Code: Select all

http://192.168.1.1:8080/json.htm?type=command&param=getdevices&rid=16

which gives me the actual value.
Question remains: How do I get yesterdays' value samen time same device.

Your remark about readings being not comparable because temperature outside and more circumstances may vary is correct but for my usercase that is not important right now.

[willemd]

OK thanks. @waltervl your warning of interfering in the domoticz.db sounds reasonable.

It is not really interfering. You are not changing anything. You are only retrieving values, just in a different way from using JSON.

Normally it would be better to use JSON to interface with the dbase to avoid problems when the dbase changes. In this case you are selecting the value field by date from a meter table, one of the core structures of Domoticz. Hardly expected to change.

And if you want to run JSON from a DzVents script, you will get a similar "ugly" structure, using a http response instead of a shell response.

Anyway, it is up to you .....

To get the hourly values using a JSON you can use

Code: Select all

http://xxx.xxxx.xxx.xxx:8080/json.htm?type=command&param=graph&sensor=counter&idx=IDX&range=day

and then you need to run a loop to process the hourly values into a cumulative total or use the "mu"field which seems close enough.

[waltervl]

Or use

Code: Select all

http://xxx.xxxx.xxx.xxx:8080/json.htm?type=command&param=graph&sensor=counter&idx=IDX&range=week

and you will get the summary values of the last 7 days (including today)

[HvdW]

@willemd
Those are the values I was looking for.
And you're right.

It is not really interfering. You are not changing anything. You are only retrieving values, just in a different way from using JSON.

Normally it would be better to use JSON to interface with the dbase to avoid problems when the dbase changes. In this case you are selecting the value field by date from a meter table, one of the core structures of Domoticz. Hardly expected to change.

And if you want to run JSON from a DzVents script, you will get a similar "ugly" structure, using a http response instead of a shell response.

Code: Select all

http://xxx.xxxx.xxx.xxx:8080/json.htm?type=command&param=graph&sensor=counter&idx=IDX&range=day

This gives me the information I wanted.

I'll go play with it and probably ask more questions in a few days or show a working script.

[HvdW]

I did like this:

Code: Select all

--Gas meten

-- Howto at Luftdaten https://www.domoticz.com/forum/viewtopic.php?f=72&t=23406&hilit=luftdaten
-- Call Gasmeter with http://192.168.1.1:8080/json.htm?type=devices&rid=16

return {
	active = true,
	on = {
		timer = { 'every 1 minutes' },           -- every minute for testing, 10 minutes when active
		httpResponses = { 'GasmeterRetrieved' }     -- matches callback string below
	},
	execute = function(domoticz, item)

		if (item.isTimer) then
			domoticz.openURL({
				-- url = 'http://192.168.1.1:8080/json.htm?type=devices&rid=16',
				-- url = 'http://192.168.1.1:8080/json.htm?type=command&param=setused&idx=58&switchtype=1&used=true',
				url = 'http://192.168.1.1:8080/json.htm?type=command&param=graph&sensor=counter&idx=16&range=day',
				method = 'GET',
				callback = 'GasmeterRetrieved'
			})

		elseif (item.isHTTPResponse) then
			if (item.ok and item.isJSON) then 
				domoticz.utils.dumpTable(item.json)   -- dumpTable laat alle waarden zien, mooi voor debugging
				domoticz.devices('Gas yesterday').updateSensor(item.json.result[1].CounterToday)
				domoticz.devices('Yestergas').updateSensor(item.json.result[1].Data)
			else
				-- oops
				domoticz.log('Error fetching Gasmeter data', domoticz.LOG_ERROR)
				domoticz.log(item.data, domoticz.LOG_ERROR)
			end
		end
	end
}

But now I am at a loss how to handle the Sensor update process.

Code: Select all

30 22:02:00.461 Status: dzVents: > mu: 10.308
2023-11-30 22:02:00.461 Status: dzVents: > d: 2023-11-30 21:00
2023-11-30 22:02:00.461 Status: dzVents: > 25:
2023-11-30 22:02:00.461 Status: dzVents: > v: 0.000
2023-11-30 22:02:00.461 Status: dzVents: > mu: 10.308
2023-11-30 22:02:00.461 Status: dzVents: > d: 2023-11-30 22:00
2023-11-30 22:02:00.461 Status: dzVents: > ValueQuantity:
2023-11-30 22:02:00.461 Status: dzVents: > CostEnergyR1: 800
2023-11-30 22:02:00.461 Status: dzVents: > ValueUnits:
2023-11-30 22:02:00.461 Status: dzVents: > DividerWater: 100.0
2023-11-30 22:02:00.461 Status: dzVents: > CostEnergyR2: 800
2023-11-30 22:02:00.461 Status: dzVents: > CostWater: 16473
2023-11-30 22:02:00.473 Status: dzVents: Info: ------ Finished GasCalc
2023-11-30 22:02:00.473 Error: dzVents: Error: (3.1.8) An error occurred when calling event handler GasCalc
2023-11-30 22:02:00.473 Error: dzVents: Error: (3.1.8) ...n/domoticz/scripts/dzVents/generated_scripts/GasCalc.lua:26: attempt to call a nil value (field 'updateSensor') 

I created a Gas dummy named: Gas yesterday and a text dummy named Yestergas.

Let's say I want to fetch value 1: like 24 hours before.
How do I update ?

Remark: value 1: to 25: is never displayed, it starts with 3: or 5: and goes up and up like from 33: to 57: and onward.
The url put in a browser always starts at 0:
So why is it augmenting in a script.

First success: I am able to update the text dummy Yestergas with this one:

Code: Select all

domoticz.devices('Yestergas').updateText(item.json.result[44].mu)

Maybe write some code to find the highest value and substract 24 and substitute that for the 44 in the example above?

EDIT
Suggestions for another type of dummy than text?

[habahabahaba]

Can you show the result of this request:

Code: Select all

http://192.168.1.1:8080/json.htm?type=command&param=graph&sensor=counter&idx=16&range=day

Your log is saying that item.json.result[1].Data is empty

[HvdW]

The log shows data as well as the json does

json1.jpg (22.05 KiB) Viewed 1967 times

json2.jpg (27.3 KiB) Viewed 1967 times

And the log:

log.jpg (137.04 KiB) Viewed 1967 times

It's working, however I was only able to catch it in a text file and I want to automate it to give me the value of 24 hours before now.
To be more precise: I'd like to catch the Gas consumtion at noon yesterday.

Code: Select all

2023-12-01 10:24:00.292 Status: dzVents: > 27:
2023-12-01 10:24:00.292 Status: dzVents: > d: 2023-11-30 12:00
2023-12-01 10:24:00.292 Status: dzVents: > mu: 3.528
2023-12-01 10:24:00.292 Status: dzVents: > v: 0.582

The mu that is connected with d: 2023-11-30 12:00 or maybe better d: 2023-11-30 12:00 just the d: and 12:00

[willemd]

Please check the Dzvents documentation to select the correct update function for your type of device:
https://www.domoticz.com/wiki/DzVents:_ ... ipting#Gas
If it is a GAS device then it is updateGas()

The first hour in the list will be the currentHour X days ago, X being the number of days in your history.
So yesterdays value of the previous complete hour will be (X-1)*24
CurrentHour yesterday will be (X-1)*24+1
etc.

To pick out the mu value from the response you can use
item.json.result[(X-1)*24].mu

Just to add: if you want to make it more error-proof in case your history is not complete for example, you could read the "d" value first, check the date and hour, and only then get the corresponding "mu" value. The above (X-1)*24 calculation assumes your history is complete, which might not always be the case.

[HvdW]

Thanks @willemd
The updateGas tip is what I needed.

Next problems
Here is the code:

Code: Select all

domoticz.devices('Gas yesterday').updateGas(item.json.result[27].mu)				domoticz.devices('Yestergas').updateText(item.json.result[27].mu)

and the output:

yestergas.jpg (50.85 KiB) Viewed 1928 times

So the gas seems to have a need to be multiplied by 1000

Nice outcome: The list always seems to be ending at #49, so asking the #25 outcome should be ok.

[willemd]

So the gas seems to have a need to be multiplied by 1000

Nice outcome: The list always seems to be ending at #49, so asking the #25 outcome should be ok.

It is in the documentation:
updateGas(usage): Function. Usage in dm3 (liters).

#49 because you have your history setting at 2 days.
#25 will give you the current hour yesterday, which will be a full hour, so comparing this to the current hour total only makes sense at the end of the current hour.

[HvdW]

I want to write/copy a script that can display the usage at a given time, or at the current time yesterday and the days before.
In other words: Is my daily gas consumtion at noon more or less than yesterday at noon.

Found the easy way out.
What I wanted to achieve is to know yesterdays values and display those.
Main question: what was the gas consumption at noon yesterday.
My simple approach now is to check and update every day at noon + 5 or so minutes because gas updates some minutes after the hour.
Here is the script:

Code: Select all

return {
	on = {
		timer = {'at 12:08'},
	},

	execute = function(domoticz, triggeredItem)
	    local gasconsumptiontoday = domoticz.devices('Gas').counterToday
	    domoticz.devices('Gas yesterday noon').updateGas((gasconsumptiontoday*1000))
	    domoticz.notify('Gasmeter ' .. gasconsumptiontoday, domoticz.PRIORITY_HIGH)     
	end
}

or even more concise:

Code: Select all

return {
	on = {
		timer = {'at 12:08'},
	},

	execute = function(domoticz, triggeredItem)
	    domoticz.devices('Gas yesterday noon').updateGas(domoticz.devices('Gas').counterToday*1000)
	end
}

[willemd]

Indeed an easy way out, but much more limited functionality than initially requested "usage at a given time" of course.
I think what you will find out over time is that waltervl was right: comparison makes more sense when you include degree day calculation.
And even with degree day calculation it is inaccurate. For example you might want to factor in the amount of sunshine since that might be heating up your house significantly if you have large windows, even if temperature from one day to the other is the same.