How is value for kWh for specific month calculated?

Topics (not sure which fora)
when not sure where to post, post here and mods will move it to right forum.

Moderators: leecollings, remb0

Post Reply
freekdk
Posts: 13
Joined: Monday 01 February 2021 16:33
Target OS: Linux
Domoticz version: V2024.7
Location: Netherlands
Contact:

How is value for kWh for specific month calculated?

Post by freekdk »

I have normal values in august this year for a set of PV panels. However when I look at the graph for the months there is a very high value for that month. I looked in domoticz.db and can't find the table where such a value is stored. I would expect this value calculated from the meters_calendar table, which should result in a normal value.
So the question is, from which values in which table is the value for such a month calculated?
User avatar
waltervl
Posts: 5386
Joined: Monday 28 January 2019 18:48
Target OS: Linux
Domoticz version: 2024.7
Location: NL
Contact:

Re: How is value for kWh for specific month calculated?

Post by waltervl »

It is normally calculated from the difference in the total counter value.
Domoticz running on Udoo X86 (on Ubuntu)
Devices/plugins: ZigbeeforDomoticz (with Xiaomi, Ikea, Tuya devices), Nefit Easy, Midea Airco, Omnik Solar, Goodwe Solar
freekdk
Posts: 13
Joined: Monday 01 February 2021 16:33
Target OS: Linux
Domoticz version: V2024.7
Location: Netherlands
Contact:

Re: How is value for kWh for specific month calculated?

Post by freekdk »

Thanks that did solve my problem.
FlyingDomotic
Posts: 322
Joined: Saturday 27 February 2016 0:30
Target OS: Raspberry Pi / ODroid
Domoticz version: 2020.2
Contact:

Re: How is value for kWh for specific month calculated?

Post by FlyingDomotic »

freekdk wrote: Wednesday 04 September 2024 15:57 So the question is, from which values in which table is the value for such a month calculated?
Short term data (duration depends on settings, but usually a day or two) is stored in MultiMeter table.

Long term data (one record per day) is stored in MultiMeter_Calendar (and populated from short term data
willemd
Posts: 629
Joined: Saturday 21 September 2019 17:55
Target OS: Raspberry Pi / ODroid
Domoticz version: 2024.1
Location: The Netherlands
Contact:

Re: How is value for kWh for specific month calculated?

Post by willemd »

FlyingDomotic wrote: Wednesday 04 September 2024 19:09
freekdk wrote: Wednesday 04 September 2024 15:57 So the question is, from which values in which table is the value for such a month calculated?
Short term data (duration depends on settings, but usually a day or two) is stored in MultiMeter table.

Long term data (one record per day) is stored in MultiMeter_Calendar (and populated from short term data
Not totally correct. It is depending on device type, could also be meter and meter_calendar table (or other xxxx and xxxx_calendar tables).

For a kWh device it is meter and meter_calendar.

But that was not the question asked.
PierreT
Posts: 50
Joined: Wednesday 03 May 2023 10:12
Target OS: NAS (Synology & others)
Domoticz version:
Contact:

Re: How is value for kWh for specific month calculated?

Post by PierreT »

waltervl wrote: Wednesday 04 September 2024 16:20 It is normally calculated from the difference in the total counter value.
Hmmm... Haven't had any crashes for quite a while, but when I did I noticed that the month reports are created by summing all the day totals from (multi)meter_calendar. i.e. if you have days missing because Domoticz was down at midnight on those days, then the month total will not match day[31].lastvalue - day[1].firstvalue.

As a side note, the month (and year) total normally doesn't match that latter value anyway because there is a time gap between day[n].lastvalue and day[n+1].firstvalue. On a yearly base this will cause you to lose a full hour of usage data on older smart meters.
willemd
Posts: 629
Joined: Saturday 21 September 2019 17:55
Target OS: Raspberry Pi / ODroid
Domoticz version: 2024.1
Location: The Netherlands
Contact:

Re: How is value for kWh for specific month calculated?

Post by willemd »

PierreT wrote: Thursday 05 September 2024 12:13 ...
As a side note, the month (and year) total normally doesn't match that latter value anyway because there is a time gap between day[n].lastvalue and day[n+1].firstvalue. On a yearly base this will cause you to lose a full hour of usage data on older smart meters.
There is an open issue related to that point. The issue description explains the problem, identifies the cause and proposes a solution.
https://github.com/domoticz/domoticz/issues/6143
It is not related to the type of smart meter. I have this issue with a very recent and modern meter. It is related to the timestamp of midnight records in the domoticz database.
freekdk
Posts: 13
Joined: Monday 01 February 2021 16:33
Target OS: Linux
Domoticz version: V2024.7
Location: Netherlands
Contact:

Re: How is value for kWh for specific month calculated?

Post by freekdk »

I have wrong values in table meter in the second column (=total Wh produced). All these values need to be subtracted by a certain value. When I change this value in the right value at the last date/time-value I would expect that the measured Wh in the current 5 minutes would be added to this value and entered in the table at the next entry. This is however not the case. Apparently this value is saved somewhere and used to add the Wh produced in the current 5 minutes.
At midnight this wrong value in entered in the table meter_calendar. I do have a script, to be run shortly after midnight, to change it in the right value.

How do I get the right values in the table meter? In other words where does domoticz save this value to be used to calculate the total Wh produced till that moment. I looked in multimeter, but the deviceid is not present there.
FlyingDomotic
Posts: 322
Joined: Saturday 27 February 2016 0:30
Target OS: Raspberry Pi / ODroid
Domoticz version: 2020.2
Contact:

Re: How is value for kWh for specific month calculated?

Post by FlyingDomotic »

freekdk wrote: Friday 06 September 2024 12:33 I looked in multimeter, but the deviceid is not present there.
As willemd said, I made a mistake. I'm using RFXcounter meters which are stored in multimeter tables, while you're using kWh counters, which are using meter tables.
willemd
Posts: 629
Joined: Saturday 21 September 2019 17:55
Target OS: Raspberry Pi / ODroid
Domoticz version: 2024.1
Location: The Netherlands
Contact:

Re: How is value for kWh for specific month calculated?

Post by willemd »

freekdk wrote: Friday 06 September 2024 12:33 I have wrong values in table meter in the second column (=total Wh produced). All these values need to be subtracted by a certain value. When I change this value in the right value at the last date/time-value I would expect that the measured Wh in the current 5 minutes would be added to this value and entered in the table at the next entry. This is however not the case. Apparently this value is saved somewhere and used to add the Wh produced in the current 5 minutes.
At midnight this wrong value in entered in the table meter_calendar. I do have a script, to be run shortly after midnight, to change it in the right value.

How do I get the right values in the table meter? In other words where does domoticz save this value to be used to calculate the total Wh produced till that moment. I looked in multimeter, but the deviceid is not present there.
It is not advised to update the meter table directly, for example by sqlite3. If you want to update a counter to a corrected value it is better to use the domoticz.devices().updateCounter() function in a small separate dzVents script. In that way you make sure the value is updated in the meter table, the devicestatus table and internal memory, so you don't get inconsistencies.

If after that you also want to update meter_calendar, that is less of a problem. You can use sqlite3 to update either value or counter to the values you want. Sometimes it is better and more clear to wait one day after the meter table update, so the midnight closing routine has run, and then you update meter_calendar.

Of course, the updateCounter() only updates the current value of the device, not any incorrect past values in the meter table. I would not worry about incorrect past values in the meter tables since they will disappear anyway after a number of days (depending on your log history setting). I would only update the current value of the counter and then correct meter_calendar.
PierreT
Posts: 50
Joined: Wednesday 03 May 2023 10:12
Target OS: NAS (Synology & others)
Domoticz version:
Contact:

Re: How is value for kWh for specific month calculated?

Post by PierreT »

willemd wrote: Thursday 05 September 2024 14:11
PierreT wrote: Thursday 05 September 2024 12:13 ...
As a side note, the month (and year) total normally doesn't match that latter value anyway because there is a time gap between day[n].lastvalue and day[n+1].firstvalue. On a yearly base this will cause you to lose a full hour of usage data on older smart meters.
There is an open issue related to that point. The issue description explains the problem, identifies the cause and proposes a solution.
https://github.com/domoticz/domoticz/issues/6143
It is not related to the type of smart meter. I have this issue with a very recent and modern meter. It is related to the timestamp of midnight records in the domoticz database.
Ow! You are right. It references (multi)meter_calendar for first entry and last entry of a given time period, so for a single day the gap is in fact 5 minutes plus or minus a second or two. That is 2.5 hours worth in missing data for a single month due to summing the individual day values rather than referencing the month as a whole or alternatively the days as a coherent set.
freekdk
Posts: 13
Joined: Monday 01 February 2021 16:33
Target OS: Linux
Domoticz version: V2024.7
Location: Netherlands
Contact:

Re: How is value for kWh for specific month calculated?

Post by freekdk »

willemd wrote: Friday 06 September 2024 13:17 It is not advised to update the meter table directly, for example by sqlite3. If you want to update a counter to a corrected value it is better to use the domoticz.devices().updateCounter() function in a small separate dzVents script. In that way you make sure the value is updated in the meter table, the devicestatus table and internal memory, so you don't get inconsistencies.

If after that you also want to update meter_calendar, that is less of a problem. You can use sqlite3 to update either value or counter to the values you want. Sometimes it is better and more clear to wait one day after the meter table update, so the midnight closing routine has run, and then you update meter_calendar.

Of course, the updateCounter() only updates the current value of the device, not any incorrect past values in the meter table. I would not worry about incorrect past values in the meter tables since they will disappear anyway after a number of days (depending on your log history setting). I would only update the current value of the counter and then correct meter_calendar.
It goes above my capabilities to use dzVents, but, from what I read here, this updates only the produced Wh, these are all OK in table Meter. It is the value of the total Wh, all these values in the table Meter, that need to change. I tried this by changing the last row in the table - only the total Wh - , but apparently that value is not used when a new 5 minutes Wh value comes in and this value is added to that previous total Wh.

Would changing all these values with sqlite3 in table Meter help?

I even tried to make a backup of domoticz.db, change the last row in the table Meter, and restore this database in Domoticz, by uploading it. Did not help.

Would stopping Domoticz, change these values in the database with sqlite3, and restarting Domoticz work?
FlyingDomotic
Posts: 322
Joined: Saturday 27 February 2016 0:30
Target OS: Raspberry Pi / ODroid
Domoticz version: 2020.2
Contact:

Re: How is value for kWh for specific month calculated?

Post by FlyingDomotic »

freekdk wrote: Saturday 07 September 2024 12:10 Would stopping Domoticz, change these values in the database with sqlite3, and restarting Domoticz work?
Preferred way is: stop Domoticz, backup database, do changes with sqlite3 and restart Domoticz.

In case of problem: stop Domoticz, restore database (copy database backup over corrupted/not working one), restart Domoticz.
willemd
Posts: 629
Joined: Saturday 21 September 2019 17:55
Target OS: Raspberry Pi / ODroid
Domoticz version: 2024.1
Location: The Netherlands
Contact:

Re: How is value for kWh for specific month calculated?

Post by willemd »

freekdk wrote: Saturday 07 September 2024 12:10 It goes above my capabilities to use dzVents, ....
This is an example of the dzVents script needed.
Just adjust the time at which it needs to run (instead of 11:11), the devicenr (instead of 999) and the required value (instead of 11111).
Don't forget to switch it off afterwards otherwise it will run every day.

Code: Select all

return {
	on = {
		timer = {
			'at 11:11',					-- specific time
		}
	},
	logging = {
		level = domoticz.LOG_INFO,
		marker = 'template',
	},
	execute = function(domoticz, timer)
		
		domoticz.devices(999).updateCounter(11111)
	end
}

freekdk
Posts: 13
Joined: Monday 01 February 2021 16:33
Target OS: Linux
Domoticz version: V2024.7
Location: Netherlands
Contact:

Re: How is value for kWh for specific month calculated?

Post by freekdk »

FlyingDomotic wrote: Saturday 07 September 2024 14:28
freekdk wrote: Saturday 07 September 2024 12:10 Would stopping Domoticz, change these values in the database with sqlite3, and restarting Domoticz work?
Preferred way is: stop Domoticz, backup database, do changes with sqlite3 and restart Domoticz.

In case of problem: stop Domoticz, restore database (copy database backup over corrupted/not working one), restart Domoticz.
Only changing the values in table meter did not work, also not after stopping domoticz, changing the values in table meter and restarting. There is an svalue in table devicestatus, where the last value of total Wh is stored. When changing that value also it works when you stop domoticz, change values in table meter and take the youngest value of total Wh as the new value in svalue in table devicestatus and after that start domoticz.
Post Reply

Who is online

Users browsing this forum: No registered users and 0 guests