How to get actual usage from P1 device

[JeroenG]

I'm working on script to work with cariabel eletriciy prices (these change per hour).

Getting the prices and putting them in DOmoticz is already working, but now facing the issue to calculate the correct costs of the previous hour.

How can I retreive the actual usage from the "Power" device I have connected to the P1. If I do a curl command to retreive the status with range day I get all values needed, EXCEPT when we have a day roll-over.

In normal hours-change I take the usage eu value of the start full hour and of the next full hour. the difference is the usage for this hour, and will be multiplied by the price per kWh.

However, the counter is reser on hour 0 of a day, so I can not calculate the difference between hour 23 and hour 0...

Example (simplified):

Code: Select all

{
  "d" : "2023-01-15 22:00",
  "eu" : "16387",
	},
	
{
  "d" : "2023-01-15 23:00",
  "eu" : "16815",
	},

Usage in hour 23 (22:00-23:00) is 428Wh (16815-16387)

But:

Code: Select all

{
  "d" : "2023-01-15 23:00",
  "eu" : "16815",
	},

{
  "d" : "2023-01-16 00:00",
  "eu" : "0",
	},

What would be the usage in hour 24(23:00-00:00)????

I know Domoticz is retreiving the values from the P1 meter at least every minute, but how van I use these?

[willemd]

Have a look here https://www.domoticz.com/wiki/DzVents:_ ... mart_meter

Assuming you have the P1 device in Domoticz updated, it will have several values you can work with in dzVents, depending on your script.
counterToday and counterDeliveredToday (cumulative during the day)
or usage1+usage2-return1-return2 (cumulative from start)
or actual usage and usageDelivered

so for example domoticz.devices(idx-nr-of-your-p1).usage1 will give you the cumulative total usage during low tariff periods.

[willemd]

And here is a script I use to get the prices but also calculate the costs.
Have a look at the "at *59" part of the script.

Basically I am starting the day with the fixed costs and then add the daily variable to it during the day.

Code: Select all

-- Script to load day-ahead electricity prices into Domoticz historic variables, as a base for further processing.
-- New prices are available every day at 15:00 for the coming 24 hrs.
-- These are base prices. Providers of dynamic electricity contracts will add their surcharges and government taxes to it.
-- API documentation at: https://transparency.entsoe.eu/content/static_content/Static%20content/web%20api/Guide.html
-- To get this data yourself (it is free):
--    1) Register for an account at https://transparency.entsoe.eu/dashboard/show
--    2) Send an email with subject "Restful API access" and your account email address in the body
--    3) After receipt of their confirmation, go into your account and generate your token.
--    4) Store the token in a domoticz user variable and note down the idx.
--    5) Adapt the idx on the line below.

-- devices
local idxElectricityP1=1
local idxGasP1=2

local idxCurrentDynamicElPrice=102
local idxDynamicElCosts=103
local idxDynamicElCostsCntr=104  -- counters are just added for testing, work fine as long as costs are positive

local idxCurrentDynamicGasPrice=107
local idxDynamicGasCosts=108
local idxDynamicGasCostsCntr=109 -- counters are just added for testing, work fine as long as costs are positive

-- user variables
local idxEUtoken=12  -- idx of the user variable with the API token
local idxBTWpercentage=13 -- user variable with BTW/VAT percentage
local idxDayElTransport=3 -- user variable for fixed daily transport costs, incl BTW
local idxMonthElFixed=8   -- user variable for fixed monthly costs, incl BTW
local idxDayGasTransport=6 -- user variable for fixed daily transport costs, incl BTW
local idxMonthGasFixed=8   -- user variable for fixed monthly costs, incl BTW
local idxElTax=4          -- user variable for electricity tax, incl BTW
local idxGasTax=7         -- user variable for gas tax, incl BTW

return {
	on = {
		timer = {
		    'at *:45',  -- Timer to get new electricity prices. Adapt timer to your needs. Normally new electricty prices are available after 15:00. 
		                -- Timer to get new gas prices. Prices received are in UTC so 1 hour shift needed for me.    
		                -- Prices are now re-loaded every hour, to avoid using old prices when there is one error loading once per day.
		    'at *:59',   -- trigger to calculate the costs of the past hour (missing 1 minute, added to next hour, except end of day)
		    'at *:01'    -- trigger to copy price from list (24 historic variables) to device with current price
		},
		httpResponses = {
			'EUprices', -- must match with the callback passed to the openURL command in the code below
			'EZGASprices',
		}
	},
	data =  {
            ElDayPrices               = { history = true, maxItems = 24 },
            GasDayPrices              = { history = true, maxItems = 24 },
            PreviousHrCumElCosts      = { initial = 0},
            PreviousHrCumElNett       = { initial = 0},
            PreviousHrCumGasCosts      = { initial = 0},
            PreviousHrCumGasNett       = { initial = 0}
    },
	logging = {
		level = domoticz.LOG_INFO,
		marker = 'EU and EZ day ahead prices',
	},
	execute = function(domoticz, item)
	    
	    local ElDayPrices = domoticz.data.ElDayPrices
	    local GasDayPrices = domoticz.data.GasDayPrices
	    local todayEl = domoticz.devices(idxElectricityP1).counterToday
	    local todayElreturn = domoticz.devices(idxElectricityP1).counterDeliveredToday
	    local todayGas = domoticz.devices(idxGasP1).counterToday
	    
	    if (item.isTimer) then
	        if (item.trigger == 'at *:01') then
	            -- copy dynamic price to device holding the current price
	            local VarIndex=24-tonumber(os.date("%H")) -- last price of day is index 1
	            local CurrentElPrice=tonumber(ElDayPrices.get(VarIndex).data)/1000 -- from EURO/MWh to EURO/kWh
	            CurrentElPrice=CurrentElPrice * (100 + domoticz.variables(idxBTWpercentage).value)/100 -- add BTW/VAT
	            domoticz.log('Current Dynamic El price EURO/kWh :' .. CurrentElPrice,domoticz.LOG_INFO)
	            domoticz.devices(idxCurrentDynamicElPrice).updateCustomSensor(CurrentElPrice)
	            
	            -- also copy gas prices to current device
	            if tonumber(os.date("%H"))==0 then
	                VarIndex=1 
	            else
	                VarIndex=24-tonumber(os.date("%H"))+1
	            end     
	            local CurrentGasPrice=tonumber(GasDayPrices.get(VarIndex).data) -- Euro per m3, last price is for 00:00 to 01:00 next day.
	            CurrentGasPrice=CurrentGasPrice * (100 + domoticz.variables(idxBTWpercentage).value)/100 -- add BTW/VAT
	            domoticz.log('Current Dynamic Gas price :' .. CurrentGasPrice,domoticz.LOG_INFO)
	            domoticz.devices(idxCurrentDynamicGasPrice).updateCustomSensor(CurrentGasPrice)
	            
	            if (tonumber(os.date("%H")) == 0) then
	                domoticz.devices(idxDynamicElCosts).updateCustomSensor(domoticz.data.PreviousHrCumElCosts)
	                domoticz.devices(idxDynamicGasCosts).updateCustomSensor(domoticz.data.PreviousHrCumGasCosts)
	            end 
	            
	        else
	            if (item.trigger == 'at *:59') then
    	            -- first calculate cumulative daily total and last hour electricity usage
    	            local todayElNett=domoticz.devices(idxElectricityP1).counterToday-domoticz.devices(idxElectricityP1).counterDeliveredToday
    	            domoticz.log('counterToday :' .. domoticz.devices(idxElectricityP1).counterToday,domoticz.LOG_INFO)
    	            domoticz.log('counterDeliveredToday :' .. domoticz.devices(idxElectricityP1).counterDeliveredToday,domoticz.LOG_INFO)
    	            domoticz.log('todayElNett :' .. todayElNett,domoticz.LOG_INFO)
    	            local lastHrElNett = todayElNett - domoticz.data.PreviousHrCumElNett
    	            domoticz.log('lastHrElNett :' .. lastHrElNett,domoticz.LOG_INFO)
    	            -- then calculate the costs and add to device
    	            local lastHrElCosts = lastHrElNett * domoticz.devices(idxCurrentDynamicElPrice).sensorValue + lastHrElNett * domoticz.variables(idxElTax).value  -- tax variable is already including BTW/VAT
    	            domoticz.log('lastHrElCosts :' .. lastHrElCosts,domoticz.LOG_INFO)
    	            local CumElCosts = domoticz.utils.round(domoticz.data.PreviousHrCumElCosts + lastHrElCosts,2)
    	            domoticz.log('CumElCosts :' .. CumElCosts,domoticz.LOG_INFO)
    	            domoticz.devices(idxDynamicElCosts).updateCustomSensor(CumElCosts)
    	            
    	            local previousCounter=0 
		            local divider=100
		            previousCounter=domoticz.devices(idxDynamicElCostsCntr).counter
		            --domoticz.devices(idxDynamicElCostsCntr).updateCounter(lastHrElCosts*divider + previousCounter*divider)
		            domoticz.devices(idxDynamicElCostsCntr).updateCounter(CumElCosts*divider)
		            domoticz.log('previousCounter :' .. previousCounter,domoticz.LOG_INFO)
		            domoticz.log('newcounter :' .. lastHrElCosts + previousCounter,domoticz.LOG_INFO)
	            
    	            -- if end of day, reset the cumulative daily totals 
    	            if (tonumber(os.date("%H"))==23) then
    	                domoticz.data.PreviousHrCumElCosts=domoticz.utils.round(domoticz.variables(idxDayElTransport).value+domoticz.variables(idxMonthElFixed).value/31,2) -- variables are including BTW/VAT
    	                domoticz.data.PreviousHrCumElNett=0
    	            else
    	                domoticz.data.PreviousHrCumElCosts=CumElCosts
    	                domoticz.data.PreviousHrCumElNett=todayElNett
    	            end    
    	            
    	            -- first calculate cumulative daily total and last hour gas usage
    	            local todayGasNett=domoticz.devices(idxGasP1).counterToday
    	            domoticz.log('counterToday :' .. domoticz.devices(idxGasP1).counterToday,domoticz.LOG_INFO)
    	            domoticz.log('todayGasNett :' .. todayGasNett,domoticz.LOG_INFO)
    	            local lastHrGasNett = todayGasNett - domoticz.data.PreviousHrCumGasNett
    	            domoticz.log('lastHrGasNett :' .. lastHrGasNett,domoticz.LOG_INFO)
    	            -- then calculate the costs and add to device
    	            local lastHrGasCosts = lastHrGasNett * domoticz.devices(idxCurrentDynamicGasPrice).sensorValue +lastHrGasNett * domoticz.variables(idxGasTax).value -- tax variable is already including BTW/VAT
    	            domoticz.log('lastHrGasCosts :' .. lastHrGasCosts,domoticz.LOG_INFO)
    	            local CumGasCosts = domoticz.utils.round(domoticz.data.PreviousHrCumGasCosts + lastHrGasCosts,2)
    	            domoticz.log('CumGasCosts :' .. CumGasCosts,domoticz.LOG_INFO)
    	            domoticz.devices(idxDynamicGasCosts).updateCustomSensor(CumGasCosts)
    	            
    	            previousCounter=0 
		            divider=100
		            previousCounter=domoticz.devices(idxDynamicGasCostsCntr).counter
		            --domoticz.devices(idxDynamicGasCostsCntr).updateCounter(lastHrGasCosts*divider + previousCounter*divider)
		            domoticz.devices(idxDynamicGasCostsCntr).updateCounter(CumGasCosts*divider)
		            domoticz.log('previousCounter :' .. previousCounter,domoticz.LOG_INFO)
		            domoticz.log('newcounter :' .. lastHrGasCosts + previousCounter,domoticz.LOG_INFO)
	            
    	            -- if end of day, reset the cumulative daily totals 
    	            if (tonumber(os.date("%H"))==23) then
    	                domoticz.data.PreviousHrCumGasCosts=domoticz.utils.round(domoticz.variables(idxDayGasTransport).value+domoticz.variables(idxMonthGasFixed).value/31,2)  -- variables are including BTW/VAT
    	                domoticz.data.PreviousHrCumGasNett=0
    	            else
    	                domoticz.data.PreviousHrCumGasCosts=CumGasCosts
    	                domoticz.data.PreviousHrCumGasNett=todayGasNett
    	            end     
    	            
    	            
	            else
        	        -- section to launch the EU electricity API get request 
        	        local UrlStart='https://web-api.tp.entsoe.eu/api?'  -- the API website
        	        local EUtoken=domoticz.variables(idxEUtoken)        -- user variable with API token
        	        local DocType='A44'                                 -- day ahead prices document type 
        	        local PriceRegion='10YNL----------L'                -- region is set to The Netherlands (adapt to your need as per API documentation)
        	        
        	        -- depending on launch hour, get current or tomorrow's data
        	        local dayahead=0
        	        if tonumber(os.date("%H"))>=23 then                 -- day ahead available after 15:00, otherwise retrieve current day, depending on the time this script is launched
        	            dayahead=24*60*60  -- seconds for full day ahead
        	        end     
        	        local PricePeriodStart=os.date("%Y%m%d",os.time()+ dayahead) .. "0000"  -- range 00:00 to 23:00, this will return full day anyway
        	        local PricePeriodEnd=os.date("%Y%m%d", os.time()+ dayahead) .. "2300"    -- depending on time the script is launched, get current day or tomorrow's data
        	        
        	        -- compose the full URL
        	        local EUurl=UrlStart .. 'securityToken=' .. EUtoken.value .. '&documentType=' .. DocType .. '&in_Domain=' .. PriceRegion .. '&out_Domain=' .. PriceRegion .. '&periodStart=' .. PricePeriodStart .. '&periodEnd=' .. PricePeriodEnd
        	        
        	        domoticz.log("URL : " .. EUurl, domoticz.LOG_INFO)
        	        
        	        -- launch the URL
        			domoticz.openURL({
        				url = EUurl,
        				method = 'GET',
        				callback = 'EUprices', -- must match httpResponses above
        			})
        			
        			-- section to launch the EnergyZero gas prices API get request (UTC timing), run between 00:00 and 01:00
        	        local GASUrlStart='https://api.energyzero.nl/v1/energyprices?'  -- the API website
        	        local usageType=2  -- GAS
        	        
        	        -- always get current day data
        	        local GasPricePeriodStart=os.date("%Y-%m-%d",os.time()) .. "T00:00:00.000Z"  -- this first price is valid from 01:00 CET
        	        local GASPricePeriodEnd=os.date("%Y-%m-%d", os.time()) .. "T23:59:59.999Z"            	        
        	        
        	        -- compose the full URL
        	        local EZurl=GASUrlStart .. 'fromDate=' .. GasPricePeriodStart .. '&tillDate=' .. GASPricePeriodEnd .. '&interval=4&usageType=' .. usageType .. '&inclBtw=false'
        	        
        	        domoticz.log("URL : " .. EZurl, domoticz.LOG_INFO)
        	        
        	        -- launch the URL
        			domoticz.openURL({
        				url = EZurl,
        				method = 'GET',
        				callback = 'EZGASprices', -- must match httpResponses above
        			})
        			
        	    end		
    		end	
		else 
		    if (item.isHTTPResponse) then
		        -- response to openURL (HTTP GET) request was received
		        if (item.trigger=="EUprices") then
    			    if (item.ok) then
    			        if (item.isXML) then   -- should be XML
    			            --domoticz.log('start dumptable', domoticz.LOG_INFO)
    			            --domoticz.utils.dumpTable(item.xml)  -- dumpTable can be used for debugging
    			            --domoticz.log('end dumptable', domoticz.LOG_INFO)
    			            
    			            ElDayPrices.reset()  -- remove historic prices from previous run
    			            
    			            for id = 1, 24 do
                                domoticz.log('adding EL price : ' .. item.xml.Publication_MarketDocument.TimeSeries.Period.Point[id]['price.amount'],domoticz.LOG_INFO)
                                ElDayPrices.add(item.xml.Publication_MarketDocument.TimeSeries.Period.Point[id]['price.amount'])
                            end

    			        else
    				        domoticz.log('No XML received', domoticz.LOG_INFO)
    				    end
    			    else
    				    domoticz.log('There was a problem handling the request. Item not ok', domoticz.LOG_INFO)
    				    domoticz.log(item, domoticz.LOG_INFO)
    			    end
	            else
	                -- trigger was not eu electricity prices but energyzero gas prices
                    domoticz.log('start gasprices', domoticz.LOG_INFO)
    	            if (item.ok) then
    	                domoticz.log('gasprices ok', domoticz.LOG_INFO)
    			        if (item.isJSON) then   -- should be JSON
    			            domoticz.log('start dumptable', domoticz.LOG_INFO)
    			            domoticz.utils.dumpTable(item.json)  -- dumpTable can be used for debugging
    			            domoticz.log('end dumptable', domoticz.LOG_INFO)
    			            
    			            GasDayPrices.reset()  -- remove historic prices from previous run
    			            
    			            for gasid = 1, 24 do
    			                -- this adds prices from current day 01:00-02:00 to next day 00:00-01:00 intervals
                                domoticz.log('adding GAS price : ' .. item.json.Prices[gasid].price,domoticz.LOG_INFO)
                                GasDayPrices.add(item.json.Prices[gasid].price)
                            end
                          
    			        else
    				        domoticz.log('No JSON received', domoticz.LOG_INFO)
    				    end
    			    else
    				    domoticz.log('There was a problem handling the request. Item not ok', domoticz.LOG_INFO)
    				    domoticz.log(item, domoticz.LOG_INFO)
    			    end
			    end
		    end
	    end
	end
}

[JeroenG]

Thanks Willem,

this looks like what I'm trying to achive myself . I will have a closer look this evening.

[JeroenG]

Modified the script from Willem (above) according my needs and combined with my own scripts which collect the electricity prices from enever.nl

Advantage is that the flexibility of the hourly prices is visible in Domoticz (and pushed to Influx/Grafana to give more insight over longer period).

For anyone interested: https://github.com/GoumJer/domoticz_energy_prices

[hve22]

Modified the script from Willem (above) according my needs and combined with my own scripts which collect the electricity prices from enever.nl

Advantage is that the flexibility of the hourly prices is visible in Domoticz (and pushed to Influx/Grafana to give more insight over longer period).

For anyone interested: https://github.com/GoumJer/domoticz_energy_prices

I've tried the code from JeroenG but somehow i got the wrong dynamic costsin my domotics custom sensor. The explanation of how to edit the code is different from the code itself. So somehow i'm lost in the code
Can you provide a new manual in relation with the latest code?

[Pergotje]

@JeroenG I managed to get your script working.
But how does it work?
If the update_costs retrieves the prices, retrieves the power usage, calculates the costs and updates the cost sensor every hour,
then what does the Dvents do that the update_costs doesn't do?
Seems double, what am I missing?