Calculating actual energy consumption

[mikeathome]

I have the 'SolarEdge via Web API' hardware installed and it works like a clock. Also, my 'P1 Smart Meter' hardware works fantastic.

To know my 'true' energy consumption I need to calculate it, basically like this:
Energy bought from the utlitiy company (P1)
plus
Energy produced by my solar panels (SolarEdge)
minus
Energy sold to the utility company (P1)

I am currently doing this i Excel but would like to have this value within the Domoticz' Utility tab. Can someone point me towards where I can learn how to create, I am guessing now, a dummy device that does this calculation.

I am sure this is a common question so please excuse me, but I have not been able to find a good entry point to learn this

/Mike

[manjh]

I would recommend a simple LUA script, and then store the result in a virtual device.

[mikeathome]

I would recommend a simple LUA script, and then store the result in a virtual device.

Thank you @manjh! It will be my first LUA script, and also my first virtual device!

Anyone aware of an example LUA script including reading values from a device and storing it in a virtual device?

[akamming]

Anyone aware of an example LUA script including reading values from a device and storing it in a virtual device?

Your actual energy consumption won't work with the web api which domoticz is using. Because your P1 meter gives you an actual value and the value in the API can be up till 15 mins old. So i installed https://github.com/tjko/sunspec-monitor to get the actual info from my inverter.

For this you need to enable modbus tcp on your solaredge inverter (instructions can be found here: https://knowledge-center.solaredge.com/ ... l-note.pdf)

and ofcourse you need to cerate a virtual device of the type kwh meter.

And then you can make a script which at every interval of the P1 meter also reads the invertor, compares the data, calculates the actual energy usage and stores to the virtual kwh meter.

For this purpose i used a dzvents script: This one is tailor made for my situation. You ofcourse have to adjust to your own config. I myself stopped using the internal domoticz interface to solaredge web api, cause this script is much more accurate. So for this script you als have to create virtual devices for the thing i measure. Like the kwhmeter for the solarpanels themselvers, the inverter temperature and the inverter efficiency

anyway here's the script. Have fun..!

Code: Select all

-- This script handles everything with Energy, current functionality
--
-- on every update of the smart Meter
-- it will retrieve the solaredge values and update those sensors
-- and then calculate and update the ActualPowerConsumption by combining the Smart Meter and Solar Edge data 

return {
    logging = {
 		level = domoticz.LOG_ERROR, 			-- Adjust to your needs
 		marker = 'ActualPowerConsumption'
 	},

	on = {
		devices = {
		    'Slimme Meter' -- Name of P1 Smart Meter device 
	    },
		shellCommandResponses = {
			'Energy' -- must match with the callback passed to the executeShellCommand 
		}

	},
	execute = function(domoticz, item)
	    if (item.isDevice) then
			domoticz.executeShellCommand({
				command = 'sunspec-status -m 0 -j 192.168.2.13',   -- IP Adress of convertor
				callback = 'Energy', -- see shellCommandResponses above.
				timeout = 5, -- Max runtime 50 seconds
			})

        elseif (item.isShellCommandResponse) then
            if (item.ok) then
                result=item.data
                domoticz.log ("JSON output Inverter is "..result, domoticz.LOG_DEBUG)
                
                domoticz.log("Got JSON output, proceeding",domoticz.LOG_DEBUG)
            
                -- no error: declare needed opbjects
    		    local SM=domoticz.devices("Slimme Meter") -- P1 Smart Meter Device
    		    local SE=domoticz.devices("Zonnepanelen") -- Virtual Electricity device (usage, counter) in which the actual Solar Output will be stored 
    		    local SV=domoticz.devices("Stroom Verbruik") -- Virtual Electricity devie (usage, counter) in which the actual power consumption will be stored
    		    local OT=domoticz.devices("Omvormer Temperatuur") -- Virtual Temperature device, in which the inverter temperature will be stored
    		    local OE=domoticz.devices("Omvormer Efficiency") -- Virtual Efficiency device, in which the inverter efficiency will be stored
    
                -- reading smart meter
    		    local usage=SM.usage
    		    local usagedelivered=SM.usageDelivered
    
    		    domoticz.log ("Usage = "..usage ..", Delivered = "..usagedelivered..", Produced = "..item.json.ac_power..", P1 cumulative="..SM.usage1+SM.usage2-SM.return1-SM.return2, domoticz.LOG_DEBUG)
    	        
    	        -- log  timestamp for debugging purposes
    	        local Time=require('Time')
    	        domoticz.log("Timestamp inverter = "..item.json.timestamp..", which is "..Time(item.json.timestamp).secondsAgo.." second(s) ago", domoticz.LOG_DEBUG)
    	        
    	        -- update SolarEdge virtual sensors
    		    domoticz.log("SE.updateElectricity("..item.json.ac_power..", "..item.json.total_production..")", domoticz.LOG_FORCE)
    		    SE.updateElectricity(item.json.ac_power, item.json.total_production)
    		    
    		    if (item.json.temperature>0) then
        		    domoticz.log("OT.updateTemperature("..item.json.temperature..")", domoticz.LOG_FORCE)
        		    OT.updateTemperature(item.json.temperature)
    		    else 
    		        domoticz.log("temperature sensor is switched off, not updating temperature",domoticz.LOG_DEBUG)
    		    end
    
                -- calculate real energy consumption
                local virtuelemeterstand=item.json.total_production+SM.usage1+SM.usage2-SM.return1-SM.return2
                local vermogen=SM.usage-SM.usageDelivered+item.json.ac_power
                
                -- update real actual usage sensor
                domoticz.log("SV.updateElectricity("..vermogen..","..virtuelemeterstand..")", domoticz.LOG_FORCE)
                SV.updateElectricity(vermogen,virtuelemeterstand)
                
    		    if (item.json.ac_power>100) then
    		        -- calculating enough energy to calculate efficiency
    		        local Efficiency=item.json.ac_power/item.json.dc_power*100
    		        domoticz.log("OE.updatePercentage("..Efficiency..")",domoticz.LOG_FORCE)
    		        OE.updatePercentage(Efficiency)
    	        else
    	            domoticz.log("production < 100Watt, too low to measure Efficiency, so not updating efficiency sensor",domoticz.LOG_DEBUG)
    	        end
            else
            	domoticz.log('failure in script', domoticz.LOG_ERROR)
            	domoticz.log(item,domoticz.LOG_ERROR)
            end
        end
	end
}

[lost]

I would recommend a simple LUA script, and then store the result in a virtual device.

Just take care Domoticz takes samples in DB/graphs on a 5mn slice basis. So I would recommend to compute values continuously and only store data in virtual device every 10mn (>5mn slice) to avoid loosing data.

I have a script that does average power drain calculus for my home on a 10mn slice basis that may serve as a exemple but I think this'll be a bit difficult for someone that discovers scripting/lua...

[akamming]

I would recommend a simple LUA script, and then store the result in a virtual device.

Just take care Domoticz takes samples in DB/graphs on a 5mn slice basis. So I would recommend to compute values continuously and only store data in virtual device every 10mn (>5mn slice) to avoid loosing data.

I have a script that does average power drain calculus for my home on a 10mn slice basis that may serve as a exemple but I think this'll be a bit difficult for someone that discovers scripting/lua...

then still it's an approximation with weird values every now and then. The cause of that is that the SolarEdge Web API only reports new values every about 15 minutes. And even then it;s always a few minutes old. In domoticz it's a polling interface, so you don't get the current values.

So you can make more accurate by calculating over a longer period by. But still you are always a few minutes off, causing weird values when you switch on or off appliances which use a lot of energy.

That's why i switched to the modbustcp. On every update of the P1, The dzvents als retrieves the actual values of the inverter by modbustcp and then calculates actual energy usage at the current time (and also updates the counters, so you know daily/hourly totals etc..).

[mikeathome]

Thank you @akamming and @lost!

I would never have thought of the issues with the API's 15 minutes delay if you hadn't mentioned it. Currently I am normally only reviewing daily data, but it explains some strange readings when viewing production/consumption in "real time".

This will be a fun project to dig into at my vacation later this summer (beware ... I might reach out ). I have another high prio project as well. After a SSD crash (yes - SSD!) last week I've realized I must get a daily rsync backup to my NAS to work as well.

[lost]

then still it's an approximation with weird values every now and then. The cause of that is that the SolarEdge Web API only reports new values every about 15 minutes. And even then it;s always a few minutes old. In domoticz it's a polling interface, so you don't get the current values.

So you can make more accurate by calculating over a longer period by. But still you are always a few minutes off, causing weird values when you switch on or off appliances which use a lot of energy.

I don't have this refresh rate issue with the power metering device I use because I can configure both a refresh rate (for instance 1mn) AND trigger refresh on some % measurement change (down to 1%). So I'm rarely 15s without receiving I/V/W/CosPhy measurements and never more that 1mn (can occur at night, with almost flat figures not triggering % change)+no pooling needed there.

In such case, quite different, my concern was with Domoticz 5mn samples missing most information about my real power drain profile!

So I scripted a (real) max and mean (approx, but with 1% change reports still correct; calculated from samples integrated over time duration between reports) power drain over 10mn slices that reliably make their way in DB/graphs. Publishing them will not help there!

[gizmocuz]

The data in the day graphs are sampled every 5 minutes. But all data is received in realtime (and also displayed in all widgets)
Sure if data on the remote site is delayed with 15 minutes, there is nothing we can do about this.
If you have a P1 Smart Meter and set the poll interval to 10 seconds, you will get precieze average values every 10 seconds

[lost]

The data in the day graphs are sampled every 5 minutes. But all data is received in realtime (and also displayed in all widgets.

Yes, data all data is received and also last_updates are correct: That's both figures availability that allows scripted integration over time to compute an approx mean (some devices may be able to do this internally with their FW, but my Qubino Smart-Meter is not).

Getting reliable max power allows managing heavy consumers cut to avoid breaking electricity provider contract limit (and the home shutdown that'll come shortly!). Mean values over a configurable time slice allows to keep better kWh prices in locations that now charge as a function of daily/monthly max quarter power figures.

[akamming]

The data in the day graphs are sampled every 5 minutes. But all data is received in realtime (and also displayed in all widgets)
Sure if data on the remote site is delayed with 15 minutes, there is nothing we can do about this.
If you have a P1 Smart Meter and set the poll interval to 10 seconds, you will get precieze average values every 10 seconds

that's indeed the case. Domoticz just registers what it gets and stores in the DB when it retrieves. However SolarAPI sends data which can be up to 15 mins old.

would be nice to have a native domoticz Solaredge integration for the ModbusTCP protocol.. then it becomes realtime..

anyway for now my dzvens script also works fine..

[manjh]

This discussion has more or less focused on the (lack of) accuracy, with readings being done every 5 minutes or so.
But keep in mind mikeathome is currently doing the calculations in an Excell spreadsheet, so I doubt he is very worried about data being 5 minutes old. Of course it is worth mentioning.

I have a script running that will detect a period of excess power. As soon a that is enough, I can decide to switch on an electric heater, or the air conditioner.. Once I get an EV in the future, I could use it to charge the battery.
All this can be done on data which is not "real time". Maybe not perfect, but good enough.

[mrelfire]

I had a related experience where I needed to streamline how I monitored my energy consumption. Initially, I manually tracked in Excel, much like you're doing. However, integrating this data into a more intelligent system made a big difference.

For your situation with Domoticz, creating a dummy device that does the calculation is a logical step. While I'm not an expert in Domoticz scripting, their forums and user guides would be the best places to start.

Moreover, for a broader perspective on intelligent energy solutions, you might find https://thingsboard.io/smart-energy/ helpful. It offers insights into efficient energy monitoring and management, which could be beneficial in understanding and improving your overall energy consumption. This platform provides valuable tools like real-time data visualization, which could complement your efforts in Domoticz.

[mikeathome]

I had a related experience where I needed to streamline how I monitored my energy consumption. Initially, I manually tracked in Excel, much like you're doing. However, integrating this data into a more intelligent system made a big difference.

For your situation with Domoticz, creating a dummy device that does the calculation is a logical step. While I'm not an expert in Domoticz scripting, their forums and user guides would be the best places to start.

Moreover, for a broader perspective on intelligent energy solutions, you might find https://thingsboard.io/smart-energy/ helpful. It offers insights into efficient energy monitoring and management, which could be beneficial in understanding and improving your overall energy consumption. This platform provides valuable tools like real-time data visualization, which could complement your efforts in Domoticz.

Thank you @mrelfire for the link to Thingsboard - I have to look deeper into it - but it looks promising!