How to save data from JSON

[MdeBoer]

Hello everyone i want to save a JSON tabel in "data" of DzVents

Code: Select all

  
data =  {
	historycalprices        = { history = true, maxItems = 24 }
	},

The JSON I want to save is this, only the "prices" object with price in the fist blok en the readingDate in the second

Code: Select all

{
 "Prices": [
  {
   "price": 0.22,
   "readingDate": "2022-11-19T23:00:00Z"
  },
  {
   "price": 0.21,
   "readingDate": "2022-11-20T00:00:00Z" -- it has 22 more of this, but you possibly know what I mean
  },
 ],
 "intervalType": 4,
 "average": 0.21,
 "fromDate": "2022-11-19T23:00:00Z",
 "tillDate": "2022-11-20T22:59:59.999Z"
}

if tried to find it out by myself by the Wiki on: https://www.domoticz.com/wiki/DzVents:_ ... _scripting but have no idea maybe someone can give me an idea

[boum]

Why would you want to save the JSON? Why not parsing it and storing only the values you want (price?)?
You can also just store the string if you want/have to parse it later.

[waltervl]

Here you have an example to save prices from a Json into devices. viewtopic.php?t=30004
Later you can do some statistics on this.

[MdeBoer]

Here you have an example to save prices from a Json into devices. viewtopic.php?t=30004
Later you can do some statistics on this.

Thanks wil take look at it!

[MdeBoer]

Why would you want to save the JSON? Why not parsing it and storing only the values you want (price?)?
You can also just store the string if you want/have to parse it later.

What I like to build is, once a day, in this case every day at 15:00 new prices of electricity come out for the day ahead (hour prices). i like to know which prices that are and choose which hour is the best for doing washes and other high use of electicity activities.

[willemd]

I can't help you yet but I plan to do something similar over the next week(s) and will share my experiences. Currently I have a very cheap fixed energy contract but I want to get insight into dynamic pricing contracts for the future, starting with electricity.

I plan to
1) take the 24hr future price data from the EU transparancy site via their API, see guide here : https://transparency.entsoe.eu/content/ ... Guide.html
2) load that data into historic variables in Domoticz (or alternatively custom devices), since Domoticz does not have any other options to load future prices
3) create one device that shows the actual current price, update once per hour from the historic variables onto the device using a dzvents script.
4) calculate my energy costs with that price and compare it with my current contract (might need additional variables or devices for energy supplier surcharges, tax etc.)

For step2 I will look into the domoticz features available. I assume there is something available (openURL ? JSON?), if not I will write either a python program or use shell scripts to transform and load data.

Once the data is in, addditional scripts can be made for optimisation of usage and delivery.

I will keep you posted.

Where do you get your data? Energy supplier?

[MdeBoer]

Hey nice! I have similar plans for the future. I also want to shutdown my solar system if prices are negative and I have my battery full. Future idea, I have no battery yet.

My code so far:

Code: Select all

return {
    on =    {
                    timer           = {"at 00:03"},
                    httpResponses   = {"Energyzero"}
            },
    
    data =  {
            dayprices               = { history = true, maxItems = 24 }
            },

    logging =   {   
                    level           =   domoticz.LOG_DEBUG, -- set to LOG_ERROR when script works as expected
                    marker          =   Energyzero
                },
            
    execute = function(dz, item)
        
        local dayprices = dz.data.dayprices
        
        if (item.isTimer) then
            
            if (item.trigger == "at 00:03") then
                local fromDate  = (dz.time.rawDate)
                local tillDate  = (dz.time.rawDate)
                local interval  = "4"
                local usageType = "1"
                local inclBtw   = "true"
            
                -- /v1/energyprices?fromDate=2022-11-20T00%3A00%3A00.000Z&tillDate=2022-11-20T23%3A59%3A59.999Z&interval=4&usageType=1&inclBtw=true
                local URLString = "https://api.energyzero.nl/v1/energyprices?fromDate=" .. fromDate .. "T00%3A00%3A00.000Z&tillDate=" .. tillDate
                .. "T23%3A59%3A59.999Z&interval=" .. interval .. "&usageType=" .. usageType .. "&inclBtw=" .. inclBtw
                
                dz.openURL({
                    url = URLString,
                    method = 'GET',
                    callback = "Energyzero"
                })
            end
        
        elseif (item.isHTTPResponse) then
            if (item.ok) then
                if (item.isJSON) then
                    -- dz.utils.dumpTable(item.json) -- Only for debug purposes
                    
                    dayprices.reset()
                    
                    for id = 1, 24 do
                        dz.log(item.json.Prices[id].price, dz.LOG_DEBUG)
                        dayprices.add(item.json.Prices[id].price)
                    end
                end
            else
                dz.log("No valid response from Energyzero " .. item.statusCode, dz.LOG_ERROR)
                dz.notify("Telegram BOT", "Energyzero API Failed. No valid response from server", dz.PRIORITY_EMERGENCY, dz.NSS_TELEGRAM)
            end
            -- dz.log(item, dz.LOG_DEBUG)
        end
    end
}

API I use: https://api.energyzero.nl/v1/energyprices (ANWB energie)

[willemd]

Wow, you already got it working. That's great. I re-used part of your script. Thanks.

I am doing the same thing but with XML data. I had a lot of trouble getting the data fields out of the response (due to lack of documentation and examples) but now managed to do it. I will show this in a separate thread in order not to hijack your thread here.

[MdeBoer]

Hello, I'm a little further now:

Code: Select all

return {
    on =    {
                    timer           = {"at 00:02", "at *:03"},
                    httpResponses   = {"Energyzero"}
            },
    
    data =  {
            dayprices               = { history = true, maxItems = 24 }
            },

    logging =   {   
                    level           =   domoticz.LOG_DEBUG, -- set to LOG_ERROR when script works as expected
                    marker          =   Energyzero
                },
            
    execute = function(dz, item)
        
        local dayprices = dz.data.dayprices
        
        local alert = dz.devices("Next Hour kWh Price")
        
        if (item.isTimer) then
            
            if (item.trigger == "at 00:02") then
                local fromDate  = (dz.time.rawDate)
                local tillDate  = (dz.time.rawDate)
                local interval  = "4"
                local usageType = "1"
                local inclBtw   = "true"
            
                -- /v1/energyprices?fromDate=2022-11-20T00%3A00%3A00.000Z&tillDate=2022-11-20T23%3A59%3A59.999Z&interval=4&usageType=1&inclBtw=true
                local URLString = "https://api.energyzero.nl/v1/energyprices?fromDate=" .. fromDate .. "T00%3A00%3A00.000Z&tillDate=" .. tillDate
                .. "T23%3A59%3A59.999Z&interval=" .. interval .. "&usageType=" .. usageType .. "&inclBtw=" .. inclBtw
                
                dz.openURL({
                    url = URLString,
                    method = 'GET',
                    callback = "Energyzero"
                })
            
            elseif ((item.trigger == "at *:03") and (dayprices.size == 24)) then
                
                local Price_Low  = dayprices.min()
                local Price_High = dayprices.max()
                local Price_Avg  = dayprices.avg()
                
                local now = dz.time.rawDate .. ", " .. dz.time.rawTime .. ': '
                local alertLevel = dz.ALERTLEVEL_RED
                
                local Price_Now = dayprices.get(dz.time.hour + 1).data -- + 1 because 00:00 = 1 01:00 = 2 enz..
                
                if (Price_Now > Price_Avg) then
                    alert.updateAlertSensor(alertLevel, now .. Price_Now)
                else
                    alertLevel = dz.ALERTLEVEL_GREEN
                    alert.updateAlertSensor(alertLevel, now .. Price_Now)
                end
                    
                -- debug logging
                dz.log(Price_Now .. " Atm Price", dz.LOG_DEBUG)
                dz.log(Price_Low .. " Lowest price", dz.LOG_DEBUG)
                dz.log(Price_High .. " Highest price", dz.LOG_DEBUG)
                dz.log(Price_Avg .. " Avg Price", dz.LOG_DEBUG)
            end
        
        elseif (item.isHTTPResponse) then
            if (item.ok) then
                if (item.isJSON) then
                    -- dz.utils.dumpTable(item.json) -- Only for debug purposes
                    
                    dayprices.reset()
                    
                    for id = 24, 1, -1 do   -- we first receive the 00:00 hour then 01:00. Because of shifting 00:00 ends on 24
                        dz.log(item.json.Prices[id].price, dz.LOG_DEBUG)
                        dz.log(item.json.Prices[id].readingDate, dz.LOG_DEBUG)
                        dayprices.add(item.json.Prices[id].price)
                    end
                end
            else
                dz.log("No valid response from Energyzero " .. item.statusCode, dz.LOG_ERROR)
                dz.notify("Telegram BOT", "Energyzero API Failed. No valid response from server", dz.PRIORITY_EMERGENCY, dz.NSS_TELEGRAM)
            end
            -- dz.log(item, dz.LOG_DEBUG)
        end
    end
}

Keep you posted

[willemd]

I am launching my script now at 23:15 to get the prices for the next day. In addition I launch it every hour to copy the price from the historic variables into a device showing the current price. Still need to add the cost calculation for actual usage (after usage is known but before new price is loaded)

Code: Select all

if (item.isTimer) then
	        if (item.trigger == 'every hour') then
	            -- copy dynamic price to device holding the current price
	            local VarIndex=24-tonumber(os.date("%H")) -- last price of day is index 1
	            local CurrentPrice=tonumber(ElDayPrices.get(VarIndex).data)/1000 -- from EURO/MWh to EURO/kWh
	            domoticz.log('Current Dynamic price EURO/kWh :' .. CurrentPrice,domoticz.LOG_INFO)
	            domoticz.devices(idxCurrentDynamicPrice).updateCustomSensor(CurrentPrice)
	        else     
    	        -- section to launch the API get request 
    	        etc. etc. etc.

[willemd]

When I compare the data output of your api with the data published on their website (and others) it looks like the data is shifted by 1 hour, i.e. the first hour of the day is missing in the API data (or: website data is UTC+1, API data is UTC)