Page 1 of 1
How to save data from JSON
Posted: Sunday 20 November 2022 21:20
by MdeBoer
Hello everyone i want to save a JSON tabel in "data" of DzVents
Code: Select all
data = {
historycalprices = { history = true, maxItems = 24 }
},
The JSON I want to save is this, only the "prices" object with price in the fist blok en the readingDate in the second
Code: Select all
{
"Prices": [
{
"price": 0.22,
"readingDate": "2022-11-19T23:00:00Z"
},
{
"price": 0.21,
"readingDate": "2022-11-20T00:00:00Z" -- it has 22 more of this, but you possibly know what I mean
},
],
"intervalType": 4,
"average": 0.21,
"fromDate": "2022-11-19T23:00:00Z",
"tillDate": "2022-11-20T22:59:59.999Z"
}
if tried to find it out by myself by the Wiki on:
https://www.domoticz.com/wiki/DzVents:_ ... _scripting but have no idea maybe someone can give me an idea
Re: How to save data from JSON
Posted: Monday 21 November 2022 0:52
by boum
Why would you want to save the JSON? Why not parsing it and storing only the values you want (price?)?
You can also just store the string if you want/have to parse it later.
Re: How to save data from JSON
Posted: Monday 21 November 2022 9:30
by waltervl
Here you have an example to save prices from a Json into devices.
viewtopic.php?t=30004
Later you can do some statistics on this.
Re: How to save data from JSON
Posted: Tuesday 22 November 2022 18:18
by MdeBoer
waltervl wrote: ↑Monday 21 November 2022 9:30
Here you have an example to save prices from a Json into devices.
viewtopic.php?t=30004
Later you can do some statistics on this.
Thanks wil take look at it!
Re: How to save data from JSON
Posted: Tuesday 22 November 2022 18:34
by MdeBoer
boum wrote: ↑Monday 21 November 2022 0:52
Why would you want to save the JSON? Why not parsing it and storing only the values you want (price?)?
You can also just store the string if you want/have to parse it later.
What I like to build is, once a day, in this case every day at 15:00 new prices of electricity come out for the day ahead (hour prices). i like to know which prices that are and choose which hour is the best for doing washes and other high use of electicity activities.
Re: How to save data from JSON
Posted: Wednesday 23 November 2022 9:49
by willemd
I can't help you yet but I plan to do something similar over the next week(s) and will share my experiences. Currently I have a very cheap fixed energy contract but I want to get insight into dynamic pricing contracts for the future, starting with electricity.
I plan to
1) take the 24hr future price data from the EU transparancy site via their API, see guide here :
https://transparency.entsoe.eu/content/ ... Guide.html
2) load that data into historic variables in Domoticz (or alternatively custom devices), since Domoticz does not have any other options to load future prices
3) create one device that shows the actual current price, update once per hour from the historic variables onto the device using a dzvents script.
4) calculate my energy costs with that price and compare it with my current contract (might need additional variables or devices for energy supplier surcharges, tax etc.)
For step2 I will look into the domoticz features available. I assume there is something available (openURL ? JSON?), if not I will write either a python program or use shell scripts to transform and load data.
Once the data is in, addditional scripts can be made for optimisation of usage and delivery.
I will keep you posted.
Where do you get your data? Energy supplier?
Re: How to save data from JSON
Posted: Wednesday 23 November 2022 18:47
by MdeBoer
Hey nice! I have similar plans for the future. I also want to shutdown my solar system if prices are negative and I have my battery full. Future idea, I have no battery yet.
My code so far:
Code: Select all
return {
on = {
timer = {"at 00:03"},
httpResponses = {"Energyzero"}
},
data = {
dayprices = { history = true, maxItems = 24 }
},
logging = {
level = domoticz.LOG_DEBUG, -- set to LOG_ERROR when script works as expected
marker = Energyzero
},
execute = function(dz, item)
local dayprices = dz.data.dayprices
if (item.isTimer) then
if (item.trigger == "at 00:03") then
local fromDate = (dz.time.rawDate)
local tillDate = (dz.time.rawDate)
local interval = "4"
local usageType = "1"
local inclBtw = "true"
-- /v1/energyprices?fromDate=2022-11-20T00%3A00%3A00.000Z&tillDate=2022-11-20T23%3A59%3A59.999Z&interval=4&usageType=1&inclBtw=true
local URLString = "https://api.energyzero.nl/v1/energyprices?fromDate=" .. fromDate .. "T00%3A00%3A00.000Z&tillDate=" .. tillDate
.. "T23%3A59%3A59.999Z&interval=" .. interval .. "&usageType=" .. usageType .. "&inclBtw=" .. inclBtw
dz.openURL({
url = URLString,
method = 'GET',
callback = "Energyzero"
})
end
elseif (item.isHTTPResponse) then
if (item.ok) then
if (item.isJSON) then
-- dz.utils.dumpTable(item.json) -- Only for debug purposes
dayprices.reset()
for id = 1, 24 do
dz.log(item.json.Prices[id].price, dz.LOG_DEBUG)
dayprices.add(item.json.Prices[id].price)
end
end
else
dz.log("No valid response from Energyzero " .. item.statusCode, dz.LOG_ERROR)
dz.notify("Telegram BOT", "Energyzero API Failed. No valid response from server", dz.PRIORITY_EMERGENCY, dz.NSS_TELEGRAM)
end
-- dz.log(item, dz.LOG_DEBUG)
end
end
}
API I use:
https://api.energyzero.nl/v1/energyprices (ANWB energie)
Re: How to save data from JSON
Posted: Thursday 24 November 2022 0:54
by willemd
Wow, you already got it working. That's great. I re-used part of your script. Thanks.
I am doing the same thing but with XML data. I had a lot of trouble getting the data fields out of the response (due to lack of documentation and examples) but now managed to do it. I will show this in a separate thread in order not to hijack your thread here.
Re: How to save data from JSON
Posted: Friday 25 November 2022 21:54
by MdeBoer
Hello, I'm a little further now:
Code: Select all
return {
on = {
timer = {"at 00:02", "at *:03"},
httpResponses = {"Energyzero"}
},
data = {
dayprices = { history = true, maxItems = 24 }
},
logging = {
level = domoticz.LOG_DEBUG, -- set to LOG_ERROR when script works as expected
marker = Energyzero
},
execute = function(dz, item)
local dayprices = dz.data.dayprices
local alert = dz.devices("Next Hour kWh Price")
if (item.isTimer) then
if (item.trigger == "at 00:02") then
local fromDate = (dz.time.rawDate)
local tillDate = (dz.time.rawDate)
local interval = "4"
local usageType = "1"
local inclBtw = "true"
-- /v1/energyprices?fromDate=2022-11-20T00%3A00%3A00.000Z&tillDate=2022-11-20T23%3A59%3A59.999Z&interval=4&usageType=1&inclBtw=true
local URLString = "https://api.energyzero.nl/v1/energyprices?fromDate=" .. fromDate .. "T00%3A00%3A00.000Z&tillDate=" .. tillDate
.. "T23%3A59%3A59.999Z&interval=" .. interval .. "&usageType=" .. usageType .. "&inclBtw=" .. inclBtw
dz.openURL({
url = URLString,
method = 'GET',
callback = "Energyzero"
})
elseif ((item.trigger == "at *:03") and (dayprices.size == 24)) then
local Price_Low = dayprices.min()
local Price_High = dayprices.max()
local Price_Avg = dayprices.avg()
local now = dz.time.rawDate .. ", " .. dz.time.rawTime .. ': '
local alertLevel = dz.ALERTLEVEL_RED
local Price_Now = dayprices.get(dz.time.hour + 1).data -- + 1 because 00:00 = 1 01:00 = 2 enz..
if (Price_Now > Price_Avg) then
alert.updateAlertSensor(alertLevel, now .. Price_Now)
else
alertLevel = dz.ALERTLEVEL_GREEN
alert.updateAlertSensor(alertLevel, now .. Price_Now)
end
-- debug logging
dz.log(Price_Now .. " Atm Price", dz.LOG_DEBUG)
dz.log(Price_Low .. " Lowest price", dz.LOG_DEBUG)
dz.log(Price_High .. " Highest price", dz.LOG_DEBUG)
dz.log(Price_Avg .. " Avg Price", dz.LOG_DEBUG)
end
elseif (item.isHTTPResponse) then
if (item.ok) then
if (item.isJSON) then
-- dz.utils.dumpTable(item.json) -- Only for debug purposes
dayprices.reset()
for id = 24, 1, -1 do -- we first receive the 00:00 hour then 01:00. Because of shifting 00:00 ends on 24
dz.log(item.json.Prices[id].price, dz.LOG_DEBUG)
dz.log(item.json.Prices[id].readingDate, dz.LOG_DEBUG)
dayprices.add(item.json.Prices[id].price)
end
end
else
dz.log("No valid response from Energyzero " .. item.statusCode, dz.LOG_ERROR)
dz.notify("Telegram BOT", "Energyzero API Failed. No valid response from server", dz.PRIORITY_EMERGENCY, dz.NSS_TELEGRAM)
end
-- dz.log(item, dz.LOG_DEBUG)
end
end
}
Keep you posted
Re: How to save data from JSON
Posted: Saturday 26 November 2022 10:21
by willemd
I am launching my script now at 23:15 to get the prices for the next day. In addition I launch it every hour to copy the price from the historic variables into a device showing the current price. Still need to add the cost calculation for actual usage (after usage is known but before new price is loaded)
Code: Select all
if (item.isTimer) then
if (item.trigger == 'every hour') then
-- copy dynamic price to device holding the current price
local VarIndex=24-tonumber(os.date("%H")) -- last price of day is index 1
local CurrentPrice=tonumber(ElDayPrices.get(VarIndex).data)/1000 -- from EURO/MWh to EURO/kWh
domoticz.log('Current Dynamic price EURO/kWh :' .. CurrentPrice,domoticz.LOG_INFO)
domoticz.devices(idxCurrentDynamicPrice).updateCustomSensor(CurrentPrice)
else
-- section to launch the API get request
etc. etc. etc.
Re: How to save data from JSON
Posted: Saturday 03 December 2022 19:12
by willemd
MdeBoer wrote: ↑Friday 25 November 2022 21:54
When I compare the data output of your api with the data published on their website (and others) it looks like the data is shifted by 1 hour, i.e. the first hour of the day is missing in the API data (or: website data is UTC+1, API data is UTC)