getting the temp from a specific day in a month (in a lua table) [Solved]

[plugge]

Hi,
Still strugeling with lua tables...
I want to read the temp from a specific date in a month.
I have a lua table through a json call

Code: Select all

2020-11-09 23:08:00.445 Status: dzVents: !Info: JSON: {
2020-11-09 23:08:00.445 "result" :
2020-11-09 23:08:00.445 [
2020-11-09 23:08:00.445 {
2020-11-09 23:08:00.445 "d" : "2020-10-09",
2020-11-09 23:08:00.445 "ta" : 12.529999999999999,
2020-11-09 23:08:00.445 "te" : 15.0,
2020-11-09 23:08:00.445 "tm" : 10.0
2020-11-09 23:08:00.445 },
2020-11-09 23:08:00.445 {
2020-11-09 23:08:00.445 "d" : "2020-10-10",
2020-11-09 23:08:00.445 "ta" : 10.4,
2020-11-09 23:08:00.445 "te" : 13.0,
2020-11-09 23:08:00.445 "tm" : 8.0
2020-11-09 23:08:00.445 },
etc...

My expectation was that the dates would be the keys to the entries, but that is not the case. The date is simply a value in the table.
How do I access, f.e., the table for date "2020-10-09" to retrieve one of the temperatures.
Do I have to iterate through the whole table? If so, what would be the best way to do this?
TIA for any help.

[waaren]

Do I have to iterate through the whole table?

I depends on the type of table. In this case Yes because the return table is setup as an array like structure with integer keys.
You can break out of the iteration when you have found the record you need using the break statement.

It is easier to help if you share the complete script.
In this case assuming you have the the complete json in item.json you can iterate over the return table with below construct.

Code: Select all

for _, record in ipairs(item.json.result) do
    if record.d == '2020-10-09' then

        dz.log('Max temperature at ' .. record.d  .. ' was ' .. dz.utils.round(record.te, 1) ..  ' °C' , dz.LOG_FORCE)
        dz.log('Min temperature at ' .. record.d  .. ' was ' .. dz.utils.round(record.tm, 1) ..  ' °C' , dz.LOG_FORCE)
        dz.log('Avg temperature at ' .. record.d  .. ' was ' .. dz.utils.round(record.ta, 1) ..  ' °C' , dz.LOG_FORCE)
        break
    end
end

[plugge]

Thank you once more, waaren!
I don't really have a script yet, except the script which retrieves the complete json, but that was easy.
The idea is to extract yesterdays average temp after midnight to calculate "graaddagen" and store them in a temp device for later use.
If I have the script running I'll post it here.

Still wondering though why the data is not setup with the dates as the keys to the temp values.
I'm thinking about a generic lookup for values in other graphs based on dates (or time).

[waaren]

Still wondering though why the data is not setup with the dates as the keys to the temp values.

Because this type of JSON is designed for creating graphs and reports and not to extract one record.

[plugge]

Because this type of JSON is designed for creating graphs and reports and not to extract one record.

True.
Just found a lua sqlite wrapper http://lua.sqlite.org/index.cgi/home. That seems more appropriate for retrieving specific data values, but a little more overhead and work.

[waaren]

Just found a lua sqlite wrapper http://lua.sqlite.org/index.cgi/home. That seems more appropriate for retrieving specific data values.

You could use a sqlite wrapper but be aware that you are ignoring the database locks used by domoticz. In my tests with sqlite started from scripts I had some serious crashes.
As Lua is very close to C in terms of performance I would not care too much about a one dimensional loop over all return values. I strongly doubt if database access via an sqlite wrapper will be quicker in this case.

[plugge]

In my tests with sqlite started from scripts I had some serious crashes.

Oops. Thanks for the warning. Nothing's broken, so I won't fix it with the wrapper

[plugge]

Hi waaren, this is what I just scribbled to calculate the degree days, just before midnight, and store it in a dummy temp sensor.
Next job: get today's gas usage and store the division in a custom sensor m3/gd.
I hope the direct access to record 32 holds when the year changes. It saves the need to iterate.

Code: Select all

return {
	on = {
		devices = {},
		timer = {"at 23:45"
		    },
		httpResponses = {"averageTemp"  -- callback trigger
		    },
	},
	data = {},
	logging = {},
	execute = function(dz, item)
	
    local function logWrite( msg)
            dz.log( tostring(msg), dz.LOG_FORCE)
    end
	
	    
    if (item.isTimer) then
        logWrite("Calling dz temp average..")
        dz.openURL({
            -- 131 outdoor temp sensor
            url = "http://ip-adres:port/json.htm?type=graph&sensor=temp&idx=131&range=month",
            callback = "averageTemp",
        })
    end

    if (item.isHTTPResponse and item.callback == "averageTemp" and item.ok) then
        logWrite("Processing tempTable")
        local tempTable = dz.utils.fromJSON(item.data)
        local heatingstart = 18 -- standard correction factor
        local gd = 0 -- default to zero
        local today = 32 -- direct access, the current day is always the last of 32 entries, except when you first start measuring
        if (heatingstart-tempTable.result[today]["ta"]) > 0 then 
            gd = heatingstart-tempTable.result[today]["ta"]
            -- idx 267 Graaddagen dummy sensor
            dz.devices(267).updateTemperature(gd)
        end
    end

end
}

[waaren]

I hope the direct access to record 32 holds when the year changes. It saves the need to iterate.

Sorry to say but this is bad programming practice. You actually don't save anything measurable in terms of performance and it wil fail when a date is missing because domoticz could be down around 00:00 or you restored a database or you deleted a record.

[plugge]

I hope the direct access to record 32 holds when the year changes. It saves the need to iterate.

Sorry to say but this is bad programming practice. You actually don't save anything measurable in terms of performance and it wil fail when a date is missing because domoticz could be down around 00:00 or you restored a database or you deleted a record.

Very true. When I find the time, I'll change that.

[waaren]

When I find the time, I'll change that.

Below example uses lodash (standard packaged with dzVents) to do the iteration for you

Code: Select all

local scriptVar = 'averageTemp'

return
{
    on =
    {
        timer =
        {
            'at 23:45',
        },

        httpResponses =
        {
            scriptVar,
        },
    },

    logging =
    {
        level = domoticz.LOG_DEBUG, -- set to domoticz.LOG_ERROR when all OK
        marker = scriptVar,
    },

    execute = function(dz, item)
        local _ = dz.utils._           -- lodash

        if item.isTimer then
            dz.log('Calling dz temp average..', dz.LOG_DEBUG)

            dz.openURL(
            {
                -- 131 outdoor temp sensor
                url = dz.settings['Domoticz url'] ..  '/json.htm?type=graph&sensor=temp&idx=131&range=month',
                callback = scriptVar,
            })
            return
        end

        if item.isHTTPResponse and item.isJSON then
            local heatingStart = 18 -- standard correction factor
            local rt = item.json.result

            dz.log('Finding today in the result table using lodash find', dz.LOG_DEBUG)
            local today = _.find(rt, function (rt) return rt.d == dz.time.rawDate end )

            if today and ( heatingStart - today.ta ) > 0 then
                -- idx 267 Graaddagen dummy sensor
                dz.log('Average temp for today is ' .. dz.utils.round(today.ta, 2 ) .. ' °C ', dz.LOG_DEBUG)
                dz.devices(267).updateTemperature( dz.utils.round( today.ta, 2 ) )
            end
        else
            dz.log('Return table is not OK', dz.LOG_ERROR)
            dz.log(item, dz.LOG_DEBUG)
        end
    end
}

[plugge]

Below example uses lodash (standard packaged with dzVents) to do the iteration for you

Wow, elegant script. I've no experience yet with lodash.
I will try to implement it this week.
Thank you so much, waaren.