Domoticz

Open source Home Automation System
https://forum.domoticz.com/

dzVents OpenURL - Error parsing json to LUA table: (invalid json string)

https://forum.domoticz.com/viewtopic.php?t=38847

Page 1 of 1

dzVents OpenURL - Error parsing json to LUA table: (invalid json string)

Posted: Wednesday 13 July 2022 18:48
by javalin
Hi,
I am trying to get the weather details from this link

The result from the openurl is in the following format:

[{"SDATA":"2022-07-12 18:00","TZ":5.9,"TP":8.2},{"SDATA":"2022-07-12 19:00","TZ":5.9,"TP":7.7},{"SDATA":"2022-07-12 20:00","TZ":5.9,"TP":7.5},{"SDATA":"2022-07-12 21:00","TZ":"NaN","TP":7.7},{"SDATA":"2022-07-12 22:00","TZ":6.3,"TP":7.7},{"SDATA":"2022-07-12 23:00","TZ":6.1,"TP":8},{"SDATA":"2022-07-13 00:00","TZ":6.1,"TP":8.2},{"SDATA":"2022-07-13 01:00","TZ":6.1,"TP":7.7},{"SDATA":"2022-07-13 02:00","TZ":5.9,"TP":8},{"SDATA":"2022-07-13 03:00","TZ":5.9,"TP":8.4},{"SDATA":"2022-07-13 04:00","TZ":5.6,"TP":8.2},{"SDATA":"2022-07-13 05:00","TZ":5.6,"TP":8},{"SDATA":"2022-07-13 06:00","TZ":5.6,"TP":8},{"SDATA":"2022-07-13 07:00","TZ":5.4,"TP":7.3},{"SDATA":"2022-07-13 08:00","TZ":5.6,"TP":7.3},{"SDATA":"2022-07-13 09:00","TZ":5.4,"TP":7.7},{"SDATA":"2022-07-13 10:00","TZ":5.6,"TP":7.7},{"SDATA":"2022-07-13 11:00","TZ":5.4,"TP":7.3},{"SDATA":"2022-07-13 12:00","TZ":5.4,"TP":8.2},{"SDATA":"2022-07-13 13:00","TZ":5.6,"TP":8.4},{"SDATA":"2022-07-13 14:00","TZ":5.9,"TP":8.4},{"SDATA":"2022-07-13 15:00","TZ":5.6,"TP":8},{"SDATA":"2022-07-13 16:00","TZ":5.9,"TP":8},{"SDATA":"2022-07-13 17:00","TZ":"NaN","TP":"NaN"},{"SDATA":"2022-07-13 18:00","TZ":"NaN","TP":"NaN"}]

Now I am trying to retrieve the specific values from the output, converting the JSON to a LUA table but I think JSON format is not valid, so a get this error:

Code: Select all

Error parsing json to LUA table: (invalid json string)
Looking in the forum, internet and the dzVents wiki but not getting any value out of it. Just working rawData doing something like this

Code: Select all

local json = item.data:gsub("%[",""):gsub("%]","")
and something like

Code: Select all

dz.utils.stringSplit(json,',')
Any ideas? Appreciate your suggestions or hints in the right direction

Re: dzVents OpenURL - Error parsing json to LUA table: (invalid json string)

Posted: Wednesday 13 July 2022 20:44
by waltervl
From that link I get indeed a non valid Json, it includes html due to the last parameter in the URL.
The following link gives a valid Json: https://hidrografico.pt/json/boia.graph ... ar=3&per=1

Re: dzVents OpenURL - Error parsing json to LUA table: (invalid json string)

Posted: Wednesday 13 July 2022 21:35
by javalin
Even with the new link i get an error parsing json to LUA table, would be because json structure has square brackets?

Re: dzVents OpenURL - Error parsing json to LUA table: (invalid json string)

Posted: Friday 15 July 2022 14:49
by waltervl
Could well be but I do not know what the definition of a correct Json in context of Domoticz is. I am sorry. Perhaps someone else knows?

Re: dzVents OpenURL - Error parsing json to LUA table: (invalid json string)

Posted: Friday 15 July 2022 15:26
by mgugu
The JSON obtained with the link from waltervl is valid according to on-line validator (https://jsonlint.com/) so it should be valid in Domoticz too.

Re: dzVents OpenURL - Error parsing json to LUA table: (invalid json string)

Posted: Friday 15 July 2022 17:04
by plugge
javalin wrote: ↑Wednesday 13 July 2022 21:35 Even with the new link i get an error parsing json to LUA table, would be because json structure has square brackets?
No, square brackets are not the problem.
Share your script, so that we can see what/how it is processed.

Re: dzVents OpenURL - Error parsing json to LUA table: (invalid json string)

Posted: Saturday 16 July 2022 16:17
by javalin
Thank you, this is my script:

Code: Select all

local scriptVar = 'hidrografico'
return
{
    on =
    {
        devices =
        {
           'TesteDzvents', --Script can be triggered by a device
        },
        timer =
        {
           -- 'every 2 minutes',--'at *:36',
        },
        httpResponses =
        {
            scriptVar,
        },
    },

    logging =
    {
        level = domoticz.LOG_DEBUG,
        marker = 'hidrografico',
    },
   
    data = 
    {
         --   temperatures = { history = true, maxItems = 10 },
    },


    execute = function(dz, item)
    
        --main
        if item.isTimer or item.isDevice then
            dz.openURL({
                url = 'https://hidrografico.pt/json/boia.graph.php?id_est=1005&id_eqp=1009&gmt=GMT&dtz=Europe/Lisbon&dbn=monican&par=3&per=2',
                callback = scriptVar, -- httpResponses above.
            })

        elseif item.isHTTPResponse then
            json = dz.utils.fromJSON(item.data)
            dz.utils.dumpTable(json)
        end
end
}
With the return from the HTTP call states we get first error: "Error parsing json to LUA table: (invalid json string)", and second when we try to dump the table.

I can avoid second one scripting:

Code: Select all

        
     elseif item.isHTTPResponse then
        json = item.data:gsub("%[","{"):gsub("%]","}"):gsub("%{",""):gsub("%}",""):gsub('"',""):gsub("SDATA:",""):gsub("THTP:","")
        jsone = dz.utils.stringSplit(json,',')
        local linesIndex = 1
            for keyField in pairs(jsone) do 
                    dz.log(jsone[linesIndex], dz.LOG_DEBUG )
                    linesIndex = linesIndex + 1
        end
     end
end
}
But first error from HTTP response remains.

Kind regards

Re: dzVents OpenURL - Error parsing json to LUA table: (invalid json string)

Posted: Saturday 16 July 2022 21:14
by plugge
@javalin I can confirm that I get the same error and I see nothing wrong with the script.
(I only have scripts using XML, that works.)
Anyone else any ideas?

Re: dzVents OpenURL - Error parsing json to LUA table: (invalid json string)

Posted: Wednesday 20 July 2022 2:19
by plugge
javalin wrote: ↑Saturday 16 July 2022 16:17 With the return from the HTTP call states we get first error: "Error parsing json to LUA table: (invalid json string)"
@javalin, I tried dzVents JSON on another url (weather tutiempo.net https://api.tutiempo.net/json/?lan=en&a ... s&lid=3768) and that works without error.
So, nothing wrong with dzVents, the JSON from your hydrografico.pt is indeed not valid, even though jsonlint.com does not complain.

Re: dzVents OpenURL - Error parsing json to LUA table: (invalid json string)

Posted: Wednesday 20 July 2022 10:30
by jvdz
The issue with this site is that it will generate an UTF8 file with BOM:
Schermafbeelding 2022-07-20 102755.png
Schermafbeelding 2022-07-20 102755.png (9.23 KiB) Viewed 701 times
To get rid of that you can add this line into your code before doing the decode:

Code: Select all

-- strip BOM from response
json = json:match("^[^%[{]*(.*)")
This simply strips any character before [ or {.

When you only want to get rid of the BOM in case it is other data than JSON, you could also remove any possible BOM characters at the beginning of the result:

Code: Select all

json = json:match("^[\239\187\191\255\254\000]*(.*)")
Jos :)
All times are UTC+02:00
Page 1 of 1

Powered by phpBB® Forum Software © phpBB Limited