dzVents OpenURL - Error parsing json to LUA table: (invalid json string)

[javalin]

Hi,
I am trying to get the weather details from this link

The result from the openurl is in the following format:

[{"SDATA":"2022-07-12 18:00","TZ":5.9,"TP":8.2},{"SDATA":"2022-07-12 19:00","TZ":5.9,"TP":7.7},{"SDATA":"2022-07-12 20:00","TZ":5.9,"TP":7.5},{"SDATA":"2022-07-12 21:00","TZ":"NaN","TP":7.7},{"SDATA":"2022-07-12 22:00","TZ":6.3,"TP":7.7},{"SDATA":"2022-07-12 23:00","TZ":6.1,"TP":8},{"SDATA":"2022-07-13 00:00","TZ":6.1,"TP":8.2},{"SDATA":"2022-07-13 01:00","TZ":6.1,"TP":7.7},{"SDATA":"2022-07-13 02:00","TZ":5.9,"TP":8},{"SDATA":"2022-07-13 03:00","TZ":5.9,"TP":8.4},{"SDATA":"2022-07-13 04:00","TZ":5.6,"TP":8.2},{"SDATA":"2022-07-13 05:00","TZ":5.6,"TP":8},{"SDATA":"2022-07-13 06:00","TZ":5.6,"TP":8},{"SDATA":"2022-07-13 07:00","TZ":5.4,"TP":7.3},{"SDATA":"2022-07-13 08:00","TZ":5.6,"TP":7.3},{"SDATA":"2022-07-13 09:00","TZ":5.4,"TP":7.7},{"SDATA":"2022-07-13 10:00","TZ":5.6,"TP":7.7},{"SDATA":"2022-07-13 11:00","TZ":5.4,"TP":7.3},{"SDATA":"2022-07-13 12:00","TZ":5.4,"TP":8.2},{"SDATA":"2022-07-13 13:00","TZ":5.6,"TP":8.4},{"SDATA":"2022-07-13 14:00","TZ":5.9,"TP":8.4},{"SDATA":"2022-07-13 15:00","TZ":5.6,"TP":8},{"SDATA":"2022-07-13 16:00","TZ":5.9,"TP":8},{"SDATA":"2022-07-13 17:00","TZ":"NaN","TP":"NaN"},{"SDATA":"2022-07-13 18:00","TZ":"NaN","TP":"NaN"}]

Now I am trying to retrieve the specific values from the output, converting the JSON to a LUA table but I think JSON format is not valid, so a get this error:

Code: Select all

Error parsing json to LUA table: (invalid json string)

Looking in the forum, internet and the dzVents wiki but not getting any value out of it. Just working rawData doing something like this

Code: Select all

local json = item.data:gsub("%[",""):gsub("%]","")

and something like

Code: Select all

dz.utils.stringSplit(json,',')

Any ideas? Appreciate your suggestions or hints in the right direction

[waltervl]

From that link I get indeed a non valid Json, it includes html due to the last parameter in the URL.
The following link gives a valid Json: https://hidrografico.pt/json/boia.graph ... ar=3&per=1

[javalin]

Even with the new link i get an error parsing json to LUA table, would be because json structure has square brackets?

[waltervl]

Could well be but I do not know what the definition of a correct Json in context of Domoticz is. I am sorry. Perhaps someone else knows?

[mgugu]

The JSON obtained with the link from waltervl is valid according to on-line validator (https://jsonlint.com/) so it should be valid in Domoticz too.

[plugge]

Even with the new link i get an error parsing json to LUA table, would be because json structure has square brackets?

No, square brackets are not the problem.
Share your script, so that we can see what/how it is processed.

[javalin]

Thank you, this is my script:

Code: Select all

local scriptVar = 'hidrografico'
return
{
    on =
    {
        devices =
        {
           'TesteDzvents', --Script can be triggered by a device
        },
        timer =
        {
           -- 'every 2 minutes',--'at *:36',
        },
        httpResponses =
        {
            scriptVar,
        },
    },

    logging =
    {
        level = domoticz.LOG_DEBUG,
        marker = 'hidrografico',
    },
   
    data = 
    {
         --   temperatures = { history = true, maxItems = 10 },
    },


    execute = function(dz, item)
    
        --main
        if item.isTimer or item.isDevice then
            dz.openURL({
                url = 'https://hidrografico.pt/json/boia.graph.php?id_est=1005&id_eqp=1009&gmt=GMT&dtz=Europe/Lisbon&dbn=monican&par=3&per=2',
                callback = scriptVar, -- httpResponses above.
            })

        elseif item.isHTTPResponse then
            json = dz.utils.fromJSON(item.data)
            dz.utils.dumpTable(json)
        end
end
}

With the return from the HTTP call states we get first error: "Error parsing json to LUA table: (invalid json string)", and second when we try to dump the table.

I can avoid second one scripting:

Code: Select all

        
     elseif item.isHTTPResponse then
        json = item.data:gsub("%[","{"):gsub("%]","}"):gsub("%{",""):gsub("%}",""):gsub('"',""):gsub("SDATA:",""):gsub("THTP:","")
        jsone = dz.utils.stringSplit(json,',')
        local linesIndex = 1
            for keyField in pairs(jsone) do 
                    dz.log(jsone[linesIndex], dz.LOG_DEBUG )
                    linesIndex = linesIndex + 1
        end
     end
end
}
       
 

But first error from HTTP response remains.

Kind regards

[plugge]

@javalin I can confirm that I get the same error and I see nothing wrong with the script.
(I only have scripts using XML, that works.)
Anyone else any ideas?

[plugge]

With the return from the HTTP call states we get first error: "Error parsing json to LUA table: (invalid json string)"

@javalin, I tried dzVents JSON on another url (weather tutiempo.net https://api.tutiempo.net/json/?lan=en&a ... s&lid=3768) and that works without error.
So, nothing wrong with dzVents, the JSON from your hydrografico.pt is indeed not valid, even though jsonlint.com does not complain.

[jvdz]

The issue with this site is that it will generate an UTF8 file with BOM:

Schermafbeelding 2022-07-20 102755.png (9.23 KiB) Viewed 694 times

To get rid of that you can add this line into your code before doing the decode:

Code: Select all

-- strip BOM from response
json = json:match("^[^%[{]*(.*)")

This simply strips any character before [ or {.

When you only want to get rid of the BOM in case it is other data than JSON, you could also remove any possible BOM characters at the beginning of the result:

Code: Select all

json = json:match("^[\239\187\191\255\254\000]*(.*)")

Jos