dzVents error parsing json to LUA table

[Copitano]

Description: Raspbian GNU/Linux 10 (buster)
Release: 10
Codename: buster
Dz Version: 2022.2
Build Hash: eea9db734
Compile Date: 2022-11-05 13:05:35
dzVents Version: 3.1.8
Python Version: 3.7.3 (default, Jun 29 2023, 18:03:57) [GCC 8.3.0]


After Enphase found it necessary to place an unsolicited update on my Envoy, the dzVenst script that I have been using for years no longer works due to the token that is now required. So looking for an alternative solution that works under an older version of Dz (2021.1).
The reason for the latter is because it runs a Modbus RTU script that no longer functions on Dz >=2023.x viewtopic.php?p=315320#p315320 I have not been able to solve that issue yet. Conversely, the Envoy script does not run on older versions of Dz because there is still no option to enter the Enphase token <2023.x.

So I'm now looking for a solution vieuwed from the other side (di Dz 2021.1).

The end goal is that at least both scripts can run on the same version of Dz so the data from both can be used for calculations. This means the ratio between solar radiation (Modbus RTU serial to USB) and the yield of the solar panels (from the Envoy from Enphase).

I've come quite far, but am now encountering an error message that I don't understand much about.
These are the scripts.
Script1:

Code: Select all

local curlStatement = 'curl -f -k -H \'Authorization: Bearer '
local envoyUrl      = 'https://My IP'
local envoyEndPoint = '/ivp/pdm/energy'
local tokenVar      = 'MyToken'
local curlCommand   = 'leeg' 

return {
    on = {
        timer = {
            'every 1 minutes',
            'at *:08', 'at *:23',
            'at *:38', 'at *:53'
        },
    },

    logging = {
        level  = domoticz.LOG_INFO,
        marker = '003 Get PVProduction'
    },

    execute = function (domoticz)

        curlCommand = curlStatement .. tokenVar .. '\' ' .. envoyUrl .. envoyEndPoint
        domoticz.executeShellCommand ({
            command =  curlCommand,
            callback = 'productiePanelen', 
            timeout  = 20
        })
        domoticz.log (curlCommand)
        domoticz.log ('Triggering 030 Log zonnepanelen productie')

    end
}

Script2:

Code: Select all

local zonnepanelenNowIdx       = 851 -- Dummy Custom Sensor
local zonnepanelenTodayIdx     = 852 -- Dummy Custom Sensor
local zonnepanelenSevenDaysIdx = 853 -- Dummy Custom Sensor
local zonnepanelenLifetimeIdx  = 854 -- Dummy Custom Sensor

local wattsNow
local wattHoursToday
local wattHoursSevenDays
local wattHoursLifetime


return {
    on = {
        shellCommandResponses = {'productiePanelen'} 
    },

    logging = {
        level  = domoticz.LOG_INFO,
        marker = '031 Log PVProduction'
    },

   execute = function (domoticz, item)
        domoticz.log ('Triggered by 003 Get PV Production')

        if item.statusCode ~= 0 then domoticz.log ("item.statusCode: " .. item.statusCode) end 
        if item.json       ~= nil
        then
-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
            wattsNow           =                       item.json .wattsNow
            wattHoursToday     = domoticz.utils.round((item.json .wattHoursToday     /    1000), 2)
            wattHoursSevenDays = domoticz.utils.round((item.json .wattHoursSevenDays /    1000), 2)
            wattHoursLifetime  = domoticz.utils.round((item.json .wattHoursLifetime  / 1000000), 2)
-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
            domoticz.devices    (zonnepanelenNowIdx)       .updateCustomSensor (wattsNow)
            domoticz.devices    (zonnepanelenTodayIdx)     .updateCustomSensor (wattHoursToday)
            domoticz.devices    (zonnepanelenSevenDaysIdx) .updateCustomSensor (wattHoursSevenDays)
            domoticz.devices    (zonnepanelenLifetimeIdx)  .updateCustomSensor (wattHoursLifetime)
-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
            domoticz.log        ('Opbrengst Nu........: ' .. item.json.wattsNow  .. ' Watt')
            domoticz.log        ('Opbrengst Vandaag...: ' .. wattHoursToday      .. ' kWh')
            domoticz.log        ('Opbrengst 7 dagen...: ' .. wattHoursSevenDays  .. ' kWh')
            domoticz.log        ('Opbrengst Lifetime..: ' .. wattHoursLifetime   .. ' MWh')
-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
        else
            domoticz.log ('item.json == nil')
        end
    end
}

Calling the second script seems to work fine. I also see the searched data in the Dz log. Under the Error tab, that is. Devices are not updated so I can't use it as input for calculations yet.
This is the output on ERROR tab of Dz log:

Code: Select all

2024-03-24 22:34:01.147 Error: dzVents: Error: (3.1.8) 031 Log PVProduction: Error parsing json to LUA table: /home/pi/domoticz/dzVents/runtime/JSON.lua:1234: /home/pi/domoticz/dzVents/runtime/JSON.lua:808: expected comma or '}' at byte 710 of: {
2024-03-24 22:34:01.147 "production": {
2024-03-24 22:34:01.147 "pcu": {
2024-03-24 22:34:01.147 "wattHoursToday": 12516,
2024-03-24 22:34:01.147 "wattHoursSevenDays": 106246,
2024-03-24 22:34:01.147 "wattHoursLifetime": 23428351,
2024-03-24 22:34:01.147 "wattsNow": 0
2024-03-24 22:34:01.147 },
2024-03-24 22:34:01.147 "rgm": {
2024-03-24 22:34:01.147 "wattHoursToday": 0,
2024-03-24 22:34:01.147 "wattHoursSevenDays": 0,
2024-03-24 22:34:01.147 "wattHoursLifetime": 0,
2024-03-24 22:34:01.147 "wattsNow": 0
2024-03-24 22:34:01.147 },
2024-03-24 22:34:01.147 "eim": {
2024-03-24 22:34:01.147 "wattHoursToday": 0,
2024-03-24 22:34:01.147 "wattHoursSevenDays": 0,
2024-03-24 22:34:01.147 "wattHoursLifetime": 0,
2024-03-24 22:34:01.147 "wattsNow": 0
2024-03-24 22:34:01.147 }
2024-03-24 22:34:01.147 },
2024-03-24 22:34:01.147 "consumption": {
2024-03-24 22:34:01.147 "eim": {
2024-03-24 22:34:01.147 "wattHoursToday": 0,
2024-03-24 22:34:01.147 "wattHoursSevenDays": 0,
2024-03-24 22:34:01.147 "wattHoursLifetime": 0,
2024-03-24 22:34:01.147 "wattsNow": 0
2024-03-24 22:34:01.147 }
2024-03-24 22:34:01.147 }

I think I'm very close to the solution. Presumably something goes wrong after the line that says 'marker = '031 Log PVProduction' in the second script.

However, I don't know enough about dzVents or json to be able to solve this. Who sees where this goes wrong/can help solve this issue?

[willemd]

I suggest you put some extra logging statements into your script to see exactly where the error occurs.

As far as I can see the json is correctly formatted.
Not sure if it works or not, but I would not use "if item.json ~= nil". I would use "if item.isJSON".
I assume the extra space in "item.json .wattsNow" does not matter?
And should that not be "item.json.production.pcu.wattsNow" etc ?
Although I would then expect a different error message.

[Copitano]

I suggest you put some extra logging statements into your script to see exactly where the error occurs.

As far as I can see the json is correctly formatted.
Not sure if it works or not, but I would not use "if item.json ~= nil". I would use "if item.isJSON".
I assume the extra space in "item.json .wattsNow" does not matter?
And should that not be "item.json.production.pcu.wattsNow" etc ?
Although I would then expect a different error message.

This is from the status tab

Code: Select all

2024-03-25 17:19:01.289 Status: dzVents: Info: 031 Log PVProduction: ------ Start external script: Envoy6.lua: ShellCommandResponse: "productiePanelen"
2024-03-25 17:19:01.290 Status: dzVents: Info: 031 Log PVProduction: Triggered by 003 Get PV Production
2024-03-25 17:19:01.290 Status: dzVents: Info: 031 Log PVProduction: item.json == nil
2024-03-25 17:19:01.291 Status: dzVents: Info: 031 Log PVProduction: ------ Finished Envoy6.lua

Looks like the script does not even hit the "item.json .wattsNow" and goes directly from if to else d.i. the message "item.json==nil"
Changed "if item.json ~= nil" to "if item.isJSON". No difference.
And your right it should be "item.json.production.pcu.wattsNow" already changed that.

[willemd]

So for some reason it does not appear to be a correctly formatted json. That is also what the first message stated.

Can you find out what byte 710 is?

What if you paste it into a on-line json formatter?

What if you do a dumptable of item?

[habahabahaba]

May be trouble is in CURL ? It returns broken json?

I'm usualy using domoticz.openURL method - works fine.

Anyway try to put url string that you are forming directly to browser - what is the result?

[Kedi]

Put this at the beginning of script 2;

Code: Select all

domoticz.log(item.data, domoticz.LOG_INFO)

and see what is returned by the curl command.
And use an editor to see if the output is valid json, a " , ' or " } " is probably missing
Better is to put both scripts together.