Reading XML with LUA?

[Toulon7559]

Another challenge.

Found a URL-string to directly read an XML-file from my SAJ-inverter:

Code: Select all

http://<IP-address>/real_time_data.xml

When the inverter is operational, the resulting xml-file looks like

Code: Select all

<real_time_data><state>Normal</state><v-grid>228.8</v-grid><i-grid>0.25</i-grid><f-grid>49.98</f-grid><p-ac>35</p-ac><temp>20.4</temp><e-today>0.0</e-today><t-today>1.5</t-today><e-total>4576.2</e-total><CO2>4562.47</CO2><t-total>14437.1</t-total><v-pv1>234.9</v-pv1><i-pv1>0.15</i-pv1><v-pv2>-</v-pv2><i-pv2>-</i-pv2><v-bus>361.2</v-bus></real_time_data>

Obviously tried to tune the script version 04 from the previous message, but get a persistent error on line 29 for 'attempt to perform arithmetic on nil value': apparently a scriptline further down in the script (somewhere at/after line 60 etc.?) makes a function-call, but with a faulty inputvalue for the string-search.

Somebody seeing what's wrong in the script-version tuned to the example xml-file?

Code: Select all

------------------------------------------------------------------------------
-- Version SAJ/00 20181223
--
-- Domoticz lua script to convert XML output directly read from SAJ_inverter
-- Reads the status based on the unique IP of the Inverter and passes it
-- to virtual sensor(s) in Domoticz
--
------------------------------------------------------------------------------
-- Source-script from Domoticz-forum, adapted by Toulon7559 (c)2018
------------------------------------------------------------------------------
-- Line 11: Start of Script
commandArray = {}
-- Line 13: Definition of XML Capture
function XML_Capture(cmd,flatten)
   local f = assert(io.popen(cmd, 'r'))
   local s = assert(f:read('*a'))
   f:close()
   if flatten  then
      s = string.gsub(s, '^%s+', '')
      s = string.gsub(s, '%s+$', '')
      s = string.gsub(s, '[\n\r]+', ' ')
   end
   return s
end
-- Line 25: Definition of XML dissect
function GetValueFromXML(XMLvalue)
   -- Extract the start/end-positions of the values within the substrings
   local XMLvalueLength=string.len(XMLvalue)+2
   local XMLvalueStartPos=string.find(XML_string,"<" .. XMLvalue .. ">")+XMLvalueLength
   local XMLvalueEndPos=string.find(XML_string,"</" .. XMLvalue .. ">")
   -- Get the value substring and convert it to a number
   local ReturnValue=tonumber(string.sub(XML_string,XMLvalueStartPos,XMLvalueEndPos-1))
   print(XMLvalue .. "=" .. ReturnValue)
   return ReturnValue
end

-- Line 37: Set debug to true to print to log file.
debug = true

-- Line 40: Define the idx of your virtual sensors - i.e. what you have called your sensors and their IDX numbers

PE_IDX = 446
-- VPV_IDX =
-- IAC_IDX =
-- Temp_IDX =
-- Hours_IDX =

-- Line 48: Define your SAJ Inverter device IP@

Device_IP = "192.168.0.109"

if debug == true then
        print("Reading values from: 'http://"..Device_IP.."/real_time_data.xml'")
end

-- Line 56: Read the XML data from the device

XML_string=XML_Capture("curl -s 'http://"..Device_IP.."/real_time_data.xml'",1)

valid = string.find(XML_string, "<real_time_data>")    -- check we are looking in the right place

    if debug == true then
        print(XML_string)
    end

    if valid == nil then
        print ("Bad XML status read - info NOT updated")
    else

-- Line 70: Choose in the XML_string the info-fields (substrings) based on their labels
-- Extract the values from the substrings and (if required) scale for upload to Domoticz

-- Line 73: read AC-power string
    power_actual = GetValueFromXML("p-ac")

-- Line 76: read temperature string
    temp = GetValueFromXML("temp")

-- Line 79: read PV-voltage string
    vpv1 =  GetValueFromXML("v-pv1")

-- Line 82: read PV-current string
    ipv1 =  GetValueFromXML("i-pv1")

-- Line 85: read day-energy string
    energy_day= GetValueFromXML("e-today") * 1000 -- correction for energy in Wh as required in Domoticz
    print("energy_day=" .. energy_day)

-- Line 89: read life-energy string
    energy_life= GetValueFromXML("e-total") * 1000 -- correction for energy in Wh as required in Domoticz
    print("energy_life=" .. energy_life)

-- Line 93: read hours_total string
    hours_total = GetValueFromXML("t-total")

-- Line 96: Upload Power & Energy to Virtual Device at Domoticz

    commandArray[1] = {['UpdateDevice'] = PE_IDX.."|0|"..power_actual..";"..energy_life}  -- send updated values to Domoticz

        if debug == true then
            print("power/energy returned = ".."'"..power_actual.."/"..energy_life.."'")
        end

    end

return commandArray

[Toulon7559]

Further experiments after this message did not yet give a positive result.

Still error reports for the script reading the SAJ-inverter over the IP-address.
But the experiments may give a hint to the 'culprit'!

a) If in line 74 the argument is ("p-ac") or ("<p-ac>"), then the error report in Domoticz' log is:
line 29, attempt to perform arithmetic on nil value

b) If in line 74 the argument is (p-ac), then the error report in Domoticz' log is:
line 74, attempt to perform arithmetic on global 'p' (a nil value)

c) If in line 74 the argument is (<p-ac>), then the error report in Domoticz' log is:
line 74, unexpected symbol near '<'

Combining these error reports, it seems to point to the function definition lines starting at line 25:
in some way the local function for extraction has a problem with the input argument XMLvalue
Line 28 seems always OK, but line 29 is not performing the expected 'translation'.

With experiments splitting the various scriptlines and variation of the contents of XMLvalue, I deduct that the problem is in
Original line =

Code: Select all

local XMLvalueStartPos=string.find(XML_string,"<" .. XMLvalue .. ">") 

Test-version =

Code: Select all

local XMLvalueStartPos=string.find(XML_string,XMLvalue) 

Whatever I try in the Test-version related to contents of XMLvalue, that line always produces an error related to XMLvalue "<p-ac>","</p-ac>" etc.
Somewhere from a) above I have the nagging feeling that in this script the result of string.find is a boolean, while a number is required.
But in version 03 of my script the function string.find produced numbers [?????]

xtenz and fellow Forum-members, any hint for a remedy in that section?

[Toulon7559]

Sometimes one has to become practical/ pragmatic ......

My script in this thread remains with unsolved questions, while this script is successfully meeting the basic requirements for read-out of the SAJ-Inverter of power, energy and temperature.
Accept that in this latter script the detailed info is not coming forward from the inverter.
Although? Somebody already making some extensions with that purpose?

Somebody with a variant of this script for a direct readout of a SOLIS-inverter (or compatible)?

[waaren]

Somebody with a variant of this script for a direct readout of a SOLIS-inverter (or compatible)?

If you are looking for an easier way to read XML; in dzVents version 2.5.1 the function fromXML has been added. It converts an XML string to a table. The XML tags become the table keys and the XML values ...

example script

Code: Select all

return {
	on = {
		timer = {'every minute' },
	},
	execute = function(dz)
	   dz.utils.dumpTable(dz.utils.fromXML('<real_time_data><state>Normal</state><v-grid>228.8</v-grid><i-grid>0.25</i-grid><f-grid>49.98</f-grid><p-ac>35</p-ac><temp>20.4</temp><e-today>0.0</e-today><t-today>1.5</t-today><e-total>4576.2</e-total><CO2>4562.47</CO2><t-total>14437.1</t-total><v-pv1>234.9</v-pv1><i-pv1>0.15</i-pv1><v-pv2>-</v-pv2><i-pv2>-</i-pv2><v-bus>361.2</v-bus></real_time_data>'))
	end
}

produces:

Spoiler: show

Code: Select all

2019-12-02 10:23:00.111 Status: dzVents: Info: ------ Start internal script: dz fromXML:, trigger: every minute
2019-12-02 10:23:00.116 Status: dzVents: > real_time_data:
2019-12-02 10:23:00.116 Status: dzVents: > CO2: 4562.47
2019-12-02 10:23:00.116 Status: dzVents: > v-bus: 361.2
2019-12-02 10:23:00.116 Status: dzVents: > i-pv1: 0.15
2019-12-02 10:23:00.116 Status: dzVents: > i-grid: 0.25
2019-12-02 10:23:00.116 Status: dzVents: > e-today: 0.0
2019-12-02 10:23:00.116 Status: dzVents: > f-grid: 49.98
2019-12-02 10:23:00.116 Status: dzVents: > state: Normal
2019-12-02 10:23:00.116 Status: dzVents: > p-ac: 35
2019-12-02 10:23:00.116 Status: dzVents: > v-pv1: 234.9
2019-12-02 10:23:00.116 Status: dzVents: > temp: 20.4
2019-12-02 10:23:00.116 Status: dzVents: > i-pv2: -
2019-12-02 10:23:00.116 Status: dzVents: > e-total: 4576.2
2019-12-02 10:23:00.117 Status: dzVents: > v-grid: 228.8
2019-12-02 10:23:00.117 Status: dzVents: > t-today: 1.5
2019-12-02 10:23:00.117 Status: dzVents: > v-pv2: -
2019-12-02 10:23:00.117 Status: dzVents: > t-total: 14437.1
2019-12-02 10:23:00.117 Status: dzVents: Info: ------ Finished dz fromXML

[Toulon7559]

@waaren

True, but if a script is working without errors, there is no real reason to touch & change, unless with very little effort.

'Devil is in details': finding those details which cause error reports, usually takes a lot of time.
Saving a lot of time & hard thinking if somebody already has a 'Complete' script reading the detailed info from the related inverter including the translation to related Domoticz' devices. A 'nearly complete' script is also an nice starter.

Building from scratch is a nice challenge, but only if needed .........