Reading Custom Sensor fails

[Toulon7559]

'Soft' errors are the worst, and cause most headscratching ......

In lua-scripts (because of the simple structure) I frequently use script-lines for the read-out for Customer Sensors of svalues like

Code: Select all

Dust0 = tonumber(otherdevices_svalues[Dust0_sensor])  -- virtual, singlevalue dummy0 for GP2Y10

Usually that works, but sometimes (and sometimes persistent) get the errorreport that (taking the above code as example)
either Dust0_sensor is declared a 'nil'
or (in the subsequent use of Dust0) Dust0 is declared 'nil'

Which is surprising, because in the widget or in the svalue (under Setup/Options/Events) a value is present with a good validity time.
And sometimes in one script such line is OK, and in another script continuously setting alarm.

Somebody knowing the reason (and a remedy)?

[Toulon7559]

Example error report

Code: Select all

attempt to index global 'otherdevices_svalues' (a nil value)

Apparently the debugger considers otherdevices_svalues not as an instruction, but as a variable.
But the 'vague' aspect is that most scripts run without errors, and this kind of error reports is only appearing at some scripts.

Still looking for an alternative method to read the svalues.
Somebody a hint?

[waaren]

Still looking for an alternative method to read the svalues.
Somebody a hint?

Sorry no hint; I may have missed something but looking in the source I don't think that domoticz sends more info to the Lua event system than the known tables.

• Can you share the complete script on the forum that might help in understanding what could go wrong.

• Is Dust0_sensor the name of your device or is this the name you get as the key from

Code: Select all

for key, value in pairs(devicechanged) do

[Toulon7559]

@waaren

As example a piece of script which has the 'questionable/problematic' instruction at line 69.
If the script is commanded at Puttly's CLI, then the error report is

Code: Select all

lua: ...lua/script_time_AWEKAS_DirectLink_upload01j_RPIB.lua:69: attempt to index global 'otherdevices_svalues' (a nil value)
stack traceback:
        ...lua/script_time_AWEKAS_DirectLink_upload01j_RPIB.lua:69: in main chunk
        [C]: ?

These scriptlines do not call a Custom Sensor, but they call 'normal' Devices of Domoticz [?????] .
If that line 69 is commented (and also the next lines 70 and 71), then the 'problem' pops up for line 74.
Strange thing is that a very comparable script (also uploading to AWEKAS) applying the same devices has no problems at all:
that seems reason to believe that it is something very subtle in the operation of lua-instructions and handling of devices.

Code: Select all

-- AWEKAS PWS upload script, version AWEKAS_01j_RPIB, Layout for AWEKAS_C = MyAWEKAS DirectLink according to API_Push protocol_rev2.1 of 2.1.2015
-- (c)2017~2019 adapted by Toulon7559 for periodic upload of Nexus-info received via RFXCom, 
-- using local time, supplemented by pressure-info from a BMP180 sensor, by UVI from UVN800/ML8511 and light-info from calculations in Domoticz.

-- Line 05, Assignment of Domoticz' sensors & devices
Outside_Temp_Hum0 = 'TFA1_Temp_RV_Hoog' -- Main sensorset T&H, read by RFXCom
Outside_Temp_Hum1 = 'TFA4_Temp_RV_West' -- Reference & Backup sensorset T&H, read by RFXCom
Barometer = 'BMP180_Temp_Baro' -- local BMP180, through I2C at Raspberry
RainMeter = 'TFA_Nexus_Neerslag' -- Main sensorset, read by RFXCom
WindMeter = 'TFA_Nexus_Wind' -- Main sensorset, read by RFXCom
UVMeter0 = 'UVN800' -- UVI-sensor, Virtual Device copied from Domoticz2
UVMeter1 = 'SolarRadiation_UV' -- Custom Sensor/ Virtual Device, for derived result of calculation for Lux-value
UVMeter2 = 'UVI_ML8511' -- UVI-sensor, Virtual Device copied from Domoticz2
LuxMeter = 'Lux_Calc' -- Custom Sensor / Virtual Device, calculated Lux-value (placeholder/fallback by the script from jmleglise)
-- Sunshine = ' ' -- To be defined
Soiltemp = 'TFA2_Temp_RV_Laag' -- Secondary sensorset T&H, Grass-height, read by RFXCom

-- Line 18, AWEKAS Settings, Prepare for AWEKAS_URL, Pos1+Pos2 and set Interval
baseurl = "http://www.awekas.at/extern/eingabe_pruefung.php?val="
ID = "<xxxxx>"                             -- Insert here your AWEKAS-username
PASSWORD = "<yyyyyyy>"                     -- Insert here your AWEKAS-password CONVERTED to md5 hashcode!
Interval = 10; Offset = 7                  -- Interval & Offset in minutes for running the script

-- Line 24, AWEKAS protocol, Settings for Pos14+Pos23
Language = "nl"        -- Alternatives: de (=german), en (=english), it (=italian), fr (=french)
SoftwareID="lua_2.zz"  -- Specific/unique code from AWEKAS for this lua-script

-- Line 28, Auxiliary/ conversion Function(s)
-- inputs with metric units do not require other conversion routines

local function luxtoWattsm2(Lx)
--   if Lx == nil then return 0 end
   return Lx * 0.0079
end

-- Line36, Get Current date & time from os as date.year, date.month, date.day, date.hour, date.min, date.sec
date = os.date("*t")
year = date.year
month = date.month
if month < 10 then month = "0" .. month
end
day = date.day
if day < 10 then day = "0" .. day
end
hour = date.hour
hourvalue = hour
if hour < 10 then hour = "0" .. hour
end
minutes = date.min
if minutes < 10 then minutes = "0" .. minutes
end
seconds = date.sec
if seconds < 10 then seconds = "0" .. seconds
end

-- Line56, AWEKAS protocol, Data for Pos3+Pos4 
timestring = day .. "." .. month .. "." .. year .. ";" .. hour .. ":" .. minutes

-- Line 59, Start script and compile first part of the URL-string
-- Start = check against Current os.date as date.year, date.month, date.day, date.hour, date.min, date.sec
date = os.date("*t")
-- if (date.min % Interval == Offset) then
    AWEKAS_URL= baseurl .. ID .. ";" .. PASSWORD .. ";" .. timestring
-- end

-- Line 66, AWEKAS protocol, Compile & Add Pos5+6
-- Line 67, Data Extraction from svalues for TFA1_Temp_RV_Hoog
if Outside_Temp_Hum0 ~= '' then
    sTempTFA1, sRVTFA1 = otherdevices_svalues[Outside_Temp_Hum0]:match("([^;]+);([^;]+)")
    sTemp_out0 = tonumber(sTempTFA1)
    sRV_out0 = tonumber(sRVTFA1)
else
-- Line 73, Data Extraction from svalues for TFA4_Temp_RV_West
    sTempTFA4, sRVTFA4 = otherdevices_svalues[Outside_Temp_Hum1]:match("([^;]+);([^;]+)")
    sTemp_out1 = tonumber(sTempTFA4)
    sRV_out1 = tonumber(sRVTFA4)
end


<the subsequent lines of the script are not relevant to the problem-description>

[waaren]

As example a piece of script which has the 'questionable/problematic' instruction at line 69.

No clue why othervalues and/or othervalues_svalues do (sometimes) not exist at that location in the code. Can you try to replace that part of the code with this?

Code: Select all

if Outside_Temp_Hum0 ~= '' and otherdevices_svalues and  otherdevices and otherdevices[Outside_Temp_Hum0] then
    sTempTFA1, sRVTFA1 = otherdevices_svalues[Outside_Temp_Hum0]:match("([^;]+);([^;]+)")
    sTemp_out0 = tonumber(sTempTFA1)
    sRV_out0 = tonumber(sRVTFA1)
elseif Outside_Temp_Hum1 ~= '' and otherdevices_svalues and otherdevices and otherdevices[Outside_Temp_Hum1] then
-- Line 73, Data Extraction from svalues for TFA4_Temp_RV_West
    sTempTFA4, sRVTFA4 = otherdevices_svalues[Outside_Temp_Hum1]:match("([^;]+);([^;]+)")
    sTemp_out1 = tonumber(sTempTFA4)
    sRV_out1 = tonumber(sRVTFA4)
else
	-- decide here what should happen now. Should the code just hand back control then
	-- return 
	-- Or set the sTemp_out0 and sRV_out0 to a default value and continue ?
end

I know this will not solve the issue but just might help to work around it.

[Toulon7559]

@waaren

Thanks!

Making progress, because with your proposed construction passing the 'if-then'-checks for sTemp_out and for sRV_out, but apparently for a practical solution in the subject script such a kind of 'if' is required for every next variation of 'otherdevices'.

Seems strange, because nowhere documented, and (see earlier messages) not required for all scripts.

Perhaps better to switch to dzVents or to Python .......

[Toulon7559]

Just thinking ...

Could it be that the problem is not in the call 'otherdevices', but in the definition of the devices?

In the 'problematic' lua-script, the original definition of the devices is (based on working examples)

Code: Select all

-- Line 05, Assignment of Domoticz' sensors & devices
Outside_Temp_Hum0 = 'TFA1_Temp_RV_Hoog' -- Main sensorset T&H, read by RFXCom
Outside_Temp_Hum1 = 'TFA4_Temp_RV_West' -- Reference & Backup sensorset T&H, read by RFXCom
Barometer = 'BMP180_Temp_Baro' -- local BMP180, through I2C at Raspberry
RainMeter = 'TFA_Nexus_Neerslag' -- Main sensorset, read by RFXCom
WindMeter = 'TFA_Nexus_Wind' -- Main sensorset, read by RFXCom
UVMeter0 = 'UVN800' -- UVI-sensor, Virtual Device copied from Domoticz2
UVMeter1 = 'SolarRadiation_UV' -- Custom Sensor/ Virtual Device, for derived result of calculation for Lux-value
UVMeter2 = 'UVI_ML8511' -- UVI-sensor, Virtual Device copied from Domoticz2
LuxMeter = 'Lux_Calc' -- Custom Sensor / Virtual Device, calculated Lux-value (placeholder/fallback by the script from jmleglise)
-- Sunshine = ' ' -- To be defined
Soiltemp = 'TFA2_Temp_RV_Laag' -- Secondary sensorset T&H, Grass-height, read by RFXCom

Comparison helps.
Changed that section of the script to

Code: Select all

-- Line 05, Assignment of Domoticz' sensors & devices
local Outside_Temp_Hum0 = 'TFA1_Temp_RV_Hoog' -- Main sensorset T&H, read by RFXCom
local Outside_Temp_Hum1 = 'TFA4_Temp_RV_West' -- Reference & Backup sensorset T&H, read by RFXCom
local Barometer = 'BMP180_Temp_Baro' -- local BMP180, through I2C at Raspberry
local RainMeter = 'TFA_Nexus_Neerslag' -- Main sensorset, read by RFXCom
local WindMeter = 'TFA_Nexus_Wind' -- Main sensorset, read by RFXCom
local UVMeter0 = 'UVN800' -- UVI-sensor, Virtual Device copied from Domoticz2
local UVMeter1 = 'SolarRadiation_UV' -- Custom Sensor/ Virtual Device, for derived result of calculation for Lux-value
local UVMeter2 = 'UVI_ML8511' -- UVI-sensor, Virtual Device copied from Domoticz2
local LuxMeter = 'Lux_Calc' -- Custom Sensor / Virtual Device, calculated Lux-value (placeholder/fallback by the script from jmleglise)
-- local Sunshine = ' ' -- To be defined
local Soiltemp = 'TFA2_Temp_RV_Laag' -- Secondary sensorset T&H, Grass-height, read by RFXCom

Result: no more complaints related to 'otherdevices' and it's variations!

However, the debugger at Putty's CLI now complains about the next section of the script

Code: Select all

-- Line 94, Output of Temp&RelHum
if ((Outside_Temp_Hum0 ~= '') or (Outside_Temp_Hum1 ~= '')) then
   AWEKAS_URL = AWEKAS_URL .. ";" .. string.format("%3.1f", (sTemp_out))
   AWEKAS_URL = AWEKAS_URL .. ";" .. string.format("%.0f", (sRV_out))
else
   AWEKAS_URL = AWEKAS_URL .. ";;"
end

with error-report

Code: Select all

lua: ...ua/script_time_AWEKAS_DirectLink_upload01j2_RPIB.lua:96: bad argument #2 to 'format' (number expected, got nil)
stack traceback:
        [C]: in function 'format'
        ...ua/script_time_AWEKAS_DirectLink_upload01j2_RPIB.lua:96: in main chunk
        [C]: ?

Same strange thing:
the construction of the scriptlines is 100% according to working examples, but apparently not working in this script when checked from Putty's CLI.
When looking in Domoticz' Log, I see different error reports for different scriptlines.
More and more getting the feeling of missing some aspect of lua-scripting/-debugging, but cannot put my finger on it ......

[waaren]

More and more getting the feeling of missing some aspect of lua-scripting/-debugging, but cannot put my finger on it ......

Maybe I misread your postings and hopefully I am wrong but I get the idea that you execute the script from the Lua interpreter at the CLI ? Is this the case ?

[Toulon7559]

@waaren

Dual, combined approach for testing of lua-scripts:
1) for a quick/initial testing I run a selected script from Putty's CLI (quite often at sudo-level) and let the interpreter do it's job
All quoted error reports originate from this type of testing.
2) for 'normal' speed 'operational' testing, I let the lua-script run under control of Domoticz' Events script-timing and read Domoticz' Log to see the results (and errors)

E.g. for testing of RRDTool scripts I am aware that subtle differences are applicable for calling a file in one or the other testing method, but not yet touched/found such aspect for lua-scripts.

[waaren]

Dual, combined approach for testing of lua-scripts:
1) for a quick/initial testing I run a selected script from Putty's CLI (quite often at sudo-level) and let the interpreter do it's job
All quoted error reports originate from this type of testing.

Please be aware that while testing using the Lua interpreter from the CLI, The following Lua tables are not available to your scripts.

Code: Select all

otherdevices
otherdevices_lastupdate
otherdevices_svalues
uservariables
uservariables_lastupdate
timeofday
globalvariables

[Toulon7559]

@waaren

Useful information, but at the same time puzzling, because many of my scripts (using these elements) apparently run without problems from Putty's CLI .....

It raises 2 questions:
1) knowing these 'exclusions', that 'error-free' testing from Putty's CLI ís surprising. Explanation?
2) if taking this info as a warning, is a 'quick-test' method available for scripts using these 'excluded' elements?

[waaren]

It raises 2 questions:
1) knowing these 'exclusions', that 'error-free' testing from Putty's CLI ís surprising. Explanation?

The fact that these tables are not declared does not mean the Lua interpreter will generate an error. When referred to such a table-name, Lua continue with it as being nil. An error will only be generated when you try to refer to a value or index of such a table. Probably easier to explain when you provide me with an example.

2) if taking this info as a warning, is a 'quick-test' method available for scripts using these 'excluded' elements?

I use a second domoticz instance for that kind of testing. Normally on a test system but when really needed to test with hardware (zwave stick/ rflink / P1 cable) connected to my production system, I temporary stop my production domoticz instance and use a test instance.
My production instance is located at /opt/domoticz and when really no other option, I copy this completely to /usr/local/domotica/test and test it there.

[Toulon7559]

@waaren

For 'normal' lua-testing thinking approx. along the same lines.
Running 'production' on one Raspberry, a script-under-test initially is running on a second Raspberry (= Set3 of my configuration), which is dedicated to side-jobs like testing.
To facilitate that setup, e.g. for data-transfer often using json- and xml-files stored at a commonly accessible location.
When sufficiently tested, the script-under-test shifts to 'production'.

'Lessons' learned from this thread:
1) (obviously) the lua-interpreter at Putty's CLI is not 'the real thing', but where to find and how to explain the differences?
2) 'real' lua-testing requires a 'real' lua-instance, with related required runtime for try&error

Would use of dzVents or Python be 'better', or just 'different'?