Real-time solar data : azimuth, Altitude, Lux sensor... [closed]

[MikeF]

Hi @jmleglise, this turned out to be a case of clearing the browser cache - thanks to gizmocuz's response - so I can now create the custom virtual sensors.

[marmachine]

I agree with @jmleglise the new created custom sensor is the best to use... i just can't wait for a new production update to change my temporarily workaround sensor! However despite of the workaround, it is a beautifull piece of art!

[marmachine]

Same issue again guys... right now, empty response from ogimet.com

Code: Select all

472    if string.find(synop,"Status: 500") == nil then
473        -- check the value for synop to debug
474        if( DEBUG == 1) then
475            print("| rslt = "..synop)
476        end
477        rslt = split(synop,",")
478        CodeStation = rslt[1]
479        rslt = split(synop, " "..CodeStation.. " ")
480        Trame = string.gsub(rslt[2], "=", "")
481        Trame = CodeStation .." ".. Trame
482        rslt = split(Trame, " ")
483        Octa = string.sub(rslt[3], 1, 1)  -- 3rd char is the cloud layer.  0=no cloud , 1-8= cloudy from 1 to 8 max , 9 =Fog , / = no data
484        if Octa == "/" then   -- not defined ? take the previous value
485            Octa = uservariables['octa']
486        elseif Octa == "9" then
487            Octa = 8
488        end
489    else
490        Octa = uservariables['octa']
491    end

below is my log:

2016-05-03 12:29:03.755 LUA: | Your nearest SYNOP Station for ogimet, use 06260 as WMOID value
2016-05-03 12:29:03.755 LUA: | curl "http://www.ogimet.com/cgi-bin/getsynop? ... 1605030900"
2016-05-03 12:29:03.885 LUA: | rslt =
2016-05-03 12:29:03.886 Error: EventSystem: in Virtual Lux sensor: [string "--[[ Virtual Lux sensor and other real-t..."]:480: bad argument #1 to 'gsub' (string expected, got nil)

Note: last update of my Azimuth sensor was 09:58:03 this morning by the way, so this issue is already going on for a while

When i test on ogimet website i am getting a response though (but not on the above curl!)
http://www.ogimet.com/display_synops2.p ... &send=send

Update: Okay it's back on since 13.34

[Quax1507]

I am trying to get working but it gives me error:

"Error: EventSystem: in /home/pi/domoticz/scripts/lua/script_time_SolarSensor.lua: /home/pi/domoticz/scripts/lua/script_time_SolarSensor.lua:203: attempt to perform arithmetic on a nil value"

Line 203 is:

Code: Select all

if tonumber(otherdevices_svalues['Lux'])+round(weightedLux,0)>0   -- No update ...

Sensor Lux is named correct.

Any ideas?

[marmachine]

Hi Quax1507, yes i have an idea... please read back, the issue is that the virtual (Lux) sensor doesn't have a value at first... i've posted my workaround for that here earlier!

Cheers!

[Quax1507]

Thank You marmachine!
That helped

I thought the fix was already included...

[Quax1507]

Script worked for about a day.
Now I have got a new Problem:

Error: EventSystem: in /home/pi/domoticz/scripts/lua/script_time_SolarSensor.lua: /home/pi/domoticz/scripts/lua/script_time_SolarSensor.lua:149: bad argument #1 to 'gsub' (string expected, got nil)

Line 149 is:

Code: Select all

Trame = string.gsub(rslt[2], "=", "")

Any ideas?

Edit:

Perhaps a Problem with ogimet?

Message: "Status: 500 Debe incluir al menos una fecha inicial"

What does is say?

[jmleglise]

Hi everyone,

I guess that marmachine live in Netherland. And you Quax, where are you from ? You are not very luck both of you, because It's seem that you get often no answer from ogimet. (and I never have that problem ... except between 0h- 2h am)

Could you active DEBUG =1 to check if synop is null as I suppose. (with the 6th line of this part below). And if so, I advice you to modify the script like this :

Code: Select all

	cmd='curl "http://www.ogimet.com/cgi-bin/getsynop?block='..WMOID..'&begin='..UTC..'"'
	if( DEBUG == 1) then print(cmd) end
	local ogimet=assert(io.popen(cmd))
	local synop = ogimet:read('*all')
	ogimet:close()
	if( DEBUG == 1) then print('ogimet:'..synop) end
		
	if string.find(synop,"Status: 500") == nil or (string.len(synop) > 1)    -- MODIFY HERE ! check the content of the answer of the API. 
	then

Sorry, I let you test this as I have not much time now to test myself.

[TheSpanishInq]

Nice script! Will definitely use it to control our screens when I get the hardware in place (pending funding approval ).

Regarding ogimet, is there a special reason why you subtract 1 hour from UTC? Is that because ogimet doesn't return any value after the hour until where 15 to 25 minutes in the hour?

I've made some slight modifications to the script in order for it to use UTC date and time to get a value between 12 and 2am:

Code: Select all

       -- Get  SYNOP  message from  Ogimet web  site
      --[[
       hourUTCminus1 = os.date("!%H")-1
       if string.len(hourUTCminus1) == 1 then
          hourUTCminus1 = "0" .. hourUTCminus1
       end
       UTC = os.date("%Y%m%d").. hourUTCminus1.."00" -- os.date("!%M")
       ]]--
       UTC = os.date("!%Y%m%d%H").."00"

and to avoid the 'gsub got nul' error:

Code: Select all

    if string.find(synop,"Status: 500") == nil
    then
        if string.find(synop, WMOID) ~= nil    -- first 15 to 25 minutes of the hour ogimet returns an empty string
        then
            rslt = split(synop,",")
            CodeStation = rslt[1]
            rslt = split(synop, " "..CodeStation.. " ")
            Trame = string.gsub(rslt[2], "=", "")
            Trame = CodeStation .." ".. Trame
            rslt = split(Trame, " ")
--      if( DEBUG == 1 ) then print(logprefix .. 'rslt[3]:'..rslt[3]) end
            Octa = string.sub(rslt[3], 1, 1)  -- 3rd char is the cloud layer.  0=no cloud , 1-8= cloudy from 1 to 8 max , 9 =Fog , / = no data
            if Octa == "/"
            then   -- not defined ? take the previous value
                print(logprefix .. 'Octa = /. i.e. no data available from WMOID '..WMOID)
                Octa = uservariables['octa']
            elseif Octa == "9" 
            then
                Octa = 8
            end
            commandArray[indexArray] = {['Variable:octa'] = tostring(Octa)}
            indexArray=indexArray+1
        else
            print(logprefix .. 'ogimet returned no valid data yet.')
            Octa = uservariables['octa']
        end
    else
        print(logprefix .. 'ogimet returned error 500: '..synop)
        Octa = uservariables['octa']
    end

I've also moved the update of uservariable 'octa' inside the if - end so that it only gets updated when the value has changed.
(And I use 'local logprefix = "script_time_SolarSensor: "' do distinct the messages of the script in the log, just a best practise of mine, could be made prettier by adding a log-function to the script...).

[javi77]

hi jmleglise
thanks for the great job about Real-time solar data : azimuth, Altitude, Lux sensor...
i have some issues in the script-that doesn't work:

Error: EventSystem: in /home/pi/domoticz/scripts/lua/script_time_SolarSensor.lua: /home/pi/domoticz/scripts/lua/script_time_SolarSensor.lua:21: syntax error near 'local

This is line 21:
local latitude = 32.1057430

what i miss?
hope you can give me a hand..
merci

[TheSpanishInq]

Copy/past issue? Could you post the first 30 lines from your script?

[javi77]

Hi
here is the 30 first lines of the script
i marked with XX some of my data
by the way can be the problem was how i created the virtual devices?
--[[ Virtual Lux sensor and other real-time solar data

~/domoticz/scripts/lua/script_time_SolarSensor.lua

-- Autors ----------------------------------------------------------------
V1.0 - S?bastien Joly - Great original work
V1.1 - Neutrino - Adaptation to Domoticz
V1.2 - Jmleglise - An acceptable approximation of the lux below 1° altitude for Dawn and dusk + translation + several changes to be more userfriendly.
V1.3 - Jmleglise - keep the time of black night in lasptUpdate
]]--

-- Variables to customize ------------------------------------------------
local city = "My City " -- Your city for Wunderground API
local countryCode = "XX" -- Your country code for Wunderground API
local idxLux_V ='16' -- Your virtual Lux Device ID
local idxSolarAzimuth ='18' -- Your virtual Azimuth Device ID
local idxSolarAltitude ='17' -- Your virtual Solar Altitude Device ID
local idxUserVarOcta='11' -- Your user variable ID , named octa
local wuAPIkey = "XXXXX" -- Your Weather Underground API Key
local latitude = XX.XXXXXX -- your home
local longitude = XX.XXXXXX -- your home
local altitude = XX.XX -- Your home altitude : run once in debug = 1 to found your altitude in Log and write it here
local WMOID = '12345' -- Your nearest SYNOP Station for ogimet (to get Cloud layer). Run once with debug=1 to get it in the log. (or, better, choose it there : http://www.ogimet.com/gsynop_nav.phtml.en )
local DEBUG = 1 -- 0 , 1 for domoticz log , 2 for file log
-- and customize the URL of api.wunderground around line 104 according to your country.

[TheSpanishInq]

Mmm, cut and pasted it into a script, didn't give any errors for me.
Could it be there's some 'hidden' low ASCII sign somewhere, not visible in your editor? Cleanup the spaces maybe?

[javi77]

by the way,in order to create one of hte virtual sensors i need to install a beta version rigth?
can you guide me exactly how to create the virtual devices?

[TheSpanishInq]

Don't think you need to run Beta to add a virtual sensor, not sure if that restriction exists for the Pi.
Try the forum search and the wiki. All the info is there.
http://www.domoticz.com/wiki/Hardware_S ... y_Hardware
Once you have done that you can create Virtual sensors.

[lalex]

For those who want to run this on Synology NAS using pre-build domoticz packages (not sure if folders are the same when you build yourself)

Code: Select all

--json = (loadfile "/home/pi/domoticz/scripts/lua/JSON.lua")()  -- For Linux
--json = (loadfile "D:\\Domoticz\\scripts\\lua\\json.lua")()  -- For Windows
json = (loadfile "/usr/local/domoticz/var/scripts/lua/JSON.lua")()  -- For Synology with via package installed Domoticz 

EDIT: don't forget that you have to create the JSON.lua script yourself inside the above lua folder.
https://www.domoticz.com/wiki/Lua_-_json.lua

[lalex]

The ogimet data started to fail.
After debugging found that UTC is calculated wrong.

Code: Select all

hourUTCminus1 = os.date("!%H")-1

gives 8 while I need to have 2. The ogimet WMOID have a UTC related to them. If it doesn't match you don't get data anymore.
I checked that my NAS has correctly timezone set. Which is the case.
So I changed the above line (I did it a bit more elegant with another local var, but you get the point)

Code: Select all

hourUTCminus1 = 2

The '2' to totally depends on the WMOID you are using.
To find out, goto

Code: Select all

http://www.ogimet.com/usynops.phtml.en

select your country and press 'send'

Now you get a list of WMOID and also a number in the second column for example 2.00.

enter in your web browser

Code: Select all

http://www.ogimet.com/cgi-bin/getsynop?block=<your WMOID>&begin=201608090200

(0200 is the UTC listed next to the WMOID I'm using, change accordingly to match your WMOID as explained above it is 2.00 for this example)

if you get an empty page, no good. If you get something like

Code: Select all

06375,2016,08,09,02,00,AAXX 09021 06375 07679 82603 10134 20110 30183 40209 55002 60005 75186 88/// 333 541// 60005 81/33 88/50 90710 91005 91106=

congrats!

Note, I just tried this and it worked. But I'm not sure this is the real fix as I see in the same country different UTC listed behind
different WMOID.
My script is working but not sure for how long. I'm assuming this UTC stays constant for each WMOID.

EDIT: guess I'm not fully there yet. Unblocked sun on the house but stays ~22K Lux. Can't be right.

[qwerk]

I also got the fail.
I changed the line hourUTCminus1 = os.date("!%H")-1 to hourUTCminus1 = 8
and now I get a new reading every 5 minutes. and it looks ok.

Thanks for the find


edit: hmmmm, the values are real low, maybe 8 is wrong.
edit2: so I changed it backed after confirming that os.date("!%H")-1 gets an answer when I run it in LUA. now I get responces in domoticz.

[lalex]

Yeah I'm not convinced this is the solution either as I'm now seeing 13:00 in the WMOID overview where it used to be 2:00 for some and 6:00 for others all in the same country. So it is not UTC.

I'll read the script more to understand what it is doing.

I was noticing already that the Lux value was staying very constant and far to much with a too perfect shape.

I've also checked my current location for wunderground and that was all ok.
However, the WMOID I was getting from the Debug logging was somewhere in Norway while I'm in the Netherlands.

EDIT:
I took over same modifications TheSpanishInq made and now script indeed is robust against not getting data from ogimet.
So it seems that the script may be asking for ogimet data when that is not available yet.
It takes some time for ogimet to update data for the next hour it seems like.

EDIT2: Indeed, ogimet data is being requested by script for 15:00 but ogimet website reports WMOID is still on 14:00

[lalex]

But why is WMOID having data 14:00 and is the script requesting 15:00 while it is actually 17:15 here??

EDIT: answer. Because UTC is 15:00... duh. OK. getting data again. Indeed as stated in comments script mods data is not received first 15/20 minutes.