Solar Data script : Azimuth, Altitude, Lux

Moderator: leecollings

jmleglise
Posts: 192
Joined: Monday 12 January 2015 23:27
Target OS: Raspberry Pi / ODroid
Domoticz version: 2022.1
Location: FRANCE
Contact:

Solar Data script : Azimuth, Altitude, Lux

Post by jmleglise »

NB : Since its creation in 2015, this script has evolved into several forks and is discussed in several threads of this forum. So I merged the forks, clean up the version, rewrite again the wiki that had disappeared, and I close the other threads to centralize here the best version.
------

This LUA / dzvents script calculates in real-time the position of the sun in the sky and the Lux depending of the cloudiness. It updates 4 devices :
- Azimuth : Angle between the Sun and the north, in degree. (north vector and the perpendicular projection of the sun down onto the horizon)
- Altitude : Angle of the Sun with the horizon, in degree.
- Lux : taking account of the real time cloud layer
- Radiation in Watt/m2

So, with theses data, you may automate things depending on Lux and the position of the sun in the sky whitout investing in any hardware.

Image
Image


The calculation is based on the theoretical radiations of the sun depending on its course in the sky during the year for your location, plus the real-time cloud layer.

In input data, this script needs only 2 devices : the cloud cover and a barometer. That may be provided by internet weather services.


The script and all the documentation needed are here in the wiki : https://www.domoticz.com/wiki/Lua_dzVen ... titude_Lux


For archive purpose, the old threads of discussion of the script was here :
https://domoticz.com/forum/viewtopic.php?f=61&t=10077
https://domoticz.com/forum/viewtopic.php?f=72&t=19220
My script : https://github.com/jmleglise
RFXTRX433E: Blind Somfy RTS, Portal Somfy Evolvia, chacon IO, Oregon, PIR sensor PT2262
My Last project : Location de maison de vacances a Ouistreham vue mer
KMTronic USB relay
Chinese Z-WAVE: Neo CoolCam
User avatar
EdwinK
Posts: 1820
Joined: Sunday 22 January 2017 21:46
Target OS: Raspberry Pi / ODroid
Domoticz version: BETA
Location: Rhoon
Contact:

Re: Solar Data script : Azimuth, Altitude, Lux

Post by EdwinK »

Thanks for this :)

I don't need much, just the lux, but I guess this is the best way tp get those.
Running latest BETA on a Pi-3 | Toon® Thermostat (rooted) | Hue | Tuya | IKEA tradfri | Dashticz V3 on Lenovo Huawei Tablet | Conbee
jmleglise
Posts: 192
Joined: Monday 12 January 2015 23:27
Target OS: Raspberry Pi / ODroid
Domoticz version: 2022.1
Location: FRANCE
Contact:

Re: Solar Data script : Azimuth, Altitude, Lux

Post by jmleglise »

Thank you. I cleaned up a bit and rewrote the wiki that had disappeared.

The lux is the part I'm least satisfied with. The quality really depends on the source you use for the cloud cover. It changes everything.
- In the beginning we used ogimet.com but this source is losing weather stations.
- openweathermap for my region (Paris / france) is really bad for cloud cover. ( and for the weather most of the time.)

And on the other hand, some weather providers provide directly the radiation in watt/m2. In a way, it's the same thing. So we might not need such a complex script to get the lux. But as I said, at the moment , there is no weather provider in Domoticz reliable for cloudiness (for France). So their radiation data is not accurate.
My script : https://github.com/jmleglise
RFXTRX433E: Blind Somfy RTS, Portal Somfy Evolvia, chacon IO, Oregon, PIR sensor PT2262
My Last project : Location de maison de vacances a Ouistreham vue mer
KMTronic USB relay
Chinese Z-WAVE: Neo CoolCam
User avatar
EdwinK
Posts: 1820
Joined: Sunday 22 January 2017 21:46
Target OS: Raspberry Pi / ODroid
Domoticz version: BETA
Location: Rhoon
Contact:

Re: Solar Data script : Azimuth, Altitude, Lux

Post by EdwinK »

Just found this:

Code: Select all

2021-02-15 13:45:00.483 Error: dzVents: Error: (3.1.4) solarData 3.1: Method updateCustomSensor is not available for device "Sun Radiation" (deviceType=General, deviceSubType=Solar Radiation). If you believe this is not correct, please report.
Running latest BETA on a Pi-3 | Toon® Thermostat (rooted) | Hue | Tuya | IKEA tradfri | Dashticz V3 on Lenovo Huawei Tablet | Conbee
plugge

Re: Solar Data script : Azimuth, Altitude, Lux

Post by plugge »

EdwinK wrote: Monday 15 February 2021 13:49 Just found this:

Code: Select all

2021-02-15 13:45:00.483 Error: dzVents: Error: (3.1.4) solarData 3.1: Method updateCustomSensor is not available for device "Sun Radiation" (deviceType=General, deviceSubType=Solar Radiation). If you believe this is not correct, please report.
You used Dummy Virtual Sensor "Solar Radiation", but you should use "Custom Sensor". (I made the same error.)
User avatar
EdwinK
Posts: 1820
Joined: Sunday 22 January 2017 21:46
Target OS: Raspberry Pi / ODroid
Domoticz version: BETA
Location: Rhoon
Contact:

Re: Solar Data script : Azimuth, Altitude, Lux

Post by EdwinK »

Ah... You're right
Running latest BETA on a Pi-3 | Toon® Thermostat (rooted) | Hue | Tuya | IKEA tradfri | Dashticz V3 on Lenovo Huawei Tablet | Conbee
Jan Jansen
Posts: 229
Joined: Wednesday 30 April 2014 20:27
Target OS: Raspberry Pi / ODroid
Domoticz version: Stable
Location: The Netherlands
Contact:

Re: Solar Data script : Azimuth, Altitude, Lux

Post by Jan Jansen »

@jmleglise,

First of all, thank you! While playing with this script, I encountered an error.

Code: Select all

2021-02-16 15:00:00.233 Status: dzVents: Info: solarData 3.1: ------ Start internal script: Zon:, trigger: "every 5 minutes"
2021-02-16 15:00:00.234 Status: dzVents: Info: solarData 3.1: ================== solarData V3.1 ==================
2021-02-16 15:00:00.234 Status: dzVents: Info: solarData 3.1: Altitude:27, latitude: 51.??????, longitude: 5.?????
2021-02-16 15:00:00.235 Status: dzVents: Info: solarData 3.1: Altitude of the sun = 18.843407254741°
2021-02-16 15:00:00.235 Status: dzVents: Info: solarData 3.1: Azimuth of the sun = 217.23011507654°
2021-02-16 15:00:00.235 Status: dzVents: Info: solarData 3.1: Okta = 7.2 Cloud coverage = 90.0%
2021-02-16 15:00:00.235 Status: dzVents: Info: solarData 3.1: Direct Radiation = 93.05 W/m²
2021-02-16 15:00:00.235 Status: dzVents: Info: solarData 3.1: Total radiation = 89.27 W/m²
2021-02-16 15:00:00.235 Status: dzVents: Info: solarData 3.1: Total weighted lux = 11300.09 Lux
2021-02-16 15:00:00.236 Status: dzVents: Info: solarData 3.1: ------ Finished Zon
2021-02-16 15:00:00.574 Status: Starting automatic database backup procedure...
2021-02-16 15:00:00.236 Error: dzVents: Error: (3.1.1) solarData 3.1: An error occurred when calling event handler Zon
2021-02-16 15:00:00.236 Error: dzVents: Error: (3.1.1) solarData 3.1: /home/pi/domoticz/scripts/dzVents/generated_scripts/Zon.lua:180: attempt to perform arithmetic on a nil value (field 'lux')
2021-02-16 15:00:01.517 (Zigbee2MQTT brug) MqttClient::ping 

Regards,

Jan
plugge

Re: Solar Data script : Azimuth, Altitude, Lux

Post by plugge »

Jan Jansen wrote: Tuesday 16 February 2021 15:11

Code: Select all

2021-02-16 15:00:00.236 Error: dzVents: Error: (3.1.1) solarData 3.1: An error occurred when calling event handler Zon
2021-02-16 15:00:00.236 Error: dzVents: Error: (3.1.1) solarData 3.1: /home/pi/domoticz/scripts/dzVents/generated_scripts/Zon.lua:180: attempt to perform arithmetic on a nil value (field 'lux')
2021-02-16 15:00:01.517 (Zigbee2MQTT brug) MqttClient::ping 
The Lux device should be a Dummy Virtual Sensor "Lux", not "Custom Sensor". (It's the only exception of the four virtual sensors. The other three are "Custom Sensor".)
Jan Jansen
Posts: 229
Joined: Wednesday 30 April 2014 20:27
Target OS: Raspberry Pi / ODroid
Domoticz version: Stable
Location: The Netherlands
Contact:

Re: Solar Data script : Azimuth, Altitude, Lux

Post by Jan Jansen »

@plugge,

I just solved it myself by changing the cusom sensor to a Lux sensor. Thank you very much for your quick answer!

Regards,
Jan
jmleglise
Posts: 192
Joined: Monday 12 January 2015 23:27
Target OS: Raspberry Pi / ODroid
Domoticz version: 2022.1
Location: FRANCE
Contact:

Re: Solar Data script : Azimuth, Altitude, Lux

Post by jmleglise »

Many of you make the mistake. Look at the wiki for the creation of the devices. There is even a screenshot for the devices to be created.

https://www.domoticz.com/wiki/Lua_dzVen ... titude_Lux
My script : https://github.com/jmleglise
RFXTRX433E: Blind Somfy RTS, Portal Somfy Evolvia, chacon IO, Oregon, PIR sensor PT2262
My Last project : Location de maison de vacances a Ouistreham vue mer
KMTronic USB relay
Chinese Z-WAVE: Neo CoolCam
hestia
Posts: 357
Joined: Monday 25 December 2017 23:06
Target OS: Raspberry Pi / ODroid
Domoticz version: 2022.1
Location: Paris
Contact:

Re: Solar Data script : Azimuth, Altitude, Lux

Post by hestia »

Thanks for the update of the script.
I have some questions about it…
What is the difference between Lux and totalRadiation? In the script both are weighted.
It’s not the same unit, so the use is different?
Regarding the source for the cloud cover:
www.ogimet.com give good information but only every hours and with a delay (during the lock-down there were very few update for my site: an airport!)
Other weather services integrated in Domoticz are not so good at the present time as you shown it (openweatherMap)
The 2 services should be compared and perhaps come back to ogimet: the solar script could stay the same and another script could give the cloud cover from ogimet (I don’t know if you gave one – I could give one in this topic if needed).
Proposals regarding the script
I think there are 2 “local” missing here

Code: Select all

Cloudpercentage = domoticz.devices(idxCloudCover).percentage
okta = Cloudpercentage/12
=>

Code: Select all

local Cloudpercentage = domoticz.devices(idxCloudCover).percentage
local okta = Cloudpercentage/12.5
Why using a custom dummy instead of the Solar Radiation dummy? With a Solar Radiation dummy we could have a standard better icon...
One last proposal for the Solar Radiation would to remove the decimals:

Code: Select all

domoticz.devices(idxRadiation).updateCustomSensor(domoticz.utils.round(totalRadiation,0))
glsf91
Posts: 58
Joined: Tuesday 14 November 2017 21:56
Target OS: Linux
Domoticz version:
Contact:

Re: Solar Data script : Azimuth, Altitude, Lux

Post by glsf91 »

jmleglise wrote: Thursday 18 February 2021 15:41 Many of you make the mistake. Look at the wiki for the creation of the devices. There is even a screenshot for the devices to be created.

https://www.domoticz.com/wiki/Lua_dzVen ... titude_Lux
But under the picture with the devices is stated: "Sun Radiation : Virtual sensor type "Solar Radiation"."
glsf91
Posts: 58
Joined: Tuesday 14 November 2017 21:56
Target OS: Linux
Domoticz version:
Contact:

Re: Solar Data script : Azimuth, Altitude, Lux

Post by glsf91 »

hestia wrote: Thursday 25 February 2021 20:15 Regarding the source for the cloud cover:
www.ogimet.com give good information but only every hours and with a delay (during the lock-down there were very few update for my site: an airport!)
Other weather services integrated in Domoticz are not so good at the present time as you shown it (openweatherMap)
The 2 services should be compared and perhaps come back to ogimet: the solar script could stay the same and another script could give the cloud cover from ogimet (I don’t know if you gave one – I could give one in this topic if needed).
Can you supply the Ogimet script?
hestia
Posts: 357
Joined: Monday 25 December 2017 23:06
Target OS: Raspberry Pi / ODroid
Domoticz version: 2022.1
Location: Paris
Contact:

Re: Solar Data script : Azimuth, Altitude, Lux

Post by hestia »

the script

Code: Select all

--[[
Virtual Okta sensor (only)
from https://www.domoticz.com/forum/viewtopic.php?f=72&t=19220
from V2.4 - BakSeeDaa - dzVents version of the Solar Data Script

see also
http://www.ogimet.com/display_synops2.php?lang=en&lugar=07149&tipo=ALL&ord=REV&nil=SI&fmt=html&ano=2019&mes=03&day=30&hora=10&anof=2019&mesf=03&dayf=31&horaf=23&send=send
see https://www.ogimet.com/getsynop_help.phtml.en
https://www.ogimet.com/synops.phtml.en (web GUI)

to check for Orly => https://www.infoclimat.fr/observations-meteo/temps-reel/orly-athis-mons/07149.html

--05/04/2020: prerequisite: requires dzVents >= 3.0.17 (domoticz 2020.2 build 12702)

]]--

-- Variables to customize ------------------------------------------------
local WMOID = '07149'  -- Paris-Orly	
--local WMOID = '07156' -- Paris-Montsouris -- (String) Nearest synop station for ogimet
local warnNoCloudDataHours = 12 -- Warn limit (hours) if no cloud cover report has been received.

local idxCloudCover = 1144 -- (Integer) dz Cloud Cover (PERCENTAGE TYPE) sensor device ID
local idxOkta = 1145 -- (Integer) (Custom) sensor device ID

local HTTPCALLBACK = 'getOgimet'..WMOID

return {
	logging = {
		level   =   
                domoticz.LOG_ERROR, --select one to override system log level normal = LOG_ERROR
                --domoticz.LOG_DEBUG,
                --domoticz.LOG_INFO,
                --domoticz.LOG_ERROR,
                --domoticz.LOG_FORCE
                },
	
	on = {
	    --devices = {206},   -- a switch for testing w/o waiting minutes
		timer = {'at *:15', 'at *:25', 'at *:30', 'at *:35', 'at *:55'}, -- to tune with your service location to limit errors, http service not available always
		httpResponses = {
			HTTPCALLBACK,
		},
	},
	data = {
		lastOkta = {initial=0},
		lastOgimetTime = {initial='198001010000'},
		lastOgimetChange = {initial=nill}
	},
	
	execute = function(dz, item)
	    _G.logMarker =  dz.moduleLabel -- set logmarker to scriptname
	    local LOG_LEVEL =   
	                        dz.LOG_INFO
	                        --dz.LOG_DEBUG
	                        --dz.LOG_ERROR
	                        --dz.LOG_FORCE
-- FUNCTIONS ----
-- calculate the delta between UTC and local time
    local function getTimezone()
        local now = os.time()
        return math.floor(os.difftime(now, os.time(os.date("!*t", now))))
    end

-- convert UTC time to local time
    local function UTCtoLocal(p_UTCtime)
        -- Convert the string to an epoch time stamp and add timezone delta in seconds
        local myLocalTimestamp = dz.time.dateToTimestamp(p_UTCtime,'(%d%d%d%d)(%d%d)(%d%d)(%d%d)(%d%d)' ) + getTimezone()

        -- Convert the caclulated timestamp to the required format
        local localTimeDDMM = dz.time.timestampToDate(myLocalTimestamp,'dd/mm')
        local localTimeHHMM = dz.time.timestampToDate(myLocalTimestamp,'hh:MM')
        dz.log('UTC time ' .. p_UTCtime .. ' = local Time ' .. localTimeDDMM .. ' ' .. localTimeHHMM, dz.LOG_DEBUG)
    return localTimeDDMM, localTimeHHMM
	end
		
-- MAIN -----		
		local lastOgimetTime = dz.data.lastOgimetTime
		local lastOgimetChange = dz.data.lastOgimetChange
		
		dz.log("lastOgimetTime: " .. lastOgimetTime, LOG_LEVEL)
		local Time = require('Time')
		local theTime = Time()
			dz.log("theTime: " .. theTime.raw, LOG_LEVEL)
		
		if lastOgimetChange == nill then
		    dz.log('lastOgimetChange NIL!!!', dz.LOG_ERROR)
		    lastOgimetChange = theTime
		    dz.data.lastOgimetChange = lastOgimetChange
		end
		dz.log("lastOgimetChange: " .. lastOgimetChange.raw, LOG_LEVEL)
		
		if item.isTimer or item.isDevice then
			local ogimetDelay = 1140 -- Minimum anticipated Ogimet data lag (19 minutes)
			local qOgimetTime = os.date('!%Y%m%d%H', os.time()- ogimetDelay)..'00' -- in UTC time
			dz.log("qOgimetTime1: " .. qOgimetTime, LOG_LEVEL)
		
		    if qOgimetTime > lastOgimetTime or item.isDevice then -- item.isDevice is to force the call for testing with a dummy switch (could generate http errors if too often)
				-- There might be recent ogimet data to fetch
			    qOgimetTime = os.date('!%Y%m%d%H', os.time()-(12*3600+ogimetDelay))..'00' -- Twelve hours of data
			    dz.log("qOgimetTime2: " .. qOgimetTime, LOG_LEVEL)
				if lastOgimetTime > qOgimetTime then
				    qOgimetTime = lastOgimetTime:sub(1, -2)..'1' -- Add 1 minute to it
					dz.log( "Add 1 minute to qOgimetTime: " .. qOgimetTime, LOG_LEVEL)
				end
				-- Get synopCode (surface synopCodetic observations) message from Ogimet web site
				url ='http://www.ogimet.com/cgi-bin/getsynop?block='..WMOID..'&begin='..qOgimetTime
				dz.log( "url= " .. url, LOG_LEVEL)
				dz.log('Requesting new cloud cover data from Ogimet...', LOG_LEVEL)
				dz.openURL({url = url, method = 'GET', callback = HTTPCALLBACK}).afterSec(5)
			else
			 	dz.log('No need to request new cloud cover data from Ogimet. Using old data with UTC time stamp: '..lastOgimetTime, LOG_LEVEL)
			    return
			end
			
		end

		if not item.isHTTPResponse then
		    dz.log('Response not http' , LOG_LEVEL)
		    return
	    end
		
		local response = item
    	dz.log("response.data", LOG_LEVEL)
  		dz.log('\n' .. response.data, LOG_LEVEL)
		if response == nil then
			dz.log('http response null. Trigger: '..response.trigger, LOG_LEVEL)
			return
		end
		
		if not response.ok then
		    dz.log('http response not ok. Trigger: '..response.trigger, dz.LOG_ERROR)
	    	return
		end
		
		if (response.trigger ~= HTTPCALLBACK) then
			dz.log('http response not right trigger. Trigger: '..response.trigger, dz.LOG_ERROR)
		    return
		end

		dz.log('Ogimet data has been received', LOG_LEVEL)

		local function split(s, delimiter)
			local result = {}
			for match in (s..delimiter):gmatch('(.-)'..delimiter) do
				table.insert(result, match)
			end
			return result
		end

		-- In meteorology, an okta is a unit of measurement used to describe the amount of cloud cover
		-- at any given location such as a weather station. Sky conditions are estimated in terms of how many
		-- eighths of the sky are covered in cloud, ranging from 0 oktas (completely clear sky) through to 8 oktas
		-- (completely overcast). In addition, in the synop code there is an extra cloud cover indicator '9'
		-- indicating that the sky is totally obscured (i.e. hidden from view),
		-- usually due to dense fog or heavy snow.
		-- Total cloud cover in oktas (eighths)
		-- which is actually seen by the observer during the observation
		-- Zero means absolutely clear, while 8 means absolutely cloudy
		-- Code 9 signifies that the sky is obscured by fog, haze, and/or other phenomena
		-- and “/” means that the cloud cover is indiscernable or that the sky observation was not made (but it is a valid answer)

		-- Find the okta value for the last valid line in the response
		-- The response may contain multiple rows and some of them may not be valid.
		local okta, ogimetTime
		local lastOktaOk = false
        local OktaOk = false
		for line in response.data:gmatch("[^\r\n]+") do
			if line == nil then break end
			if (string.find(line,'NIL=') == nil)
			and (string.find(line,'Status: 500') == nil)
			and (string.find(line, WMOID) ~= nil) then
				local s = split(line, ',')
				if s and #s >= 7 then -- the 7th block separated with ,
					local x = string.sub(split(s[7], ' ')[5], 1, 1) -- the 5th block separated with blanks
					-- okta is the 1st char of this block
					-- dz.log('x of line: '.. x, LOG_LEVEL)  
					if x ~= '/' then
						okta = x
						ogimetTime = s[2]..s[3]..s[4]..s[5]..s[6]
						lastOgimetChange=theTime
						OktaOk = true
						dz.log('New okta: '.. okta, LOG_LEVEL)
						--dz.log('=> ogimetTime: '.. ogimetTime, LOG_LEVEL)
					else -- '/' means, the response is correct, but no new value is given, we keep the previous one
					    lastOktaOk = true
					    ogimetTime = s[2]..s[3]..s[4]..s[5]..s[6]
					    dz.log('Last okta', LOG_LEVEL) 
					end
				end
			end
    	end
        
        if 	ogimetTime ~= nil then
            dz.log('ogimetTime: '.. ogimetTime, LOG_LEVEL)
            if lastOgimetTime >= ogimetTime then
                dz.log('New value got is older than the previous', LOG_LEVEL)
                OktaOk = false
            end
        else
            dz.log('ogimetTime: nil', LOG_LEVEL)
            OktaOk = false
        end
    
        local lastOkta = dz.data.lastOkta
        dz.log('lastOkta: '.. lastOkta, LOG_LEVEL)
        dz.log('lastOgimetTime: '.. lastOgimetTime, LOG_LEVEL)
        dz.log('lastOgimetChange: '.. lastOgimetChange.raw, LOG_LEVEL)
        local elapseTimeHours=theTime.compare(lastOgimetChange).hours
        dz.log('elapseTimeHours: ' .. elapseTimeHours, LOG_LEVEL)
        if elapseTimeHours > warnNoCloudDataHours then
            dz.log('! No data from WMOID: ' .. WMOID.. " for more than " .. elapseTimeHours .. " hours", dz.LOG_FORCE)
            --to look for a more reliable weather station to query?
            return    
        end
        dz.data.lastOgimetChange = lastOgimetChange
		if not OktaOk then
			--dz.log('!!!!!!!!!!!! ok !!!!!!! Using the saved Okta value: '..lastOkta..' with UTC timestamp: '..lastOgimetTime, dz.LOG_DEBUG)
			-- okta = lastOkta
			dz.log('No new value', dz.LOG_FORCE)
			if lastOktaOk then -- the previous value is ok
			    okta = lastOkta
			    OktaOk = true
			    dz.log('Previous value to keep', dz.LOG_FORCE)
		    end
        end
    
	    if OktaOk then
	        if okta == '9' then okta = '8' end -- max value for cloud cover
			--okta = okta == '9' and 8 or okta -- I don't understand the previous line !!
			-- We store the last fetched value here to be used as a backup value
			dz.log('Okta value: '..okta..' with UTC '..ogimetTime, LOG_LEVEL)
			dz.data.lastOkta = okta
			dz.data.lastOgimetTime = ogimetTime
	   		local oktaPercent = dz.utils.round(okta*100/8)
		    local OktaTimeDDMM, OktaTimeHHMM = UTCtoLocal(ogimetTime)	
			dz.devices(idxCloudCover).updatePercentage(oktaPercent)
			
			local OktaTimeDDMM, OktaTimeHHMM = UTCtoLocal(ogimetTime)			
			dz.log('OktaTimeDDMM ' .. OktaTimeDDMM, dz.LOG_DEBUG)
        	dz.log('OktaTimeHHMM ' .. OktaTimeHHMM, dz.LOG_DEBUG)
        	
			dz.devices(idxOkta).rename('Okta ' .. OktaTimeHHMM)
			dz.devices(idxOkta).updateCustomSensor(okta)

			dz.log('Okta to: ' .. dz.devices(idxOkta).name .. "= ".. okta .. ' with UTC '.. OktaTimeDDMM .. OktaTimeHHMM, dz.LOG_INFO)
		end
		dz.log('SYNOP Station = ' .. WMOID, LOG_LEVEL)
		
end
}		
I've put some links in the script that give infomation, so I don't give more ;-)
Except if needed....
jeroenkl
Posts: 113
Joined: Sunday 14 July 2013 22:00
Target OS: Raspberry Pi / ODroid
Domoticz version: beta
Location: NL
Contact:

Re: Solar Data script : Azimuth, Altitude, Lux

Post by jeroenkl »

hestia wrote: Wednesday 07 April 2021 21:38 the script

Code: Select all

--[[
Virtual Okta sensor (only)
from https://www.domoticz.com/forum/viewtopic.php?f=72&t=19220
from V2.4 - BakSeeDaa - dzVents version of the Solar Data Script

see also
http://www.ogimet.com/display_synops2.php?lang=en&lugar=07149&tipo=ALL&ord=REV&nil=SI&fmt=html&ano=2019&mes=03&day=30&hora=10&anof=2019&mesf=03&dayf=31&horaf=23&send=send
see https://www.ogimet.com/getsynop_help.phtml.en
https://www.ogimet.com/synops.phtml.en (web GUI)

to check for Orly => https://www.infoclimat.fr/observations-meteo/temps-reel/orly-athis-mons/07149.html

--05/04/2020: prerequisite: requires dzVents >= 3.0.17 (domoticz 2020.2 build 12702)

]]--

-- Variables to customize ------------------------------------------------
local WMOID = '07149'  -- Paris-Orly	
--local WMOID = '07156' -- Paris-Montsouris -- (String) Nearest synop station for ogimet
local warnNoCloudDataHours = 12 -- Warn limit (hours) if no cloud cover report has been received.

local idxCloudCover = 1144 -- (Integer) dz Cloud Cover (PERCENTAGE TYPE) sensor device ID
local idxOkta = 1145 -- (Integer) (Custom) sensor device ID

local HTTPCALLBACK = 'getOgimet'..WMOID

return {
	logging = {
		level   =   
                domoticz.LOG_ERROR, --select one to override system log level normal = LOG_ERROR
                --domoticz.LOG_DEBUG,
                --domoticz.LOG_INFO,
                --domoticz.LOG_ERROR,
                --domoticz.LOG_FORCE
                },
	
	on = {
	    --devices = {206},   -- a switch for testing w/o waiting minutes
		timer = {'at *:15', 'at *:25', 'at *:30', 'at *:35', 'at *:55'}, -- to tune with your service location to limit errors, http service not available always
		httpResponses = {
			HTTPCALLBACK,
		},
	},
	data = {
		lastOkta = {initial=0},
		lastOgimetTime = {initial='198001010000'},
		lastOgimetChange = {initial=nill}
	},
	
	execute = function(dz, item)
	    _G.logMarker =  dz.moduleLabel -- set logmarker to scriptname
	    local LOG_LEVEL =   
	                        dz.LOG_INFO
	                        --dz.LOG_DEBUG
	                        --dz.LOG_ERROR
	                        --dz.LOG_FORCE
-- FUNCTIONS ----
-- calculate the delta between UTC and local time
    local function getTimezone()
        local now = os.time()
        return math.floor(os.difftime(now, os.time(os.date("!*t", now))))
    end

-- convert UTC time to local time
    local function UTCtoLocal(p_UTCtime)
        -- Convert the string to an epoch time stamp and add timezone delta in seconds
        local myLocalTimestamp = dz.time.dateToTimestamp(p_UTCtime,'(%d%d%d%d)(%d%d)(%d%d)(%d%d)(%d%d)' ) + getTimezone()

        -- Convert the caclulated timestamp to the required format
        local localTimeDDMM = dz.time.timestampToDate(myLocalTimestamp,'dd/mm')
        local localTimeHHMM = dz.time.timestampToDate(myLocalTimestamp,'hh:MM')
        dz.log('UTC time ' .. p_UTCtime .. ' = local Time ' .. localTimeDDMM .. ' ' .. localTimeHHMM, dz.LOG_DEBUG)
    return localTimeDDMM, localTimeHHMM
	end
		
-- MAIN -----		
		local lastOgimetTime = dz.data.lastOgimetTime
		local lastOgimetChange = dz.data.lastOgimetChange
		
		dz.log("lastOgimetTime: " .. lastOgimetTime, LOG_LEVEL)
		local Time = require('Time')
		local theTime = Time()
			dz.log("theTime: " .. theTime.raw, LOG_LEVEL)
		
		if lastOgimetChange == nill then
		    dz.log('lastOgimetChange NIL!!!', dz.LOG_ERROR)
		    lastOgimetChange = theTime
		    dz.data.lastOgimetChange = lastOgimetChange
		end
		dz.log("lastOgimetChange: " .. lastOgimetChange.raw, LOG_LEVEL)
		
		if item.isTimer or item.isDevice then
			local ogimetDelay = 1140 -- Minimum anticipated Ogimet data lag (19 minutes)
			local qOgimetTime = os.date('!%Y%m%d%H', os.time()- ogimetDelay)..'00' -- in UTC time
			dz.log("qOgimetTime1: " .. qOgimetTime, LOG_LEVEL)
		
		    if qOgimetTime > lastOgimetTime or item.isDevice then -- item.isDevice is to force the call for testing with a dummy switch (could generate http errors if too often)
				-- There might be recent ogimet data to fetch
			    qOgimetTime = os.date('!%Y%m%d%H', os.time()-(12*3600+ogimetDelay))..'00' -- Twelve hours of data
			    dz.log("qOgimetTime2: " .. qOgimetTime, LOG_LEVEL)
				if lastOgimetTime > qOgimetTime then
				    qOgimetTime = lastOgimetTime:sub(1, -2)..'1' -- Add 1 minute to it
					dz.log( "Add 1 minute to qOgimetTime: " .. qOgimetTime, LOG_LEVEL)
				end
				-- Get synopCode (surface synopCodetic observations) message from Ogimet web site
				url ='http://www.ogimet.com/cgi-bin/getsynop?block='..WMOID..'&begin='..qOgimetTime
				dz.log( "url= " .. url, LOG_LEVEL)
				dz.log('Requesting new cloud cover data from Ogimet...', LOG_LEVEL)
				dz.openURL({url = url, method = 'GET', callback = HTTPCALLBACK}).afterSec(5)
			else
			 	dz.log('No need to request new cloud cover data from Ogimet. Using old data with UTC time stamp: '..lastOgimetTime, LOG_LEVEL)
			    return
			end
			
		end

		if not item.isHTTPResponse then
		    dz.log('Response not http' , LOG_LEVEL)
		    return
	    end
		
		local response = item
    	dz.log("response.data", LOG_LEVEL)
  		dz.log('\n' .. response.data, LOG_LEVEL)
		if response == nil then
			dz.log('http response null. Trigger: '..response.trigger, LOG_LEVEL)
			return
		end
		
		if not response.ok then
		    dz.log('http response not ok. Trigger: '..response.trigger, dz.LOG_ERROR)
	    	return
		end
		
		if (response.trigger ~= HTTPCALLBACK) then
			dz.log('http response not right trigger. Trigger: '..response.trigger, dz.LOG_ERROR)
		    return
		end

		dz.log('Ogimet data has been received', LOG_LEVEL)

		local function split(s, delimiter)
			local result = {}
			for match in (s..delimiter):gmatch('(.-)'..delimiter) do
				table.insert(result, match)
			end
			return result
		end

		-- In meteorology, an okta is a unit of measurement used to describe the amount of cloud cover
		-- at any given location such as a weather station. Sky conditions are estimated in terms of how many
		-- eighths of the sky are covered in cloud, ranging from 0 oktas (completely clear sky) through to 8 oktas
		-- (completely overcast). In addition, in the synop code there is an extra cloud cover indicator '9'
		-- indicating that the sky is totally obscured (i.e. hidden from view),
		-- usually due to dense fog or heavy snow.
		-- Total cloud cover in oktas (eighths)
		-- which is actually seen by the observer during the observation
		-- Zero means absolutely clear, while 8 means absolutely cloudy
		-- Code 9 signifies that the sky is obscured by fog, haze, and/or other phenomena
		-- and “/” means that the cloud cover is indiscernable or that the sky observation was not made (but it is a valid answer)

		-- Find the okta value for the last valid line in the response
		-- The response may contain multiple rows and some of them may not be valid.
		local okta, ogimetTime
		local lastOktaOk = false
        local OktaOk = false
		for line in response.data:gmatch("[^\r\n]+") do
			if line == nil then break end
			if (string.find(line,'NIL=') == nil)
			and (string.find(line,'Status: 500') == nil)
			and (string.find(line, WMOID) ~= nil) then
				local s = split(line, ',')
				if s and #s >= 7 then -- the 7th block separated with ,
					local x = string.sub(split(s[7], ' ')[5], 1, 1) -- the 5th block separated with blanks
					-- okta is the 1st char of this block
					-- dz.log('x of line: '.. x, LOG_LEVEL)  
					if x ~= '/' then
						okta = x
						ogimetTime = s[2]..s[3]..s[4]..s[5]..s[6]
						lastOgimetChange=theTime
						OktaOk = true
						dz.log('New okta: '.. okta, LOG_LEVEL)
						--dz.log('=> ogimetTime: '.. ogimetTime, LOG_LEVEL)
					else -- '/' means, the response is correct, but no new value is given, we keep the previous one
					    lastOktaOk = true
					    ogimetTime = s[2]..s[3]..s[4]..s[5]..s[6]
					    dz.log('Last okta', LOG_LEVEL) 
					end
				end
			end
    	end
        
        if 	ogimetTime ~= nil then
            dz.log('ogimetTime: '.. ogimetTime, LOG_LEVEL)
            if lastOgimetTime >= ogimetTime then
                dz.log('New value got is older than the previous', LOG_LEVEL)
                OktaOk = false
            end
        else
            dz.log('ogimetTime: nil', LOG_LEVEL)
            OktaOk = false
        end
    
        local lastOkta = dz.data.lastOkta
        dz.log('lastOkta: '.. lastOkta, LOG_LEVEL)
        dz.log('lastOgimetTime: '.. lastOgimetTime, LOG_LEVEL)
        dz.log('lastOgimetChange: '.. lastOgimetChange.raw, LOG_LEVEL)
        local elapseTimeHours=theTime.compare(lastOgimetChange).hours
        dz.log('elapseTimeHours: ' .. elapseTimeHours, LOG_LEVEL)
        if elapseTimeHours > warnNoCloudDataHours then
            dz.log('! No data from WMOID: ' .. WMOID.. " for more than " .. elapseTimeHours .. " hours", dz.LOG_FORCE)
            --to look for a more reliable weather station to query?
            return    
        end
        dz.data.lastOgimetChange = lastOgimetChange
		if not OktaOk then
			--dz.log('!!!!!!!!!!!! ok !!!!!!! Using the saved Okta value: '..lastOkta..' with UTC timestamp: '..lastOgimetTime, dz.LOG_DEBUG)
			-- okta = lastOkta
			dz.log('No new value', dz.LOG_FORCE)
			if lastOktaOk then -- the previous value is ok
			    okta = lastOkta
			    OktaOk = true
			    dz.log('Previous value to keep', dz.LOG_FORCE)
		    end
        end
    
	    if OktaOk then
	        if okta == '9' then okta = '8' end -- max value for cloud cover
			--okta = okta == '9' and 8 or okta -- I don't understand the previous line !!
			-- We store the last fetched value here to be used as a backup value
			dz.log('Okta value: '..okta..' with UTC '..ogimetTime, LOG_LEVEL)
			dz.data.lastOkta = okta
			dz.data.lastOgimetTime = ogimetTime
	   		local oktaPercent = dz.utils.round(okta*100/8)
		    local OktaTimeDDMM, OktaTimeHHMM = UTCtoLocal(ogimetTime)	
			dz.devices(idxCloudCover).updatePercentage(oktaPercent)
			
			local OktaTimeDDMM, OktaTimeHHMM = UTCtoLocal(ogimetTime)			
			dz.log('OktaTimeDDMM ' .. OktaTimeDDMM, dz.LOG_DEBUG)
        	dz.log('OktaTimeHHMM ' .. OktaTimeHHMM, dz.LOG_DEBUG)
        	
			dz.devices(idxOkta).rename('Okta ' .. OktaTimeHHMM)
			dz.devices(idxOkta).updateCustomSensor(okta)

			dz.log('Okta to: ' .. dz.devices(idxOkta).name .. "= ".. okta .. ' with UTC '.. OktaTimeDDMM .. OktaTimeHHMM, dz.LOG_INFO)
		end
		dz.log('SYNOP Station = ' .. WMOID, LOG_LEVEL)
		
end
}		
I've put some links in the script that give infomation, so I don't give more ;-)
Except if needed....
thxs! works fine
User avatar
Varazir
Posts: 360
Joined: Friday 20 February 2015 22:23
Target OS: Raspberry Pi / ODroid
Domoticz version: Beta
Contact:

Re: Solar Data script : Azimuth, Altitude, Lux

Post by Varazir »

plugge wrote: Monday 15 February 2021 14:19
EdwinK wrote: Monday 15 February 2021 13:49 Just found this:

Code: Select all

2021-02-15 13:45:00.483 Error: dzVents: Error: (3.1.4) solarData 3.1: Method updateCustomSensor is not available for device "Sun Radiation" (deviceType=General, deviceSubType=Solar Radiation). If you believe this is not correct, please report.
You used Dummy Virtual Sensor "Solar Radiation", but you should use "Custom Sensor". (I made the same error.)
I hade the same error, it's working now, thanks.
Could someone change the wiki ?

So now I need to figurer out the LUX value when I want the blinder to be lowered, korrekt?
I have been using cloud cover and it's way to inaccurate with Darkstat as provider.


Thanks for the script
Raspberry PI 2 with RaZberry Controller 2016 ZWave+ and CC2531(zigbee)
Several IKEA devices/z-wave devices
User avatar
waltervl
Posts: 5148
Joined: Monday 28 January 2019 18:48
Target OS: Linux
Domoticz version: 2024.7
Location: NL
Contact:

Re: Solar Data script : Azimuth, Altitude, Lux

Post by waltervl »

Varazir wrote: Wednesday 28 July 2021 11:22
plugge wrote: Monday 15 February 2021 14:19
EdwinK wrote: Monday 15 February 2021 13:49 Just found this:

Code: Select all

2021-02-15 13:45:00.483 Error: dzVents: Error: (3.1.4) solarData 3.1: Method updateCustomSensor is not available for device "Sun Radiation" (deviceType=General, deviceSubType=Solar Radiation). If you believe this is not correct, please report.
You used Dummy Virtual Sensor "Solar Radiation", but you should use "Custom Sensor". (I made the same error.)
I hade the same error, it's working now, thanks.
Could someone change the wiki ?
I changed the Wiki instructions to create a virtual custom sensor.
Domoticz running on Udoo X86 (on Ubuntu)
Devices/plugins: ZigbeeforDomoticz (with Xiaomi, Ikea, Tuya devices), Nefit Easy, Midea Airco, Omnik Solar, Goodwe Solar
User avatar
Varazir
Posts: 360
Joined: Friday 20 February 2015 22:23
Target OS: Raspberry Pi / ODroid
Domoticz version: Beta
Contact:

Re: Solar Data script : Azimuth, Altitude, Lux

Post by Varazir »

I have this script to close and open the Blinder but it's not working.

Code: Select all

249	2021-08-04 14:40:01	LUX	75415	0/75415

Code: Select all

return {
	on = {
		timer = {
			'between 14:00 and 60 minutes before sunset every hour',
		},
	},
	logging = {
		level = domoticz.LOG_FORCE, -- Select one of LOG_DEBUG, LOG_INFO, LOG_ERROR, LOG_FORCE to override system log level
		marker = "Blinder"
    },
    
	execute = function(dz, device)
	    local function logWrite(str,level)
            dz.log(tostring(str),level or dz.LOG_MODULE_EXEC_INFO)
        end
	    local lux = dz.devices(249)
	    local blinder = dz.devices(83)
	    
            logWrite('10............... LUX ' .. lux )
		
	        if lux <= "40000" then
    		    logWrite('20................It is cloudy outside')
	    	    blinder.open()
    	    else 
    	        logWrite('30................It is sunny outside')
    	        blinder.close()
	    	end 
	end
}
Raspberry PI 2 with RaZberry Controller 2016 ZWave+ and CC2531(zigbee)
Several IKEA devices/z-wave devices
hestia
Posts: 357
Joined: Monday 25 December 2017 23:06
Target OS: Raspberry Pi / ODroid
Domoticz version: 2022.1
Location: Paris
Contact:

Re: Solar Data script : Azimuth, Altitude, Lux

Post by hestia »

Code: Select all

local lux = dz.devices(249)
here lux is a device

Code: Select all

lux <= "40000"
here lux is expected to be a char
Perhaps

Code: Select all

local lux = dz.devices(249).lux
the value of the device in lux (numeric)
and

Code: Select all

lux <= 40000
hestia
Posts: 357
Joined: Monday 25 December 2017 23:06
Target OS: Raspberry Pi / ODroid
Domoticz version: 2022.1
Location: Paris
Contact:

Re: Solar Data script : Azimuth, Altitude, Lux

Post by hestia »

you also have an existing script that do the job
and based on the sun position
blinds move on the sun
:D
Post Reply

Who is online

Users browsing this forum: No registered users and 0 guests