buienradar lua script

[nf999]

based on a lot of ideas from this forum i recreated a lua only (no dzvents no python) buienradar script, to add a weather rain device showing the rain based on buienradar, rain expected in a dummy text sensor and a switch to use in sunscreen scripts.

The script does require curl, but that's all. If on rpi run apt-get install curl if curl is missing.

for latest version of the script click here

Code: Select all

-- BuienRadar Module

-- curl in os required!!
-- create dummy text device from dummy hardware 'Buien Radar'
-- create dummy rain sensor from dummy hardware 'BuienRadarMeter'
-- create dummy switch from dummy hardware for triggering screens etc based on rain
-- add as time based lua script
-- set your longitude & lattitude below!

-- 2017-12-27 working version
-- 2017-12-28 trying to get the rain device working .. missing documentation ..
-- 2017-12-30 isolated the bug, i was overwriting the updatedevice .. should be ok now

commandArray = {}

local myBuienRadarDevice='Buien Radar'
local myBuienRadarMeter='BuienRadarMeter'
local myBuienRadarFlg='flgBuienRadar'

-- longitude lattitude
local lat='your-lattitude-here'
local lon='your-longitude-here'

time = os.date("*t")
if time.min % 5 == 0 then 
    
    print('BuienRadar module')

    function os.capture(cmd, rep)  -- execute command to get site
        -- rep is nr of repeats if result is empty
        r = rep or 1
        local s = ""
        while ( s == "" and r > 0) do
            r = r-1
            local f = assert(io.popen(cmd, 'r'))
            s = assert(f:read('*a'))
            f:close()
        end
        if ( rep - r > 1 ) then
            print("os.capture needed more than 1 call: " .. rep-r)
        end
      return s
    end
 
    --  get data from buienradar
    local commando = "curl --max-time 5 -s 'http://br-gpsgadget-new.azurewebsites.net/data/raintext/?lat=" .. lat .. "&lon=" .. lon .. "'"
    -- print("commando: " .. commando)
    local tmp = os.capture(commando, 3)
 
    -- print('buienRadar data:\n' .. tmp)
    
    if ( tmp == "" ) then
        print("buienRadar: Empty result from curl command")
    else
        -- analyse data 

        -- to mm/h 10^((waarde-109)/32)        
        function tomm(r)
            return 10^((r-109)/32)
        end
        
        -- to string formatted
        function tos(r, c)
            c = c or 1
            return string.format("%." .. c .. "f", r)
        end

        local c=0
        local rainNow=0
        local rainNowAvg = 0
        local rainSoon = 0
        local rainTime = ""
        local rainMax = 0
        for k,v in tmp:gmatch('(.-)|(.-)\r?\n') do
            -- k is rain value, v is time
            kn = tonumber(k)
            if c<=1 then
                if rainNow < kn then
                    rainNow = kn
                end
                if kn > 0 then
                    rainNowAvg = rainNowAvg + tomm(kn)/2
                end
            end
            if c<=3 and rainSoon < kn then 
                rainSoon = kn
            end
            if rainTime == "" and kn > 0 then
                rainTime = v
            end
            if kn > rainMax then
                rainMax = kn
            end
            c = c+1
        end

        
        if rainNow>0 then
            tmp = "now; " .. tos(tomm(rainNow)) .. "mm/h"
            if rainMax > rainNow then
                tmp = tmp .. " upto " .. tos(tomm(rainMax)) .. "mm/h"
            end
        elseif rainSoon>0 then
            tmp = "soon in 15mins; " .. tos(tomm(rainSoon)) .. "mm/h"
            if rainMax > rainSoon then
                tmp = tmp .. " upto " .. tos(tomm(rainMax)) .. "mm/h"
            end
        elseif rainTime ~= "" then
            tmp = "expected @ " .. rainTime .. "; upto " .. tos(tomm(rainMax)) .. "mm/h"
        else tmp = "No rain"
        end

        -- calculate totalrainfall using rainNowAvg as average of 2 next reports
        -- print("buienRadarMeterOld: " .. otherdevices_svalues[myBuienRadarMeter])
        local rainTot = tonumber(otherdevices_svalues[myBuienRadarMeter]:match("[^;]+;([^;]+)")) + rainNowAvg/12 -- /12 to acount for 5min measurements ?? ..
        -- print("buienRadarDebug: rainNow=" .. tos(tomm(rainNow)) .. " rainNowAvg=" .. tos(rainNowAvg) .. " rainSoon=" .. 
        --      tos(tomm(rainSoon)) .. " rainTot=" .. tos(rainTot,2) .. " rainTime=" .. rainTime .. " rainMax=" .. tos(tomm(rainMax)))
        local cmd = otherdevices_idx[myBuienRadarMeter] .. "|0|" .. tos(tomm(rainNow)*100,0) .. ";" .. tos(rainTot,2)
        -- print("buienRadar: " .. cmd)
        table.insert(commandArray, { ['UpdateDevice'] = cmd } ) -- table.insert needed to avoid overwriting with next updatedevice
        
        -- write to text device
        if otherdevices[myBuienRadarDevice] ~= tmp then
            table.insert(commandArray, { ['UpdateDevice'] = otherdevices_idx[myBuienRadarDevice] .. '|0|' .. tmp })
            
            -- trigger based when rainNow or rainSoon
            if ( rainNow>0 or rainSoon>0 ) then
                if otherdevices[myBuienRadarFlg] == "Off" then
                    commandArray[myBuienRadarFlg] = "On"
                end
            elseif otherdevices[myBuienRadarFlg] == "On" then
                commandArray[myBuienRadarFlg] = "Off"
            end
    
        end
    end -- empty result
end

return commandArray

[EdwinK]

Code: Select all

2018-02-10 22:20:02.390 LUA: os.capture needed more than 1 call: 3
2018-02-10 22:20:02.390 LUA: buienRadar: Empty result from curl command

Is what I get.

[nf999]

did it improve? Looks like the curl call times out??? try the curl command from commandline? see what happens?
uncomment the line:

Code: Select all

    -- print("commando: " .. commando)

and run the commando from cli on your pc.

[EdwinK]

did it improve? Looks like the curl call times out??? try the curl command from commandline? see what happens?
uncomment the line:

Code: Select all

    -- print("commando: " .. commando)

and run the commando from cli on your pc.

At the moment I'm looking at something else first. If that doesn't work, I'll check in here again

[EdwinK]

Code: Select all

 curl http://br-gpsgadget-new.azurewebsites.net/data/raintext/?lat=51.86&lon=4.41
[1] 14331

Going to wait a couple of hours to see what happens next , but it looks it works now. At least I see some data in the devices tab

[rickiewickie]

Finally buienradar in Lua ! --> (Buienradar plugin keeps crashing after 4 updates)
I already intended to create my own, but this look great so far!

I will test it this weekend

[nf999]

The url for buienradar has changed .. update your script with:

Code: Select all

local command = "curl --max-time 5 -s 'http://gpsgadget.buienradar.nl/data/raintext?lat=" .. lat .. "&lon=" .. lon .. "'"

I also added detection to at least log when the script can't parse the curl output.

full script now:

Code: Select all

-- BuienRadar Module
--
-- curl in os required!!
-- create dummy text device from dummy hardware 'Buien Radar'
-- create dummy rain sensor from dummy hardware 'BuienRadarMeter'
-- create dummy switch from dummy hardware for triggering screens etc based on rain
-- add as time based lua script
-- set your longitude & lattitude below!

-- 2017-12-27 working version
-- 2017-12-28 trying to get the rain device working .. missing documentation ..
-- 2017-12-30 isolated the bug, i was overwriting the updatedevice .. should be ok now
-- 2018-03-28 updated buienradar url

commandArray = {}

local myBuienRadarDevice='Buien Radar'
local myBuienRadarMeter='BuienRadarMeter'
local myBuienRadarFlg='flgBuienRadar'

-- longitude lattitude
local lat='51.99'
local lon='4.50'

time = os.date("*t")
if time.min % 5 == 0 then 
    
    print('BuienRadar module')

    function os.capture(cmd, rep)   -- execute command to get site
        -- rep is nr of repeats if result is empty
        r = rep or 1
        local s = ""
        while ( s == "" and r > 0) do
            r = r-1
            local f = assert(io.popen(cmd, 'r'))
            s = assert(f:read('*a'))
            f:close()
        end
        if ( rep - r > 1 ) then
            print("os.capture needed more than 1 call: " .. rep-r)
        end
      return s
    end
 
    --  get data from buienradar
    local command = "curl --max-time 5 -s 'http://gpsgadget.buienradar.nl/data/raintext?lat=" .. lat .. "&lon=" .. lon .. "'"
    -- print("command: " .. command)
    local tmp = os.capture(command, 3)
 
    -- print('buienRadar data:\n' .. tmp)
    
    if ( tmp == "" ) then
        print("buienRadar: Empty result from curl command")
    else
        -- analyse data 

        -- to mm/h 10^((waarde-109)/32)        
        function tomm(r)
            return 10^((r-109)/32)
        end
        
        -- to string formatted
        function tos(r, c)
            c = c or 1
            return string.format("%." .. c .. "f", r)
        end

        local c=0
        local rainNow=0
        local rainNowAvg = 0
        local rainSoon = 0
        local rainTime = ""
        local rainMax = 0
        for k,v in tmp:gmatch('(.-)|(.-)\r?\n') do
            -- k is rain value, v is time
            kn = tonumber(k)
            if c<=1 then
                if rainNow < kn then
                    rainNow = kn
                end
                if kn > 0 then
                    rainNowAvg = rainNowAvg + tomm(kn)/2
                end
            end
            if c<=3 and rainSoon < kn then 
                rainSoon = kn
            end
            if rainTime == "" and kn > 0 then
                rainTime = v
            end
            if kn > rainMax then
                rainMax = kn
            end
            c = c+1
        end
        
        -- if c = 0 no data found!
        if ( c == 0 ) then
            print("buienRadar: Unparsable result from curl command")
        else
        
            if rainNow>0 then
                tmp = "now; " .. tos(tomm(rainNow)) .. "mm/h"
                if rainMax > rainNow then
                    tmp = tmp .. " upto " .. tos(tomm(rainMax)) .. "mm/h"
                end
            elseif rainSoon>0 then
                tmp = "soon in 15mins; " .. tos(tomm(rainSoon)) .. "mm/h"
                if rainMax > rainSoon then
                    tmp = tmp .. " upto " .. tos(tomm(rainMax)) .. "mm/h"
                end
            elseif rainTime ~= "" then
                tmp = "expected @ " .. rainTime .. "; upto " .. tos(tomm(rainMax)) .. "mm/h"
            else tmp = "No rain"
            end
    
            -- calculate totalrainfall using rainNowAvg as average of 2 next reports
            -- print("buienRadarMeterOld: " .. otherdevices_svalues[myBuienRadarMeter])
            local rainTot = tonumber(otherdevices_svalues[myBuienRadarMeter]:match("[^;]+;([^;]+)")) + rainNowAvg/12 -- /12 to acount for 5min measurements ?? ..
            -- print("buienRadarDebug: rainNow=" .. tos(tomm(rainNow)) .. " rainNowAvg=" .. tos(rainNowAvg) .. " rainSoon=" .. 
            --      tos(tomm(rainSoon)) .. " rainTot=" .. tos(rainTot,2) .. " rainTime=" .. rainTime .. " rainMax=" .. tos(tomm(rainMax)))
            local cmd = otherdevices_idx[myBuienRadarMeter] .. "|0|" .. tos(tomm(rainNow)*100,0) .. ";" .. tos(rainTot,2)
            -- print("buienRadar: " .. cmd)
            table.insert(commandArray, { ['UpdateDevice'] = cmd } ) -- table.insert needed to avoid overwriting with next updatedevice
            
            -- write to text device
            if otherdevices[myBuienRadarDevice] ~= tmp then
                table.insert(commandArray, { ['UpdateDevice'] = otherdevices_idx[myBuienRadarDevice] .. '|0|' .. tmp })
                
                -- trigger based when rainNow or rainSoon
                if ( rainNow>0 or rainSoon>0 ) then
                    if otherdevices[myBuienRadarFlg] == "Off" then
                        commandArray[myBuienRadarFlg] = "On"
                    end
                elseif otherdevices[myBuienRadarFlg] == "On" then
                    commandArray[myBuienRadarFlg] = "Off"
                end
            end
        end -- unparsable
    end -- empty result
end

return commandArray

[poudenes]

Nice script!!!

See sometimes this error:

Code: Select all

2018-03-28 07:55:10.236 Error: EventSystem: Warning!, lua script Script Buienradar has been running for more than 10 seconds

Does anybody know what to do with it?

[nf999]

is no real problem, although it's blocking the event system for this period I think. The number of repetitions is however. the script uses a curl command with 5 secs timeout, to ensure minimum errors but maximum result ..

Code: Select all

local command = "curl --max-time 5 

Sometimes the call to buienradar seems to return nothing, therefore I added a timeout. However after 2 timeouts the script crosses the 10secs .. Your logline was probably accompanied by:

Code: Select all

LUA: os.capture needed more than 1 call: 3

Still no big issue except that buienrader timed out. It tries x times, in the script here 3 times:

Code: Select all

local tmp = os.capture(command, 3)

If still no result it will log empty result ..

So incidentally this script will go over 10secs. If you don't want this adjust the nr of repetitions to 2, or reduce the timeout to 3 secs in os.capture. This setup was fine for me.

[gizmocuz]

Thank you for this script! Worth to note that the lat/long has to be 2 digits max otherwise no result is returned (redirect)
to follow the redirect as well, you can add -L to the curl argument list

Code: Select all

local command = "curl --max-time 5 -L -s 'http://gpsgadget.buienradar.nl/data/raintext?lat=" .. lat .. "&lon=" .. lon .. "'"

Small spelling mistake, it is latitude

[nf999]

thanx!

[HansieNL]

@nf999: If I try to use an other name for "local myBuienRadarDevice='Buien Radar'" i get an error: Error: EventSystem: in /home/pi/domoticz/scripts/lua/script_time_buienradar.lua: /home/pi/domoticz/scripts/lua/script_time_buienradar.lua:129: attempt to concatenate field '?' (a nil value)
I wanna rename "Buien Radar" to "Regen". Can you tell me if that's possible and how to fix it?

[nf999]

@nf999: If I try to use an other name for "local myBuienRadarDevice='Buien Radar'" i get an error: Error: EventSystem: in /home/pi/domoticz/scripts/lua/script_time_buienradar.lua: /home/pi/domoticz/scripts/lua/script_time_buienradar.lua:129: attempt to concatenate field '?' (a nil value)
I wanna rename "Buien Radar" to "Regen". Can you tell me if that's possible and how to fix it?

My guess is either you have not created the "Regen" device yet or made a typo in the variable name. I tested it and anything concatenated with

Code: Select all

otherdevices_idx[bla]

when "bla" is empty or device behind "bla" is non existent results in this error.

[nf999]

latest version of script:

Code: Select all

-- BuienRadar Module
--
-- curl in os required!!
-- create dummy text device from dummy hardware 'Buien Radar'
-- create dummy rain sensor from dummy hardware 'BuienRadarMeter'
-- create dummy switch from dummy hardware for triggering screens etc based on rain
-- add as time based lua script
-- set your longitude & latitude below!

-- 2017-12-27 working version
-- 2017-12-28 trying to get the rain device working .. missing documentation ..
-- 2017-12-30 isolated the bug, i was overwriting the updatedevice .. should be ok now
-- 2018-03-28 updated buienradar url
-- 2018-04-02 added follow redirect for curl & fixed typo as tipped by gizmocuz 

commandArray = {}

local myBuienRadarDevice='Buien Radar'
local myBuienRadarMeter='BuienRadarMeter'
local myBuienRadarFlg='flgBuienRadar'

-- longitude latitude
local lat='51.99'
local lon='4.50'

time = os.date("*t")
if time.min % 5 == 0 then 
    
    print('BuienRadar module')

    function os.capture(cmd, rep)   -- execute command to get site
        -- rep is nr of repeats if result is empty
        r = rep or 1
        local s = ""
        while ( s == "" and r > 0) do
            r = r-1
            local f = assert(io.popen(cmd, 'r'))
            s = assert(f:read('*a'))
            f:close()
        end
        if ( rep - r > 1 ) then
            print("os.capture needed more than 1 call: " .. rep-r)
        end
      return s
    end
 
    --  get data from buienradar
    local command = "curl --max-time 5 -L -s 'http://gpsgadget.buienradar.nl/data/raintext?lat=" .. lat .. "&lon=" .. lon .. "'"
    -- print("command: " .. command)
    local tmp = os.capture(command, 3)
 
    -- print('buienRadar data:\n' .. tmp)
    
    if ( tmp == "" ) then
        print("buienRadar: Empty result from curl command")
    else
        -- analyse data 

        -- to mm/h 10^((waarde-109)/32)        
        function tomm(r)
            return 10^((r-109)/32)
        end
        
        -- to string formatted
        function tos(r, c)
            c = c or 1
            return string.format("%." .. c .. "f", r)
        end

        local c=0
        local rainNow=0
        local rainNowAvg = 0
        local rainSoon = 0
        local rainTime = ""
        local rainMax = 0
        for k,v in tmp:gmatch('(.-)|(.-)\r?\n') do
            -- k is rain value, v is time
            kn = tonumber(k)
            if c<=1 then
                if rainNow < kn then
                    rainNow = kn
                end
                if kn > 0 then
                    rainNowAvg = rainNowAvg + tomm(kn)/2
                end
            end
            if c<=3 and rainSoon < kn then 
                rainSoon = kn
            end
            if rainTime == "" and kn > 0 then
                rainTime = v
            end
            if kn > rainMax then
                rainMax = kn
            end
            c = c+1
        end
        
        -- if c = 0 no data found!
        if ( c == 0 ) then
            print("buienRadar: Unparsable result from curl command")
        else
        
            if rainNow>0 then
                tmp = "now; " .. tos(tomm(rainNow)) .. "mm/h"
                if rainMax > rainNow then
                    tmp = tmp .. " upto " .. tos(tomm(rainMax)) .. "mm/h"
                end
            elseif rainSoon>0 then
                tmp = "soon in 15mins; " .. tos(tomm(rainSoon)) .. "mm/h"
                if rainMax > rainSoon then
                    tmp = tmp .. " upto " .. tos(tomm(rainMax)) .. "mm/h"
                end
            elseif rainTime ~= "" then
                tmp = "expected @ " .. rainTime .. "; upto " .. tos(tomm(rainMax)) .. "mm/h"
            else tmp = "No rain"
            end
    
            -- calculate totalrainfall using rainNowAvg as average of 2 next reports
            -- print("buienRadarMeterOld: " .. otherdevices_svalues[myBuienRadarMeter])
            local rainTot = tonumber(otherdevices_svalues[myBuienRadarMeter]:match("[^;]+;([^;]+)")) + rainNowAvg/12 -- /12 to acount for 5min measurements ?? ..
            -- print("buienRadarDebug: rainNow=" .. tos(tomm(rainNow)) .. " rainNowAvg=" .. tos(rainNowAvg) .. " rainSoon=" .. 
            --      tos(tomm(rainSoon)) .. " rainTot=" .. tos(rainTot,2) .. " rainTime=" .. rainTime .. " rainMax=" .. tos(tomm(rainMax)))
            local cmd = otherdevices_idx[myBuienRadarMeter] .. "|0|" .. tos(tomm(rainNow)*100,0) .. ";" .. tos(rainTot,2)
            -- print("buienRadar: " .. cmd)
            table.insert(commandArray, { ['UpdateDevice'] = cmd } ) -- table.insert needed to avoid overwriting with next updatedevice
            
            -- write to text device
            if otherdevices[myBuienRadarDevice] ~= tmp then
                table.insert(commandArray, { ['UpdateDevice'] = otherdevices_idx[myBuienRadarDevice] .. '|0|' .. tmp })
                
                -- trigger based when rainNow or rainSoon
                if ( rainNow>0 or rainSoon>0 ) then
                    if otherdevices[myBuienRadarFlg] == "Off" then
                        commandArray[myBuienRadarFlg] = "On"
                    end
                elseif otherdevices[myBuienRadarFlg] == "On" then
                    commandArray[myBuienRadarFlg] = "Off"
                end
            end
        end -- unparsable
    end -- empty result
end

return commandArray

[gizmocuz]

Thanks!

It would also be nice if you could add the following functionality (rain perception) into the script

http://www.domoticz.com/forum/viewtopic ... 8&start=20

For example you specify the window to be 15 minutes, how much change of rain there is there in the next 15 minutes, and put this in a percentage sensor
Now we can close the blinds if that is very high.... this way you have have some more time in advance

[HansieNL]

@nf999: Got it working

[nf999]

It would also be nice if you could add the following functionality (rain perception) into the script

http://www.domoticz.com/forum/viewtopic ... 8&start=20

For example you specify the window to be 15 minutes, how much change of rain there is there in the next 15 minutes, and put this in a percentage sensor
Now we can close the blinds if that is very high.... this way you have have some more time in advance

That's actually what I'm using the myBuienRadarFlg for in other words, it's already in there. The myBuienRadarFlg flips to true based on rainNow and rainSoon. rainSoon is the highest rainfall for the next approx 15mins (highest of first 3 data from buienradar), where rainNow is now (only first row of data from buienradar).

So you can use the myBuienRadarFlg to open all blinds.

Code: Select all

rainDev = "flgBuienRadar"
...
    elseif devicechanged[rainDev] == "On" then
        screens("up")
    end

The only issue I have with my screens script is that weatherunderground uv (used for closing the blinds) is really unreliable, so if you have a tip for getting sunpower in a reliable fashion?? Still planning to build an esp8266 uv sensor but that's a completely different topic :-0

[gizmocuz]

Ahhh perfect... yes, when i now look at the code, i see that you use both rainNow and rainSoon great!

[Driesser]

Hi There,

I Installed the Script and it runs every 5 minutes as intented.

I Do have problem with the flgBuienRadar. As instucted I made a dummy switch with the mane "flgBuienRadar"
But it doesn't switch on or off. I switched it on and see if it would switch off after the module have run.

What do I miss? I would like to use the flgBuienRadar in an blockly.

[HansieNL]

Flg BuienRadar switch will only be switched if there is gonna be rain expected or it starts raining. So you have to wait till it's gonna be rainy again.