Domoticz

marmachine wrote:Updated the script
There appears to be a redirect (in the request url) causing the script to fail!

my current (working) version:

Code: Select all

-- =============================================================================================================
-- IsItGonnaRain( int minutesinfuture)
-- returns: int avarage rainfall for the next minutesinfuture
-- =============================================================================================================
-- Function to get rain prediction from Buienradar.nl (dutch)
-- Original script here: https://www.domoticz.com/wiki/Is_it_gonna_rain
--
-- Written in LUA by Hans van der Heijden (h4nsie @ gmail.com)
-- Spring 2015
-- 28-03-2015 v0.3 bug: quotes around url added.
-- 27-03-2015 v0.2 return value is now average for next time
-- 26-03-2015 v0.1 Initial release
-- todo: some error checking on http and file handling (tmp file)
----------------------------------------------------------------------------------------------------------------
-- Marco van Wijngaarden
-- 30-08-2016 v1.0 Adopted and updated the above scripts
-- 30-08-2016 v1.1 Updated the request URL, apparently buienrader is redirecting to different url
-- 30-08-2016 v1.2 Updated CURL to follow redirect and reverted (back to documented) request URL
--
----------------------------------------------------------------------------------------------------------------
-- Buienradar.nl --
---------------------
-- Gratis Weerdata --
--
-- Buienradar stelt gratis weerdata beschikbaar voor particulieren en bedrijven (website/intranet). 
-- Het gebruik van onderstaande weerdata is alleen toegestaan voor niet-commerciële doeleinden. 
-- Het gebruik voor mobiele toepassingen of commerciële doeleinden vereist toestemming van Buienradar, zie ook de Disclaimer.
-- Vraag hier toestemming aan voor het gebruik van de weerdata: http://www.buienradar.nl/overbuienradar/contact 
-- Vragen, suggesties of een leuk idee? Neem contact met ons op en bouw mee aan Buienradar!
-- 
------------------------------------------------------------
-- Buienradar widget - Neerslagdata op basis van coördinaten
------------------------------------------------------------
--
-- Op basis van de door u gewenste coördinaten (latitude en longitude) kunt u de neerslag tot twee uur vooruit ophalen in tekstvorm. 
-- De data wordt iedere 5 minuten geüpdatet. 
-- Op deze pagina kunt u de neerslag in tekst vinden: http://gps.buienradar.nl/getrr.php?lat=51&lon=3
-- De waarde 0 geeft geen neerslag aan (droog), de waarde 255 geeft zware neerslag aan. 
-- Gebruik de volgende formule voor het omrekenen naar de neerslagintensiteit in de eenheid millimeter per uur (mm/u):
-- Neerslagintensiteit = 10^((waarde-109)/32)
-- Ter controle: een waarde van 77 is gelijk aan een neerslagintensiteit van 0,1 mm/u.
--
-- =============================================================================================================
-- config ------------------------------------------------------------------------------------------------------
-- =============================================================================================================
--
-- Set your local position (latitude and longitude)
local latitude                  = uservariables['latitude']         -- your home latitude, set as a user variable
local longitude                 = uservariables['longitude']        -- your home longtitude, set as a user variable
-- timewindow for prediction
local minutesinfuture           = 15                                -- (int) minutes in future to calculate rain
-- threshold for switch (when do we actually call it rain and want to act accordingly)
local switchthreshold           = 70                                -- (int) value to toggle virtual switch
-- Rain switch (ON when rain above threshold, else OFF)
local rainswitchname            = "Regen verwacht"                  -- (str) name for switch to toggle
-- Virtual device (txt) displays textual result
local idxRegenmmUur             = 113                               -- IDX for virtual (text) sensor
-- Show results in log
local debug                     = 1                                 -- 1 = ON or 0 = OFF
-- NMA notifications (only during the day)
local notify                    = false                             -- (bool) set 'false' or 'true'
-- URL for data rewquest
local requesturl                = "http://gps.buienradar.nl/getrr.php" 
-- redirects to "http://gadgets.buienradar.nl/data/raintext"
-- Where do we store the data (tmp) file
local tempfilename              = "/home/pi/domoticz/scripts/tmp/rain.tmp"
-- default to activate script (set 'false' to run once every 5 minutes or 'true' to run every minute)
local active                    = false                             -- (bool) set 'false' or 'true'
--
-- =============================================================================================================
--
-- No edit required below this line
--
-- =============================================================================================================
--
-- execute (activate) every 5 minutes
local time = os.date("*t")
if ((time.min % 5)==0) then
    -- set to active
    active = true
end
--
-- =============================================================================================================
-- Functions ---------------------------------------------------------------------------------------------------
-- =============================================================================================================

-- function to split a string --
function splitString(str, delim, maxNb)
    -- Eliminate bad cases...
    if string.find(str, delim) == nil then
        return { str }
    end

    if maxNb == nil or maxNb < 1 then
        maxNb = 0    -- No limit
    end

    local result = {}
    local pat = "(.-)" .. delim .. "()"
    local nb = 0
    local lastPos
    for part, pos in string.gmatch(str, pat) do
        nb = nb + 1
        result[nb] = part
        lastPos = pos
        if nb == maxNb then break end
    end
    -- Handle the last field
    if nb ~= maxNb then
        result[nb + 1] = string.sub(str, lastPos)
    end

    return result
end

-- function to request rain data 
function IsItGonnaRain(requesturl,latitude,longitude,tempfilename,minutesinfuture,debug)

    local averagerain = 0
    local totalrain = 0
    local rainlines = 0
    
    -- construct the request URL
    local url = requesturl..'?lat='..latitude..'&lon='..longitude
    
    -- print to log
    if (debug == 1) then print("| Request URL .............. : " .. url) end

    -- now get the data
    local trigger_action = 'curl -Lo '..tempfilename..' "'..url..'"' 
    -- CURL options:    -s for Silent (no error messages)
    --                  -o for Write output to file instead of stdout.
    --                  -L Follow redirects if the server reports that the requested page has moved (indicated with a Location: header and a 3XX response code)
    handle = os.execute(trigger_action)

    -- print request result to log
    if (debug == 1) then
        if handle then
            print("| CURL Data request ........ : Success")
        else
            print("! CURL Data request ........ : Failed")
        end
    end

    -- open file and process data
    file = io.open(tempfilename, "r")

        -- count datalines
        local linenbr = 0
        
        -- now analyse the received lines
        while true do
            line = file:read("*line")
            if not line then break end
            
            -- count received lines
            linenbr = linenbr+1
            
            -- line/data format is like 000|15:30 per line, we need to split the string
            raindata    = splitString(line,"|")
            rain        = tonumber(raindata[1])
            linetime    = raindata[2]

            -- Linetime2 holds the full date calculated from the time on each line
            linetime2   = os.time{year=os.date('%Y'), month=os.date('%m'), day=os.date('%d'), hour=string.sub(linetime,1,2), min=string.sub(linetime,4,5), sec=os.date('%S')}
            -- calculate the time difference between data/line and os time
            difference  = os.difftime (linetime2,os.time())

            -- When a line entry has a time in the future AND is within the given range, then add/totalize the rainfall
            if ((difference >= 0) and (difference <= minutesinfuture*60)) then
                
                -- print (relevant) data to log
                if (debug == 1) then print("| Rain data (line) ......... : " .. line) end
                
                -- calculate totals
                totalrain = totalrain+rain
                rainlines = rainlines+1
            end
        end
    -- close the file
    file:close()
    
    -- Returned value is average rain fall for next (int) minutes
    -- 0 is no rain, 255 is very heavy rain
    -- When needed, mm/h is calculated by 10^((value -109)/32) (example: 77 = 0.1 mm/hour)
    averagerain = totalrain/rainlines
    
    -- print summary to log
    if (debug == 1) then 
        print("| Rain data received # lines : " .. linenbr)
        print("| In timerange # lines ..... : " .. rainlines)
        print("| Total rain ............... : " .. totalrain)
        print("| Average rain calculated .. : " .. averagerain)
    end

    return(averagerain)

end

-- Functions to round a number to the given number of decimal places
-- returns result as string (text)
function round(num, idp)
  return string.format("%." .. (idp or 0) .. "f", num)
end
-- returns result as a number
function round2number(num, idp)
  return tonumber(string.format("%." .. (idp or 0) .. "f", num))
end

-- Note: If the number is rounded in order to be printed (as text) use function "round" (removed the tonumber function)
-- Converting to number then back to string would reintroduce rounding errors, so for numbers use "round2number" function

-- =============================================================================================================
-- Process -----------------------------------------------------------------------------------------------------
-- =============================================================================================================

commandArray = {}
-- start numbering at 1
indexArray=1

-- executes when active (every 5 minutes)
if (active) then

    -- when debugging set to 1, writing to log starts here
    if (debug == 1) then
        print("+--------------------------------------------------+")
        print("| =============== IS IT GONNA RAIN =============== |")
        print("+--------------------------------------------------+")
    end

    -- rain variables
    local regen = 0
    local regenmmUur = 0
    local rainstr = ""

    -- execute function (for given future minutes)
    regen = IsItGonnaRain(requesturl,latitude,longitude,tempfilename,minutesinfuture,debug)
    
    -- round result
    if (regen > 0) then
        regen = round2number(regen, 0)
    end

    -- convert rain to mm/uur
    if (regen > 0) then
        regenmmUur = 10^((regen-109)/32)
        regenmmUur = round(regenmmUur, 3)
    end

    -- textual result
    if (regen > 0) then
        rainstr = tostring("Regen verwacht: "..regen.." (".. regenmmUur.." mm/uur) binnen "..minutesinfuture.." minuten")
    else
        rainstr = tostring("Geen regen verwacht binnen komende "..minutesinfuture.." minuten")
    end

    -- write value to virtual (text) sensor
    if ((idxRegenmmUur) and (regenmmUur)) then
        commandArray[indexArray] = {['UpdateDevice'] = tostring(idxRegenmmUur)..'|0|'..rainstr}
        indexArray=indexArray+1
    end

    -- when debugging set to 1, write to log
    if (debug == 1) then 
        print("| Rain text ................ : " .. rainstr)
    end

    -- toggle switch ON
    if regen > switchthreshold and otherdevices[rainswitchname]=='Off' then
        
        -- send notification during the day
        if (timeofday['Daytime']) and (notify) then
            commandArray[indexArray] = {['SendNotification']=rainstr}
            indexArray=indexArray+1
        end
        
        -- change switch to ON
        commandArray[indexArray] = {[rainswitchname]='On'}
        indexArray=indexArray+1
    end

    -- toggle switch OFF
    if regen == 0 and otherdevices[rainswitchname] =='On' then
        
        -- send notification during the day
        if (timeofday['Daytime']) and (notify) then
            commandArray[indexArray] = {['SendNotification']=rainstr}
            indexArray=indexArray+1
        end
        
        -- change switch to OFF
        commandArray[indexArray] = {[rainswitchname]='Off'}
        indexArray=indexArray+1
    end

    -- logging end
    if (debug == 1) then
        print("+--------------------------------------------------+")
        print("| --------------------- END ---------------------- |")
        print("+--------------------------------------------------+")
    end

end

return commandArray

Egregius wrote:Is the folder where the file must come created?

Does the user wich runs domoticz has write rights to it?

What happens if you run the os.execute command in shell?

 curl -s -o /var/tmp/luaBuienradar.tmp "http://gps.buienradar.nl/getrr.php?lat=51.9904464&lon=5.7869151"
sudo curl -s -o /var/tmp/luaBuienradar.tmp "http://gps.buienradar.nl/getrr.php?lat=51.9904464&lon=5.7869151"

TakeAway wrote:

marmachine wrote:
Hi, I tried your script, but get the following error:
2016-08-31 20:50:00.023 LUA: +--------------------------------------------------+
2016-08-31 20:50:00.023 LUA: | =============== IS IT GONNA RAIN =============== |
2016-08-31 20:50:00.023 LUA: +--------------------------------------------------+
2016-08-31 20:50:00.023 LUA: | Request URL .............. : http://gps.buienradar.nl/getrr.php?lat= ... lon=4.3961
2016-08-31 20:50:00.160 LUA: ! CURL Data request ........ : Failed
2016-08-31 20:50:00.160 Error: EventSystem: in /home/pi/domoticz/scripts/lua/script_time_rain.lua: /home/pi/domoticz/scripts/lua/script_time_rain.lua:153: attempt to index global 'file' (a nil value)

What can be the problem here?

-- print request result to log
    if (debug == 1) then
        if handle then
            print("| CURL Data request ........ : Success")
        else
            print("! CURL Data request ........ : Failed")
            -- >>> add a the below line to print your CURL request = the trigger_action variable
            print("! CURL request .............. : "..trigger_action)
        end
    end

renerene wrote:Sorry, it is not a rights issue.
I've tried your startup.sh suggestion. And even went back to my original situation, wtithout tmpfs ramdrive.
Lua cannot create the tmp file; it just is not there.
Did a total re-install on the Pi a few days ago, maybe that has something to do with it. IsItGonnaRain worked before.

this is the faulty line:
read = os.execute('curl -s -o '..tempfilename..' "'..url..'"')

Hmmm, again "os.execute". Maybe domoticz cannot execute? Could be the reason for another issue

marmachine wrote:

TakeAway wrote:

marmachine wrote:
Hi, I tried your script, but get the following error:
2016-08-31 20:50:00.023 LUA: +--------------------------------------------------+
2016-08-31 20:50:00.023 LUA: | =============== IS IT GONNA RAIN =============== |
2016-08-31 20:50:00.023 LUA: +--------------------------------------------------+
2016-08-31 20:50:00.023 LUA: | Request URL .............. : http://gps.buienradar.nl/getrr.php?lat= ... lon=4.3961
2016-08-31 20:50:00.160 LUA: ! CURL Data request ........ : Failed
2016-08-31 20:50:00.160 Error: EventSystem: in /home/pi/domoticz/scripts/lua/script_time_rain.lua: /home/pi/domoticz/scripts/lua/script_time_rain.lua:153: attempt to index global 'file' (a nil value)

What can be the problem here?

The error say's that your CURL request was unsuccessfull.
So lets check your CURL request;

in the script (locate the below lines in the function) you could add a line to print the request in case of error;

Code: Select all

-- print request result to log
    if (debug == 1) then
        if handle then
            print("| CURL Data request ........ : Success")
        else
            print("! CURL Data request ........ : Failed")
            -- >>> add a the below line to print your CURL request = the trigger_action variable
            print("! CURL request .............. : "..trigger_action)
        end
    end

jvdz wrote:What happens when you simple run the LUA script from the command prompt?

I am running a script with the listed command with the scheduler and have no issues, as I prefer not to use the event system to run scripts like this for the simple reason that it could slow down other events.

TakeAway wrote:I think the problem here is that you use the wrong tmp folder.

renerene wrote:

TakeAway wrote:I think the problem here is that you use the wrong tmp folder.

No, don't think so. The script has worked before. The old rain.tmp is still there in /var/tmp, so i'm sure I am looking at the right folder.

renerene wrote:

jvdz wrote:What happens when you simple run the LUA script from the command prompt?

can you tell me more?
Do I need to make a seperate lua text file? Or do the Domoticz events already exist somewhere as seperate files? Which command to use to start the lua file from the command prompt?

renerene wrote: Interesting!
do you use crontab for that?

*/5 * * * * sudo lua /home/pi/domoticz/scripts/buienradar_rainprediction.lua >> /var/tmp/BRP.log

marmachine wrote:ahh okay, that looks familiar... your CURL looks pretty good actually
To see what you are receiving, you could copy and past the url (http://gps.buienradar.nl/getrr.php?lat= ... lon=4.3961) into your browser
This is actually what is being writen to the tempfile

The Error: line 154: attempt to index global 'file' (a nil value) is what's bothering you i think
I had the same issue, this could be a file permission issue or the folder does not exist or you don't have permission on the folder

Do you have PUTTY and are you familiar with that to access your Pi?
Can you navigate to the folder: cd /home/pi/domoticz/scripts
Check if you see "tmp" folder here: ls
Navigate into the tmp folder: cd /home/pi/domoticz/scripts/tmp
Check if you see "rain.tmp" file here: ls

Now try to set rights to this file: chmod 777 rain.tmp

To see the content of the file, you can also open it with your text editor (nano); nano rain.tmp
If you're good, you will see the same text as you saw when you opened the url in your browser

renerene wrote:TakeAway, i'm experiencing the same problem '34: attempt to index global 'file' (a nil value)'

1) the rain.tmp is not created.
2) directory for rain.tmp existst and rights are set:

Code: Select all

sudo chmod 666 /var/tmp

3) url output in browser is correct, http://gps.buienradar.nl/getrr.php?lat= ... =5.8769151
4) result of command without -s option: curl -o /var/tmp/luaBuienradar.tmp "http://gps.buienradar.nl/getrr.php?lat= ... =5.8769151"
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0

marmachine wrote:I think that the problem is that you are NOT using the CURL from my script