Python buienradar (is it going to rain) script

Python and python framework

Moderator: leecollings

digdug3
Posts: 19
Joined: Monday 03 August 2015 22:35
Target OS: Raspberry Pi / ODroid
Domoticz version: beta
Contact:

Python buienradar (is it going to rain) script

Post by digdug3 »

After first trying the lua (isitgonnarain) for a few months I decided to convert the original perl buienradar example script to python, this means no additional (perl) packages are needed anymore.

Edit 06-01-2018: Updated script for python 3.x, be sure it's installed. Code cleanup and https fixes.
Edit 08-03-2018: Updated buienradar url.

1. Create a dummy percentage device and set thedeviceidx to the idx of this new device
2. Set domoticz server parameters (Server ip + port, http/https), yourlatitude and yourlongitude
3. Put the script in /home/pi/domoticz/scripts/python as buienradar.py
4. Make it executable (optional): chmod +x /home/pi/domoticz/scripts/python/buienradar.py
5. Add a cronjob:

Code: Select all

crontab -e
and add:

Code: Select all

*/5 * * * * sudo python3 /home/pi/domoticz/scripts/python/buienradar.py silent
(This will run the script every 5 minutes and if something changes, the script will report back to domoticz)

As this is my first python script (based on chopper_robs check_device_online.py python script), feel free to optimize it:

Code: Select all

#!/usr/bin/env python3
# -*- coding: utf-8 -*-
#
# Title: buienradar.py
# Date: 25-10-2016/08-03-2018
# Info: Checks buienradar.nl if it is going to rain and reports back to domoticz
# Version : 0.6
# 0.1	Initial release
# 0.2	Added silent mode
# 0.3	Add password
#	Use direct connection to domoticz to get values
# 0.4	Add SSL and fix server(s)
# 0.5	Updated to Python 3
#	Bug fix when current status was 0
# 0.6 Updated buienradar url

import sys
import datetime
import time
import os
import subprocess
import json
import base64
import ssl
import urllib.request

# Settings for the domoticz server
domoticzserver = "https://192.168.1.200:8006"
domoticzusername = "admin"
domoticzpassword = "password"
domoticzpasscode = "passcode"
domoticzcheckssl = False

# If enabled. The script will log to the file _.log
# Logging to file only happens after the check for other instances, before that it only prints to screen.
log_to_file = False

lat = "52.379189"
lon = "4.899431"
BuienradarUrl = "https://gpsgadget.buienradar.nl/data/raintext/?lat=" + lat + "&lon=" + lon
BuienradarCheckSSL = False
Duration = 25
RainExpectedIDX = "411"

# Check silent mode for cron
silentmode = False
if (len(sys.argv)) > 1:
	if (str(sys.argv[1]) == "silent"):
		silentmode = True

# DO NOT CHANGE BEYOND THIS LINE
lastsuccess=datetime.datetime.now()
base64string = base64.encodestring(('%s:%s' % (domoticzusername,domoticzpassword)).encode()).decode().replace('\n', '')

def log(message):
	if silentmode == False:
		print(message)
	if log_to_file == True:
		logfile = open(sys.argv[0] + '.log', "a")
		logfile.write(message + "\n")
		logfile.close()

def domoticzrequest(url):
	request = urllib.request.Request(url)
	request.add_header("Authorization", "Basic %s" % base64string)
	if (domoticzcheckssl == False):
		response = urllib.request.urlopen(request, context=ssl._create_unverified_context())
	else:
		response = urllib.request.urlopen(request)
	return response.read()

def domoticzstatus(getIDX):
	domoticzurl = domoticzserver + '/json.htm?type=devices&rid=' + getIDX
	json_object = json.loads(domoticzrequest(domoticzurl).decode('utf-8'))
	status = ''

	# Search the device in JSON list
	if json_object["status"] == "OK":
		status = json_object["result"][0]["Data"]
	else:
		log (datetime.datetime.now().strftime("%H:%M:%S") + " - Error: Could not find device idx " + getIDX + " in Domoticz response.")

	return status

log (datetime.datetime.now().strftime("%H:%M:%S") + " - Script buienradar.py started.")

# Get current starttime and endtime
now = datetime.datetime.now().strftime("%H:%M")
hour, min = now.split(":")
starttime = (int(hour)*60) + int(min)
endtime = starttime + Duration

total_rain_predictions = 0
total_rain_values = 0

# Get buienradar data
request = urllib.request.Request(BuienradarUrl)
if (BuienradarCheckSSL == False):
	response = urllib.request.urlopen(request, context=ssl._create_unverified_context())
else:
	response = urllib.request.urlopen(request)

html = response.read()	# read the data


# 000|hh:mm		where 000 is mm of rain expected (000 -> no rain, 255 -> heavy rain)
for line in html.splitlines():
	#print("rain: " + data[0] + " time:" + data[1])
	data = line.decode('utf-8').split("|")
	mhour, mmin = data[1].split(":")
	calc_time = (int(mhour)*60) + int(mmin)
	if ((calc_time >= starttime) and (calc_time <= endtime)):
		total_rain_predictions = total_rain_predictions + float(data[0])
		total_rain_values = total_rain_values + 1

response.close()  # close the connection

# Calculate result
result = "0.00"

if (total_rain_values > 0):
	rain_0_100 = (total_rain_predictions/total_rain_values)*0.392156862745098		# convert 000/255 data to procent (100/255)
	result = format(rain_0_100, '.2f')
	currentstatus = format(int(''.join(filter(str.isdigit, domoticzstatus(RainExpectedIDX)))), '.2f')
	if (result != currentstatus):												# Status has changed --> notify domoticz
		log (datetime.datetime.now().strftime("%H:%M:%S") + " - Current status: " + currentstatus + " / result: " + result)
		domoticzrequest(domoticzserver + "/json.htm?type=command&param=udevice&idx=" + RainExpectedIDX + "&nvalue=0&svalue=" + result + "&passcode=" + domoticzpasscode)
	else:
		if (hour == 12 and min < 10):											# Send some data to prevent a red bar --> notify domoticz
			log (datetime.datetime.now().strftime("%H:%M:%S") + " - Current status: " + currentstatus + " / updating")
			domoticzrequest(domoticzserver + "/json.htm?type=command&param=udevice&idx=" + RainExpectedIDX + "&nvalue=0&svalue=0.00&passcode=" + domoticzpasscode)
Last edited by digdug3 on Thursday 08 March 2018 7:49, edited 4 times in total.
D3LTA
Posts: 24
Joined: Monday 15 July 2013 2:37
Target OS: Raspberry Pi / ODroid
Domoticz version: 3.8153
Location: Twente
Contact:

Re: Python buienradar (is it going to rain) script

Post by D3LTA »

thank you very much
User avatar
Egregius
Posts: 2589
Joined: Thursday 09 April 2015 12:19
Target OS: Linux
Domoticz version: v2024.7
Location: Beitem, BE
Contact:

Re: Python buienradar (is it going to rain) script

Post by Egregius »

And this is a php version :)
Results in mm/h according to formula on buienradar.

Code: Select all

$rains=curl('http://gadgets.buienradar.nl/data/raintext/?lat=50.89&lon=3.11');
$rains=str_split($rains,11);
$totalrain=0;
$aantal=0;
foreach($rains as $rain)
{
    $aantal=$aantal+1;
    $totalrain=$totalrain+substr($rain,0,3);
    if($aantal==7)
        break;
}
$newbuienradar=pow(10,((($totalrain/7)-109)/32));
if(isset($newbuienradar))
    $weer['buien']=$newbuienradar;
 
User avatar
JHO01
Posts: 24
Joined: Wednesday 02 November 2016 13:59
Target OS: Raspberry Pi / ODroid
Domoticz version: Stable
Location: Netherlands
Contact:

Re: Python buienradar (is it going to rain) script

Post by JHO01 »

Almost working, but I got this error:

Code: Select all

pi@raspberrypi:~/domoticz/scripts/python$ python gaathetregenen.py
13:53:47- script started.
Traceback (most recent call last):
  File "gaathetregenen.py", line 97, in <module>
    currentstatus = domoticzstatus()
  File "gaathetregenen.py", line 61, in domoticzstatus
    status = format(float(getvalue[:-1]), '.2f')
ValueError: could not convert string to float: Hello Worl
RPI-3, Z-stick Gen5, RFlink, Zigbee2mqtt, 1-wire DS18B20, Fibaro switches, various RF-433 sockets, BMP018, Somfy, Wemos, Tradfri.
User avatar
EdwinK
Posts: 1820
Joined: Sunday 22 January 2017 21:46
Target OS: Raspberry Pi / ODroid
Domoticz version: BETA
Location: Rhoon
Contact:

Re: Python buienradar (is it going to rain) script

Post by EdwinK »

JHO01 wrote: Sunday 26 November 2017 14:13 Almost working, but I got this error:

Code: Select all

ValueError: could not convert string to float: Hello Worl
Looks like you created a text device, instead of a percentage one.
Running latest BETA on a Pi-3 | Toon® Thermostat (rooted) | Hue | Tuya | IKEA tradfri | Dashticz V3 on Lenovo Huawei Tablet | Conbee
User avatar
EdwinK
Posts: 1820
Joined: Sunday 22 January 2017 21:46
Target OS: Raspberry Pi / ODroid
Domoticz version: BETA
Location: Rhoon
Contact:

Re: Python buienradar (is it going to rain) script

Post by EdwinK »

Egregius wrote: Wednesday 07 December 2016 9:10 And this is a php version :)
Results in mm/h according to formula on buienradar.

Code: Select all

$rains=curl('http://gadgets.buienradar.nl/data/raintext/?lat=50.89&lon=3.11');
$rains=str_split($rains,11);
$totalrain=0;
$aantal=0;
foreach($rains as $rain)
{
    $aantal=$aantal+1;
    $totalrain=$totalrain+substr($rain,0,3);
    if($aantal==7)
        break;
}
$newbuienradar=pow(10,((($totalrain/7)-109)/32));
if(isset($newbuienradar))
    $weer['buien']=$newbuienradar;
 
Is this the complete script? Looks so small
Running latest BETA on a Pi-3 | Toon® Thermostat (rooted) | Hue | Tuya | IKEA tradfri | Dashticz V3 on Lenovo Huawei Tablet | Conbee
User avatar
Egregius
Posts: 2589
Joined: Thursday 09 April 2015 12:19
Target OS: Linux
Domoticz version: v2024.7
Location: Beitem, BE
Contact:

Re: Python buienradar (is it going to rain) script

Post by Egregius »

With PHP you need a lot less code ;)
This script is complete to grab the buienradar api and store the prediction in a variable $weer['buien'] wich can later be used to push it to domoticz, or just used in php scripts.
User avatar
JHO01
Posts: 24
Joined: Wednesday 02 November 2016 13:59
Target OS: Raspberry Pi / ODroid
Domoticz version: Stable
Location: Netherlands
Contact:

Re: Python buienradar (is it going to rain) script

Post by JHO01 »

Yep, thnx, solved
RPI-3, Z-stick Gen5, RFlink, Zigbee2mqtt, 1-wire DS18B20, Fibaro switches, various RF-433 sockets, BMP018, Somfy, Wemos, Tradfri.
User avatar
EdwinK
Posts: 1820
Joined: Sunday 22 January 2017 21:46
Target OS: Raspberry Pi / ODroid
Domoticz version: BETA
Location: Rhoon
Contact:

Re: Python buienradar (is it going to rain) script

Post by EdwinK »

Looks like the url has changed. When I manually entered the url used in the script, it changes to: https://br-gpsgadget-new.azurewebsites. ... 6&lon=4.41
Running latest BETA on a Pi-3 | Toon® Thermostat (rooted) | Hue | Tuya | IKEA tradfri | Dashticz V3 on Lenovo Huawei Tablet | Conbee
digdug3
Posts: 19
Joined: Monday 03 August 2015 22:35
Target OS: Raspberry Pi / ODroid
Domoticz version: beta
Contact:

Re: Python buienradar (is it going to rain) script

Post by digdug3 »

Updated the script for the new url, add https etc. See first post.
User avatar
EdwinK
Posts: 1820
Joined: Sunday 22 January 2017 21:46
Target OS: Raspberry Pi / ODroid
Domoticz version: BETA
Location: Rhoon
Contact:

Re: Python buienradar (is it going to rain) script

Post by EdwinK »

Code: Select all

File "buienradar.py", line 23, in <module>
    import ssl
  File "/usr/local/lib/python3.5/ssl.py", line 98, in <module>
    import _ssl             # if we can't import it, let the error propagate
ImportError: No module named '_ssl'
I believe it worked all the time, now it suddenly doesn't anymore :(
Running latest BETA on a Pi-3 | Toon® Thermostat (rooted) | Hue | Tuya | IKEA tradfri | Dashticz V3 on Lenovo Huawei Tablet | Conbee
digdug3
Posts: 19
Joined: Monday 03 August 2015 22:35
Target OS: Raspberry Pi / ODroid
Domoticz version: beta
Contact:

Re: Python buienradar (is it going to rain) script

Post by digdug3 »

Could you do a:

Code: Select all

python3 --version
and a

Code: Select all

python --version
And did you change the cronjob as well?

Code: Select all

*/5 * * * * sudo python3 /home/pi/domoticz/scripts/python/buienradar.py silent
User avatar
EdwinK
Posts: 1820
Joined: Sunday 22 January 2017 21:46
Target OS: Raspberry Pi / ODroid
Domoticz version: BETA
Location: Rhoon
Contact:

Re: Python buienradar (is it going to rain) script

Post by EdwinK »

digdug3 wrote: Monday 22 January 2018 8:31 Could you do a:

Code: Select all

python3 --version
python 3.5.3
and a

Code: Select all

python --version
Python 2.7.13
And did you change the cronjob as well?

Code: Select all

*/5 * * * * sudo python3 /home/pi/domoticz/scripts/python/buienradar.py silent
Yes, although I have all my external scripts in /var/scripts, so the path is a little different:

Code: Select all

*/5 * * * * sudo python3 /var/scripts/buienradar.py silent
Running latest BETA on a Pi-3 | Toon® Thermostat (rooted) | Hue | Tuya | IKEA tradfri | Dashticz V3 on Lenovo Huawei Tablet | Conbee
digdug3
Posts: 19
Joined: Monday 03 August 2015 22:35
Target OS: Raspberry Pi / ODroid
Domoticz version: beta
Contact:

Re: Python buienradar (is it going to rain) script

Post by digdug3 »

"Google" tell me you could try to install this:

Code: Select all

sudo apt-get install libssl-dev openssl
digdug3
Posts: 19
Joined: Monday 03 August 2015 22:35
Target OS: Raspberry Pi / ODroid
Domoticz version: beta
Contact:

Re: Python buienradar (is it going to rain) script

Post by digdug3 »

Do you use jessie or stretch on your pi?
User avatar
EdwinK
Posts: 1820
Joined: Sunday 22 January 2017 21:46
Target OS: Raspberry Pi / ODroid
Domoticz version: BETA
Location: Rhoon
Contact:

Re: Python buienradar (is it going to rain) script

Post by EdwinK »

I'm using Stretch on the Pi.

Code: Select all

Reading package lists... Done
Building dependency tree       
Reading state information... Done
libssl-dev is already the newest version (1.1.0f-3+deb9u1).
openssl is already the newest version (1.1.0f-3+deb9u1).
openssl set to manually installed.
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
It seems it's already installed.
Running latest BETA on a Pi-3 | Toon® Thermostat (rooted) | Hue | Tuya | IKEA tradfri | Dashticz V3 on Lenovo Huawei Tablet | Conbee
digdug3
Posts: 19
Joined: Monday 03 August 2015 22:35
Target OS: Raspberry Pi / ODroid
Domoticz version: beta
Contact:

Re: Python buienradar (is it going to rain) script

Post by digdug3 »

Might be this: http://www.domoticz.com/forum/viewtopic ... 3&start=20
Did you install libssl?
User avatar
EdwinK
Posts: 1820
Joined: Sunday 22 January 2017 21:46
Target OS: Raspberry Pi / ODroid
Domoticz version: BETA
Location: Rhoon
Contact:

Re: Python buienradar (is it going to rain) script

Post by EdwinK »

digdug3 wrote: Monday 22 January 2018 14:56 Might be this: http://www.domoticz.com/forum/viewtopic ... 3&start=20
Did you install libssl?
I thought I did when installing domoticz.
Running latest BETA on a Pi-3 | Toon® Thermostat (rooted) | Hue | Tuya | IKEA tradfri | Dashticz V3 on Lenovo Huawei Tablet | Conbee
digdug3
Posts: 19
Joined: Monday 03 August 2015 22:35
Target OS: Raspberry Pi / ODroid
Domoticz version: beta
Contact:

Re: Python buienradar (is it going to rain) script

Post by digdug3 »

Could you do a

Code: Select all

dpkg-query -l 'libssl*'
and post the results?
I have:

Code: Select all

pi@domoticz:~ $ dpkg-query -l 'libssl*'
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name                                 Version                 Architecture            Description
+++-====================================-=======================-=======================-=============================================================================
ii  libssl-dev:armhf                     1.1.0f-3+deb9u1         armhf                   Secure Sockets Layer toolkit - development files
ii  libssl-doc                           1.1.0f-3+deb9u1         all                     Secure Sockets Layer toolkit - development documentation
un  libssl1.0-dev                        <none>                  <none>                  (no description available)
ii  libssl1.0.0:armhf                    1.0.1t-1+deb8u6         armhf                   Secure Sockets Layer toolkit - shared libraries
ii  libssl1.0.2:armhf                    1.0.2l-2+deb9u2         armhf                   Secure Sockets Layer toolkit - shared libraries
ii  libssl1.1:armhf                      1.1.0f-3+deb9u1         armhf                   Secure Sockets Layer toolkit - shared libraries
User avatar
EdwinK
Posts: 1820
Joined: Sunday 22 January 2017 21:46
Target OS: Raspberry Pi / ODroid
Domoticz version: BETA
Location: Rhoon
Contact:

Re: Python buienradar (is it going to rain) script

Post by EdwinK »

Code: Select all

Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name           Version      Architecture Description
+++-==============-============-============-=================================
ii  libssl-dev:arm 1.1.0f-3+deb armhf        Secure Sockets Layer toolkit - de
ii  libssl-doc     1.1.0f-3+deb all          Secure Sockets Layer toolkit - de
un  libssl1.0-dev  <none>       <none>       (no description available)
ii  libssl1.0.0:ar 1.0.1t-1+deb armhf        Secure Sockets Layer toolkit - sh
ii  libssl1.0.2:ar 1.0.2l-2+deb armhf        Secure Sockets Layer toolkit - sh
ii  libssl1.1:armh 1.1.0f-3+deb armhf        Secure Sockets Layer toolkit - sh
Running latest BETA on a Pi-3 | Toon® Thermostat (rooted) | Hue | Tuya | IKEA tradfri | Dashticz V3 on Lenovo Huawei Tablet | Conbee
Post Reply

Who is online

Users browsing this forum: No registered users and 1 guest