Python buienradar (is it going to rain) script

[digdug3]

Hmm, thats the same and this

Code: Select all

dpkg-query -l 'libpython*'

[EdwinK]

Hmm, thats the same and this

Code: Select all

dpkg-query -l 'libpython*'

Code: Select all

domoticz@domo-server:~ $ dpkg-query -l 'libpython*'
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name            Version      Architecture Description
+++-===============-============-============-====================================
ii  libpython-all-d 2.7.13-2     armhf        package depending on all supported P
ii  libpython-dev:a 2.7.13-2     armhf        header files and a static library fo
ii  libpython-stdli 2.7.13-2     armhf        interactive high-level object-orient
ii  libpython2.7:ar 2.7.13-2+deb armhf        Shared Python runtime library (versi
ii  libpython2.7-de 2.7.13-2+deb armhf        Header files and a static library fo
ii  libpython2.7-mi 2.7.13-2+deb armhf        Minimal subset of the Python languag
ii  libpython2.7-st 2.7.13-2+deb armhf        Interactive high-level object-orient
ii  libpython3-dev: 3.5.3-1      armhf        header files and a static library fo
ii  libpython3-stdl 3.5.3-1      armhf        interactive high-level object-orient
ii  libpython3.4:ar 3.4.4-3      armhf        Shared Python runtime library (versi
ii  libpython3.4-mi 3.4.4-3      armhf        Minimal subset of the Python languag
ii  libpython3.4-st 3.4.4-3      armhf        Interactive high-level object-orient
ii  libpython3.5:ar 3.5.3-1      armhf        Shared Python runtime library (versi
ii  libpython3.5-de 3.5.3-1      armhf        Header files and a static library fo
ii  libpython3.5-mi 3.5.3-1      armhf        Minimal subset of the Python languag
ii  libpython3.5-st 3.5.3-1      armhf        Interactive high-level object-orient
domoticz@domo-server:~ $ 

[digdug3]

Lot more than I have, but at least the same...
What about the module itself? In my stretch installation they are in

Code: Select all

ls -l /usr/lib/python*/ssl.py

[SystemLord]

Thanks for the script, working perfect!

[EdwinK]

Lot more than I have, but at least the same...
What about the module itself? In my stretch installation they are in

Code: Select all

ls -l /usr/lib/python*/ssl.py

Forgot about this post, but this is what I have

Code: Select all

ls -l /usr/lib/python*/ssl.py
-rw-r--r-- 1 root root 37805 Nov 24 18:33 /usr/lib/python2.7/ssl.py
-rw-r--r-- 1 root root 34392 Apr 24  2016 /usr/lib/python3.4/ssl.py
-rw-r--r-- 1 root root 42352 Jan 19  2017 /usr/lib/python3.5/ssl.py

Going to try it again.

[digdug3]

And what do you have in /usr/local/lib/python3.5/

[EdwinK]

Code: Select all

 cd /usr/local/lib/python3.5
domoticz@domo-server:/usr/local/lib/python3.5 $ ls
LICENSE.txt          base64.py        curses              gzip.py         mimetypes.py     pty.py           sre_compile.py    trace.py
__future__.py        bdb.py           datetime.py         hashlib.py      modulefinder.py  py_compile.py    sre_constants.py  traceback.py
__phello__.foo.py    binhex.py        dbm                 heapq.py        multiprocessing  pyclbr.py        sre_parse.py      tracemalloc.py
__pycache__          bisect.py        decimal.py          hmac.py         netrc.py         pydoc.py         ssl.py            tty.py
_bootlocale.py       bz2.py           difflib.py          html            nntplib.py       pydoc_data       stat.py           turtle.py
_collections_abc.py  cProfile.py      dis.py              http            ntpath.py        queue.py         statistics.py     turtledemo
_compat_pickle.py    calendar.py      dist-packages       idlelib         nturl2path.py    quopri.py        string.py         types.py
_compression.py      cgi.py           distutils           imaplib.py      numbers.py       random.py        stringprep.py     typing.py
_dummy_thread.py     cgitb.py         doctest.py          imghdr.py       opcode.py        re.py            struct.py         unittest
_markupbase.py       chunk.py         dummy_threading.py  imp.py          operator.py      reprlib.py       subprocess.py     urllib
_osx_support.py      cmd.py           email               importlib       optparse.py      rlcompleter.py   sunau.py          uu.py
_pydecimal.py        code.py          encodings           inspect.py      os.py            runpy.py         symbol.py         uuid.py
_pyio.py             codecs.py        ensurepip           io.py           pathlib.py       sched.py         symtable.py       venv
_sitebuiltins.py     codeop.py        enum.py             ipaddress.py    pdb.py           selectors.py     sysconfig.py      warnings.py
_strptime.py         collections      filecmp.py          json            pickle.py        shelve.py        tabnanny.py       wave.py
_sysconfigdata.py    colorsys.py      fileinput.py        keyword.py      pickletools.py   shlex.py         tarfile.py        weakref.py
_threading_local.py  compileall.py    fnmatch.py          lib-dynload     pipes.py         shutil.py        telnetlib.py      webbrowser.py
_weakrefset.py       concurrent       formatter.py        lib2to3         pkgutil.py       signal.py        tempfile.py       wsgiref
abc.py               config-3.5m      fractions.py        linecache.py    plat-linux       site-packages    test              xdrlib.py
aifc.py              configparser.py  ftplib.py           locale.py       platform.py      site.py          textwrap.py       xml
antigravity.py       contextlib.py    functools.py        logging         plistlib.py      smtpd.py         this.py           xmlrpc
argparse.py          copy.py          genericpath.py      lzma.py         poplib.py        smtplib.py       threading.py      zipapp.py
ast.py               copyreg.py       getopt.py           macpath.py      posixpath.py     sndhdr.py        timeit.py         zipfile.py
asynchat.py          crypt.py         getpass.py          macurl2path.py  pprint.py        socket.py        tkinter
asyncio              csv.py           gettext.py          mailbox.py      profile.py       socketserver.py  token.py
asyncore.py          ctypes           glob.py             mailcap.py      pstats.py        sqlite3          tokenize.py

[digdug3]

That one is empty here. So I think Python is using the wrong library.

[digdug3]

Code: Select all

python3 -c "import sys; print('\n'.join(sys.path))"

Should display the paths Python3 is using:

Code: Select all

/usr/lib/python35.zip
/usr/lib/python3.5
/usr/lib/python3.5/plat-arm-linux-gnueabihf
/usr/lib/python3.5/lib-dynload
/usr/local/lib/python3.5/dist-packages
/usr/lib/python3/dist-packages

[EdwinK]

Code: Select all

 python3 -c "import sys; print('\n'.join(sys.path))"

/usr/local/lib/python35.zip
/usr/local/lib/python3.5
/usr/local/lib/python3.5/plat-arm-linux-gnueabihf
/usr/local/lib/python3.5/lib-dynload
/usr/local/lib/python3.5/site-packages

Looks I'm missing /dist-packages

[digdug3]

dist-packages is "nothing", somehow your python is looking at /usr/local/lib/ and mine at /usr/lib/
Could you "fix" the following lines in the code:

Code: Select all

import sys
import datetime
import time
import os
...

to

Code: Select all

import sys
sys.path.append('/usr/lib/python3.5')
import datetime
import time
import os
...

Edit: fixed

[EdwinK]

Got a bit further, still an error

Traceback (most recent call last):
File "buienradar.py", line 16, in <module>
sys.path.append('/usr/lib/python3.5')
NameError: name 'sys' is not defined

[digdug3]

My fault, should be after import sys, like this:

Code: Select all

import sys
sys.path.append('/usr/lib/python3.5')
import datetime
import time
import os
...

[EdwinK]

Thanks. Going to try this again

And then... back to the old error again...

Code: Select all

Traceback (most recent call last):
  File "buienradar.py", line 25, in <module>
    import ssl
  File "/usr/local/lib/python3.5/ssl.py", line 98, in <module>
    import _ssl             # if we can't import it, let the error propagate
ImportError: No module named '_ssl'

I give up for now.

[digdug3]

I understand. It has something to do with your python using the /usr/local/lib path instead of the /usr/lib.
As for google, you are not the only one having this problem...

If I find a new solution, I'll post it here.

[EdwinK]

For now I use an older Lua script, that I believe is working, but i hope that one day I can get your version to work.

[digdug3]

Just updated the buienradar url.

[jjnj]

When using the script i get the following error:

Code: Select all

pi@raspberrypi:~/domoticz/scripts $ ./buienradar.py
/usr/bin/env: ‘python3\r’: No such file or directory

The script itself:

Code: Select all

#!/usr/bin/env python3
# -*- coding: utf-8 -*-
#
# Title: buienradar.py
# Date: 25-10-2016/08-03-2018
# Info: Checks buienradar.nl if it is going to rain and reports back to domoticz
# Version : 0.6
# 0.1	Initial release
# 0.2	Added silent mode
# 0.3	Add password
#	Use direct connection to domoticz to get values
# 0.4	Add SSL and fix server(s)
# 0.5	Updated to Python 3
#	Bug fix when current status was 0
# 0.6 Updated buienradar url

import sys
import datetime
import time
import os
import subprocess
import json
import base64
import ssl
import urllib.request

# Settings for the domoticz server
domoticzserver = "https://127.0.0.1:8080"
domoticzusername = "admin"
domoticzpassword = "password"
domoticzpasscode = "passcode"
domoticzcheckssl = False

# If enabled. The script will log to the file _.log
# Logging to file only happens after the check for other instances, before that it only prints to screen.
log_to_file = False

lat = "xx.xx"
lon = "x.xx"
BuienradarUrl = "https://gpsgadget.buienradar.nl/data/raintext/?lat=" + lat + "&lon=" + lon
BuienradarCheckSSL = False
Duration = 25
RainExpectedIDX = "513"

# Check silent mode for cron
silentmode = False
if (len(sys.argv)) > 1:
	if (str(sys.argv[1]) == "silent"):
		silentmode = True

# DO NOT CHANGE BEYOND THIS LINE
lastsuccess=datetime.datetime.now()
base64string = base64.encodestring(('%s:%s' % (domoticzusername,domoticzpassword)).encode()).decode().replace('\n', '')

def log(message):
	if silentmode == False:
		print(message)
	if log_to_file == True:
		logfile = open(sys.argv[0] + '.log', "a")
		logfile.write(message + "\n")
		logfile.close()

def domoticzrequest(url):
	request = urllib.request.Request(url)
	request.add_header("Authorization", "Basic %s" % base64string)
	if (domoticzcheckssl == False):
		response = urllib.request.urlopen(request, context=ssl._create_unverified_context())
	else:
		response = urllib.request.urlopen(request)
	return response.read()

def domoticzstatus(getIDX):
	domoticzurl = domoticzserver + '/json.htm?type=devices&rid=' + getIDX
	json_object = json.loads(domoticzrequest(domoticzurl).decode('utf-8'))
	status = ''

	# Search the device in JSON list
	if json_object["status"] == "OK":
		status = json_object["result"][0]["Data"]
	else:
		log (datetime.datetime.now().strftime("%H:%M:%S") + " - Error: Could not find device idx " + getIDX + " in Domoticz response.")

	return status

log (datetime.datetime.now().strftime("%H:%M:%S") + " - Script buienradar.py started.")

# Get current starttime and endtime
now = datetime.datetime.now().strftime("%H:%M")
hour, min = now.split(":")
starttime = (int(hour)*60) + int(min)
endtime = starttime + Duration

total_rain_predictions = 0
total_rain_values = 0

# Get buienradar data
request = urllib.request.Request(BuienradarUrl)
if (BuienradarCheckSSL == False):
	response = urllib.request.urlopen(request, context=ssl._create_unverified_context())
else:
	response = urllib.request.urlopen(request)

html = response.read()	# read the data


# 000|hh:mm		where 000 is mm of rain expected (000 -> no rain, 255 -> heavy rain)
for line in html.splitlines():
	#print("rain: " + data[0] + " time:" + data[1])
	data = line.decode('utf-8').split("|")
	mhour, mmin = data[1].split(":")
	calc_time = (int(mhour)*60) + int(mmin)
	if ((calc_time >= starttime) and (calc_time <= endtime)):
		total_rain_predictions = total_rain_predictions + float(data[0])
		total_rain_values = total_rain_values + 1

response.close()  # close the connection

# Calculate result
result = "0.00"

if (total_rain_values > 0):
	rain_0_100 = (total_rain_predictions/total_rain_values)*0.392156862745098		# convert 000/255 data to procent (100/255)
	result = format(rain_0_100, '.2f')
	currentstatus = format(int(''.join(filter(str.isdigit, domoticzstatus(RainExpectedIDX)))), '.2f')
	if (result != currentstatus):												# Status has changed --> notify domoticz
		log (datetime.datetime.now().strftime("%H:%M:%S") + " - Current status: " + currentstatus + " / result: " + result)
		domoticzrequest(domoticzserver + "/json.htm?type=command&param=udevice&idx=" + RainExpectedIDX + "&nvalue=0&svalue=" + result + "&passcode=" + domoticzpasscode)
	else:
		if (hour == 12 and min < 10):											# Send some data to prevent a red bar --> notify domoticz
			log (datetime.datetime.now().strftime("%H:%M:%S") + " - Current status: " + currentstatus + " / updating")
			domoticzrequest(domoticzserver + "/json.htm?type=command&param=udevice&idx=" + RainExpectedIDX + "&nvalue=0&svalue=0.00&passcode=" + domoticzpasscode)

I also tried it with adding sys.path.append('/usr/lib/python3.5')

[digdug3]

How did you upload the script?