Using OpenWeatherMap clouds prediction to control blind and heating

[djmoon]

Hi everyone,

I want to share my recent results about controlling floor heating and blinds with clouds predictions.

History :
I have a recent home (6 years) in Belgium with very good insulation and a perfect south exposition... Heated by floor heating with a Daikin air to water heat pump. This system work fine most of the cold season expect in this case : cold sunny day ! And belgian like cold sunny day ! In this case the heat pump see that external temperature is cold and heat water... but when the sun enter in windows the home can over heat quickkly.

It was the beginning of the project !

Goal :
- Know it's a sunny day
- Know the internal temperature
- controlling heat pump
- Measuring heat pump consumption (optional)

How to do that ?
- Connect to internet meteo service using OpenWeatherMap
- Temperature measurment using DS18B20 > you can find a lot of explaination about that
- Controling heat-pump like a thermostat with relay (GPIO on RPi)
- Measuring heat-pump consumption with pulse counter as I have a energy meter on heat pump electrical circuit

My contribution :
- I will explain soon how to populate virtuals sensors with Python 3 script. The main sensor will be : clouds in 3 hours
- I will explain how to do a average of last 3 hours clouds prediction to deduct solar impact on windows with DZevents

Are somebody interested ? My project works fine now... I have to find time to explain it here...

[djmoon]

First the Python script to get weather forecast. The most important for this project is clouds in 3 hours, We can do a simplified script with only this stuff if you want.

I'm not a professional programmer and script are inspired from other one. If people can find from what I'm interested, I forgot to write source year ago...

The principle is simple :
- interogate OpenWeatherMap serveur with API_ID and City_id (update 05/2018 : Weather underground don't provide free developer key anymore, but OpenWeather map provide free key)
- Receiving JSON text in return
- find into JSON text what we want
- send to domoticz virtual sensor value through url request

Code: Select all

# Script to get forecast from weather underground and put them on virtual sensor of DOMOTICZ

# Work on Python 3.4.2
# May 10, 2018 - Inspired by lot's of script commented by J.Teheux
# Sugest to create CRON to launch this script every 15 minutes (96 calls by day)
# example : python3 /home/pi/cron/weather_forecast.py > /dev/null 2>&1
# use "> /dev/null 2>&1" to avoid fill log file for nothing


import urllib.request, urllib.error, urllib.parse
import json
import datetime
import time

# Put your domoticz local URL
domoticz = "http://192.168.0.250:8080"

#You will need to find your particular city ID number
# search for your city by name on this site:
#http://openweathermap.org/help/city_list.txt
# https://openweathermap.org/appid
city_id = "2784548"

# You will need to create an account in Weather Undergroud (devloper) and put your API_ID here
# https://home.openweathermap.org/users/sign_up Choose Free mode and you can use service for free but limited
# https://openweathermap.org/appid 
API_id = "xxxxxxxxxx" # replace xxx by your API ID


# put id of domoticz virtual sensor previously created

# Percentage
idx_clouds = 60 #Clouds % now
idx_forecast_clouds_3h = 82 # this is the only virtual sensor used by my average script after
idx_forecast_clouds_6h = 83
idx_forecast_clouds_9h = 84

#Text
idx_weather = 85 #weather this day : like clouds, rainy, clear...
idx_forecast_weather_3h = 86
idx_forecast_weather_6h = 87
idx_forecast_weather_9h = 88
idx_wind = 61 # wind speed (have to be improved because it's text and I don't remember the unit used by Weather undergroud

#Temperature
idx_min_temp = 62 # min temperature today
idx_max_temp = 63 # max temperature today
idx_forecast_temp_3h = 57 # temperature in 3 hours
idx_forecast_temp_6h = 58
idx_forecast_temp_9h = 59


if 1==1:
    #########################################
    # CURRENT DATA from Weather underground

    # we call OpenWeatherMap with our city id and API id and we will receive data in JSON format/ You can try to put this URL in a browser and translate reponse with
    # JSON viewer like http://jsonviewer.stack.hu to see what you can use like data from Weather underground 
    def gettemp():
        """ call openweathermap api"""
        response = urllib.request.urlopen('http://api.openweathermap.org/data/2.5/weather?id=' + str(city_id) + '&APPID='+ API_id + '&units=metric')
        mydata = response.read()
        mydatastr = mydata.decode('utf-8')
        return mydatastr
  
    weather = gettemp()

    # print (weather)

    # JSON Loads help to find data inside JSON data
    w= json.loads(weather)

    # data from OpenWeatherMap
    #print (w)

    #Wind speed now
    # find inside JSON DATA in 'wind' 'speed' field 
    wind_speed = str((w['wind']['speed']))
    print(('Wind Speed : ' + wind_speed))

    # put data in domoticz virtual sensor
    response = urllib.request.urlopen(domoticz+'/json.htm?type=command&param=udevice&idx='+str(idx_wind)+'&svalue='+str(wind_speed))
    mydata2 = response.read()
    #print (mydata2)
    
    #CURRENT TEMPERATURE Not used in domoticz at this time but you can do it
    temperature = str(round(float(w['main']['temp'])))
    print(('Current Temp : '+ temperature + '°C'))

    #Current clouds
    clouds = str(w['clouds']['all'])
    response = urllib.request.urlopen(domoticz+'/json.htm?type=command&param=udevice&idx='+str(idx_clouds)+'&svalue='+str(clouds))
    mydata2 = response.read()

    ####################################################################
    #FORECAST of the day
    def getforecast():
        """ call openweathermap api"""
        response = urllib.request.urlopen('http://api.openweathermap.org/data/2.5/forecast/daily?id=' + str(city_id) + '&APPID='+ API_id + '&units=metric') 

        mydata = response.read()
        mydatastr = mydata.decode('utf-8')
        return mydatastr
      
    weather = getforecast()
    w = json.loads(str(weather))
  #  print (w)

    #TODAY'S LOW
    temperature_low = str(round(float(w['list'][0]['temp']['min'])))
    print(('Daily Low : ' + temperature_low + '°C'))   
    response = urllib.request.urlopen(domoticz+'/json.htm?type=command&param=udevice&idx='+str(idx_min_temp)+'&svalue='+str(temperature_low))
    mydata2 = response.read()
    #print (mydata2)


    #TODAY'S HIGH 
    temperature_high = str(round(float(w['list'][0]['temp']['max'])))
    print(('Daily High : ' + temperature_high + '°C'))
    response = urllib.request.urlopen(domoticz+'/json.htm?type=command&param=udevice&idx='+str(idx_max_temp)+'&svalue='+str(temperature_high))
    mydata2 = response.read()
    #print (mydata2)

    #rain or clear today?
    todayforecast = str(w['list'][0]['weather'][0]['main'])
    print(('The weather is : ' + todayforecast))
    response = urllib.request.urlopen(domoticz+'/json.htm?type=command&param=udevice&idx='+str(idx_weather)+'&svalue='+str(todayforecast))
    mydata2 = response.read()


    ##################################################
    #Forecast by 3 hours
    def getforecast():
        """ call openweathermap api"""
        response = urllib.request.urlopen('http://api.openweathermap.org/data/2.5/forecast?id=' + str(city_id) + '&APPID='+ API_id + '&units=metric') 
        mydata = response.read()
        mydatastr = mydata.decode('utf-8')
        return mydatastr
      
    weather = getforecast()
    w = json.loads(str(weather))

    # print (w)

    for a in range(0,3):
        time_date = datetime.datetime.fromtimestamp(w['list'][a]['dt']).strftime('%Y-%m-%d %H:%M:%S')

        # by 3 hours find data in next 3h, 6h, 9h you can do more if you want
        temp_3h = str(w['list'][a]['main']['temp'])
        clouds_3h = str(w['list'][a]['clouds']['all'])
        weather_3h = str(w['list'][a]['weather'][0]['main'])
 
        print('Forecast - ' + str(a) + ' - ' + time_date + ' - Temperature : ' + temp_3h + '°C - Clouds : ' + clouds_3h + ' - Weather : ' + weather_3h)

        # put data in domoticz 
        # 3h forecast
        if a==0 :
            response = urllib.request.urlopen(domoticz+'/json.htm?type=command&param=udevice&idx='+str(idx_forecast_temp_3h)+'&svalue='+str(temp_3h))
            mydata2 = response.read()
            #print (mydata2)
            response = urllib.request.urlopen(domoticz+'/json.htm?type=command&param=udevice&idx='+str(idx_forecast_clouds_3h)+'&svalue='+str(clouds_3h))
            mydata2 = response.read()
            #print (mydata2)
            response = urllib.request.urlopen(domoticz+'/json.htm?type=command&param=udevice&idx='+str(idx_forecast_weather_3h)+'&svalue='+str(weather_3h))
            mydata2 = response.read()
            #print (mydata2)
        # 6h forecast
        if a==1 : 
            response = urllib.request.urlopen(domoticz+'/json.htm?type=command&param=udevice&idx='+str(idx_forecast_temp_6h)+'&svalue='+str(temp_3h))
            mydata2 = response.read()
            #print (mydata2)
            response = urllib.request.urlopen(domoticz+'/json.htm?type=command&param=udevice&idx='+str(idx_forecast_clouds_6h)+'&svalue='+str(clouds_3h))
            mydata2 = response.read()
            #print (mydata2)
            response = urllib.request.urlopen(domoticz+'/json.htm?type=command&param=udevice&idx='+str(idx_forecast_weather_6h)+'&svalue='+str(weather_3h))
            mydata2 = response.read()
            #print (mydata2)

        # 9h forecast
        if a==2 : 
            response = urllib.request.urlopen(domoticz+'/json.htm?type=command&param=udevice&idx='+str(idx_forecast_temp_9h)+'&svalue='+str(temp_3h))
            mydata2 = response.read()
            #print (mydata2)
            response = urllib.request.urlopen(domoticz+'/json.htm?type=command&param=udevice&idx='+str(idx_forecast_clouds_9h)+'&svalue='+str(clouds_3h))
            mydata2 = response.read()
            #print (mydata2)
            response = urllib.request.urlopen(domoticz+'/json.htm?type=command&param=udevice&idx='+str(idx_forecast_weather_9h)+'&svalue='+str(weather_3h))
            mydata2 = response.read()
            #print (mydata2) 

[djmoon]

This is the simplified version with only 3h clouds forecast

Code: Select all

# Script to get forecast from OpenWeatherMap and put them on virtual sensor of DOMOTICZ

# Work on Python 3.4.2
# May 10, 2018 - Inspired by lot's of script commented by J.Teheux
# Sugest to create CRON to launch this script every 15 minutes (96 calls by day)
# example : python3 /home/pi/cron/weather_forecast.py > /dev/null 2>&1
# use "> /dev/null 2>&1" to avoid fill log file for nothing


import urllib.request, urllib.error, urllib.parse
import json
import datetime
import time

# Put your domoticz local URL
domoticz = "http://192.168.0.250:8080"

#You will need to find your particular city ID number
# https://openweathermap.org/appid
city_id = "2784548"

# You will need to create an account in OpenWeatherMap (Free) and put your API_ID here
# https://openweathermap.org/appid
API_id = "xxxxx" # xxxx = your API Key


# put id of domoticz virtual sensor previously created

# Percentage
idx_forecast_clouds_3h = 82 # this is the only virtual sensor used by my average script after


if 1==1:
    ##################################################
    #Forecast by 3 hours
    def getforecast():
        """ call openweathermap api"""
        response = urllib.request.urlopen('http://api.openweathermap.org/data/2.5/forecast?id=' + str(city_id) + '&APPID='+ API_id + '&units=metric') 
        mydata = response.read()
        mydatastr = mydata.decode('utf-8')
        return mydatastr
      
    weather = getforecast()
    w = json.loads(str(weather))

    # Clouds in next 3h
    clouds_3h = str(w['list'][0]['clouds']['all'])
    print('Forecast Clouds 3h : ' + clouds_3h)
    # put data in domoticz 
    # 3h forecast
    response = urllib.request.urlopen(domoticz+'/json.htm?type=command&param=udevice&idx='+str(idx_forecast_clouds_3h)+'&svalue='+str(clouds_3h))
    mydata2 = response.read()
    #print (mydata2)
                         

[djmoon]

Now we have a lot of virtual sensor (at least "clouds 3h" one). But this sensor can change every 15 minutes if it's a day with clouds and winds (often in Belgium !) I we use directly the clouds in 3h data the heater can turn on and off every 15 minutes !

As I use a floor heater with water, the inertia is high (at least 3 hours). I would like to know how many sun I'll have in windows in next 3 hours and turn of the heater before sun entering in windows.

Here we'll create un dzevents script with Domoticz directly to do an average of each data of last 3 hours clouds forecast for next 3 hours !
By example at 8:00 you have a view of the sky from 8:00 to 11:00. If the sky is enough clear, we suppose that we'll have enough sun in windows to heat the home > I can turn off heater. I can use this system to close blinds too.

For this function DZevents seem to be the easiest way to do that.

Principle :
- Read virtual sensor value (here clouds in 3 hours)
- Write in history memory dedicated to this script (thanks to DZevents internal function !)
- Do an average of history of last 3 hours (thanks to DZevents internal function !)
- Write result into a other virtual sensor. As I call it luminance and not clouds, I invert it : 100 - clouds value

Code: Select all

-- Average Luminance --
-- DZEVENT script to do a the average of last 3 hours clouds value (virtual sensor) and put it in a luminance virtual device sensor

local OUTPUT_SENSOR = 103 -- "Luminance" -- id of virtual device to write the averages to

-- Input Sensors
local INPUT_SENSOR = 82 -- "Clouds next 3h" id of virtual device to read the data

local LOGGING = true -- True if you want informations in LOG

return {
        active = true,
    on = {
		-- scrit every 15 minutes because weather refresh is 15 minutes too ! Not necessary to upgrade more
		timer = {
		'every 15 minutes'
	   	},
        },
    data = {
        -- define var usable for next call of this function. keep last 20 items in memory (12 could be enough 4 by hours during 3 hours)
        Clouds_history = { history = true, maxItems = 20},
    },
    logging = {
    level = domoticz.LOG_ERROR,
        -- script name in log
    marker = "Avg_Luminance" 
    },

    execute = function(domoticz)

            -- define output sensor (luminance)
            local output = domoticz.devices(OUTPUT_SENSOR)
            
            -- define input sensor (clouds)
            local sensor = domoticz.devices(INPUT_SENSOR) 
            
            -- check if clouds has been updated last 30 minutes (normally every 15 minutes weather is updated)
            if (sensor.lastUpdate.minutesAgo < 30) then
                -- optional !
                -- if clouds are more than 70% it's like 100% clouds because heat from sun don't arrive in windows
                if sensor.percentage<70 then 
                    -- add current value in history var
                    domoticz.data.Clouds_history.add(sensor.percentage)
                else
                    -- add like 100% clouds 
                    domoticz.data.Clouds_history.add(100)
                end
                
                -- if you don't want to use a "saturation" mode next line is enough
                -- domoticz.data.Clouds_history.add(sensor.percentage)
                
            end
            
            local Clouds_average
            -- do average of last 3h data in history > you have a clouds average for next 3 hours
            -- thanks to magical function of dzevents that do the all the job !
            Clouds_average = domoticz.data.Clouds_history.avgSince('03:00:00')
            
            -- round the value without .
            Clouds_average = domoticz.utils.round(Clouds_average,0)
    
            if LOGGING then domoticz.log("Clouds " .. sensor.percentage .. " - Averages clouds are " .. Clouds_average) end
  
            -- set the value in virtual sensor. do the inverse as you want luminance and not clouds 
            output.updatePercentage(100 - Clouds_average)
  
    end
}

[djmoon]

Oups... I'm just modify the code and explanations because I do a mistake and mix weather underground and open weather map in same code. Now everything use openweather map where free mode is still available (no more free mode in weather underground !)

[phoenixblue]

Hi, is it also possible to to get an forecast for the solar radiation for the next day?

[djmoon]

I don't find such information in json reponse from open weather map... sorry.

[misko903]

Oups... I'm just modify the code and explanations because I do a mistake and mix weather underground and open weather map in same code. Now everything use openweather map where free mode is still available (no more free mode in weather underground !)

@djmoon
looks very useful. It can save some money, this will be very satisfactioning for my wife!
Can you please write in few steps for average basic user what to do?
PLEASE!
Thank you!!!