Import RIVM sensor data

Python and python framework

Moderator: leecollings

User avatar
phoenixblue
Posts: 76
Joined: Friday 25 November 2016 11:20
Target OS: Windows
Domoticz version:
Contact:

Import RIVM sensor data

Post by phoenixblue »

Hi,

Small background info, because I life nearby chemical industy where it's most of the time safe ;) but for that one moment there is an problem I like to shutdown my mechanical ventilation at home.
So I have found that based on the website https://www.luchtmeetnet.nl/ the info from the RIVM is avalible to use (http://www.lml.rivm.nl/tabel/)
(sorry only usefull for the Netherlands)
From that part is possible to import the data into Domoticz, but therefore you must manual insert total 7 virtual text sensors with the next labels:
PM10 = Fijnstof (particulates)
O3 = Ozon (ozone)
NO = Stikstofmonoxide (nitric oxide)
CO = Koolstofmonoxide (carbon monoxide)
SO2 = Zwaveldioxide (sulfur dioxide)
NH3 = Ammoniak (Ammonia) -> (R717 for the Refrigeration engineers :) )
NO2 = Stikstofdioxide (nitrogen dioxide)

then inside the script:
1) set the idx codes at the right place.
2) set the used AirQuality location from the website nearby your location into the script (sorry it's in the bottem of the script)

Code: Select all

#!/usr/bin/env python
# -*- coding: utf-8 -*-

import xml.etree.ElementTree as ET
import requests
from requests.exceptions import ConnectionError

domoticz_devicesIdx = {"PM10": 1,
                       "O3": 2,
                       "NO": 3,
                       "CO": 4,
                       "SO2": 5,
                       "NH3": 6,
                       "NO2": 7
                       }

domoticz_server = "http://localhost"
domoticz_port = 8080


class AirQuality:

    def __init__(self, location):
        self.link = "http://www.lml.rivm.nl/xml/actueel.xml"
        self.location = location
        timedata = "http://www.lml.rivm.nl/xml/actueel-update.xml"
        z = requests.get(timedata)
        f = ET.fromstring(z.content)
        data = f.getchildren()[0].getchildren()[0].getchildren()
        year = data[0].text
        month = data[1].text
        day = data[2].text
        hour = data[3].text
        self.fetch()

        print "Fetching data from: %s/%s/%s, hour: %s" % (day, month, year, hour)

    def push(self, device, value):
        try:
            requests.get(
                "%s:%d/json.htm?type=command&param=udevice&idx=%d&nvalue=0&svalue=%s" %
                (domoticz_server, domoticz_port, device, value))
        except ConnectionError:
            print "I wasn't able to contact the domoticz server, is it up?"

    def fetch(self):
        # fetch the data and push it
        z = requests.get(self.link)
        elements = ET.fromstring(z.content).getchildren()[0].getchildren()
        components = {}
        for element in elements:
            data = element.getchildren()
            station = data[2].text
            component = data[3].text
            unit = data[4].text
            value = data[5].text
            if not (station.lower() == self.location.lower()):
                continue

            components[component] = '%s%s' % (value, unit)

        for component in domoticz_devicesIdx.keys():
            device = domoticz_devicesIdx[component]
            try:
                value = components[component]
            except KeyError:
                value = "n/a"

            self.push(device, value)


AirQuality("NL01489")

When you run the script it wil import al avalible data, when an sensor station has for an specific gas no sensor there will be send an n/a value.
At this moment there are an few point that must be changed in the future:
- text sensor to airquality sensor with the correct units.
- find out the save thresholds of the sensors and send it maybe to an alert level sensor.
assenzuid
Posts: 135
Joined: Friday 13 November 2015 9:11
Target OS: Raspberry Pi / ODroid
Domoticz version: Beta
Location: The Netherlands, Emmen Area
Contact:

Re: Import RIVM sensor data

Post by assenzuid »

I get below error

Code: Select all

Traceback (most recent call last):
  File "airquality.py", line 71, in <module>
    AirQuality("NL01489")
  File "airquality.py", line 34, in __init__
    self.fetch()
  File "airquality.py", line 69, in fetch
    self.push(device, value)
  File "airquality.py", line 42, in push
    (domoticz_server, domoticz_port, device, value))
  File "/usr/lib/python2.7/dist-packages/requests/api.py", line 60, in get
    return request('get', url, **kwargs)
  File "/usr/lib/python2.7/dist-packages/requests/api.py", line 49, in request
    return session.request(method=method, url=url, **kwargs)
  File "/usr/lib/python2.7/dist-packages/requests/sessions.py", line 457, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/lib/python2.7/dist-packages/requests/sessions.py", line 563, in send
    adapter = self.get_adapter(url=request.url)
  File "/usr/lib/python2.7/dist-packages/requests/sessions.py", line 642, in get_adapter
    raise InvalidSchema("No connection adapters were found for '%s'" % url)
requests.exceptions.InvalidSchema: No connection adapters were found for '192.168.1.xxx:8081/json.htm?type=command&param=udevice&i  dx=309&nvalue=0&svalue=n/a'
User avatar
EdwinK
Posts: 1820
Joined: Sunday 22 January 2017 21:46
Target OS: Raspberry Pi / ODroid
Domoticz version: BETA
Location: Rhoon
Contact:

Re: Import RIVM sensor data

Post by EdwinK »

Thanks. Just something I was looking for, living very close to one of the chemical parts of the country (Botlek).
Running latest BETA on a Pi-3 | Toon® Thermostat (rooted) | Hue | Tuya | IKEA tradfri | Dashticz V3 on Lenovo Huawei Tablet | Conbee
User avatar
phoenixblue
Posts: 76
Joined: Friday 25 November 2016 11:20
Target OS: Windows
Domoticz version:
Contact:

Re: Import RIVM sensor data

Post by phoenixblue »

EdwinK wrote:Thanks. Just something I was looking for, living very close to one of the chemical parts of the country (Botlek).
I think you can test it at this moment ;-)

Grote brand bij Esso in de Botlek

http://www.ad.nl/rotterdam/grote-brand- ... ~a00c1915/
User avatar
EdwinK
Posts: 1820
Joined: Sunday 22 January 2017 21:46
Target OS: Raspberry Pi / ODroid
Domoticz version: BETA
Location: Rhoon
Contact:

Re: Import RIVM sensor data

Post by EdwinK »

Yes. Works :)
Running latest BETA on a Pi-3 | Toon® Thermostat (rooted) | Hue | Tuya | IKEA tradfri | Dashticz V3 on Lenovo Huawei Tablet | Conbee
User avatar
rbisschops
Posts: 63
Joined: Monday 12 May 2014 11:20
Target OS: Raspberry Pi / ODroid
Domoticz version: 2.2563
Location: Schiedam, Netherlands
Contact:

Re: Import RIVM sensor data

Post by rbisschops »

Did you see this note:
De website www.lml.rivm.nl zal langzaam maar zeker zijn inhoud verliezen.
Recentelijk zijn de kaarten met een geinterpoleerde weergave van de meetdata al verwijderd.
Deze zijn namelijk ook te vinden op de gezamenlijke website.

Apparently the data will move to the other site.

Ralph
RPi 3B (master): RFXtrx433, Harmony Hub, Z-Wave devices, CoCo units, Hue, Xiaomi Aquara
RPi 2B (slave): P1 connection, MySensors with custom built watermeter sensors on Aquadis+
RPi’s with: Mosquitto MQTT, Home Bridge, Home Assistant (for evaluation only)
Derik
Posts: 1601
Joined: Friday 18 October 2013 23:33
Target OS: Raspberry Pi / ODroid
Domoticz version: BETA
Location: Arnhem/Nijmegen Nederland
Contact:

Re: Import RIVM sensor data

Post by Derik »

mmm Is this sensor usefull after the lost of this data??
I cannot find my station [ Nijmegen keizerkarel ]

Looks a great script...
Xu4: Beta Extreme antenna RFXcomE,WU Fi Ping ip P1 Gen5 PVOutput Harmony HUE SolarmanPv OTG Winddelen Alive ESP Buienradar MySensors WOL Winddelen counting RPi: Beta SMAspot RFlinkTest Domoticz ...Different backups
User avatar
EdwinK
Posts: 1820
Joined: Sunday 22 January 2017 21:46
Target OS: Raspberry Pi / ODroid
Domoticz version: BETA
Location: Rhoon
Contact:

Re: Import RIVM sensor data

Post by EdwinK »

NL10741 RIVM Nijmegen-Graafseweg
NL10742 RIVM Nijmegen-Ruyterstraat

Use one of these.
Running latest BETA on a Pi-3 | Toon® Thermostat (rooted) | Hue | Tuya | IKEA tradfri | Dashticz V3 on Lenovo Huawei Tablet | Conbee
Derik
Posts: 1601
Joined: Friday 18 October 2013 23:33
Target OS: Raspberry Pi / ODroid
Domoticz version: BETA
Location: Arnhem/Nijmegen Nederland
Contact:

Re: Import RIVM sensor data

Post by Derik »

EdwinK wrote: Tuesday 22 August 2017 20:54 NL10741 RIVM Nijmegen-Graafseweg
NL10742 RIVM Nijmegen-Ruyterstraat

Use one of these.

Thanks...

Can i run with the internal interface or?

Where can i find the stations??
Xu4: Beta Extreme antenna RFXcomE,WU Fi Ping ip P1 Gen5 PVOutput Harmony HUE SolarmanPv OTG Winddelen Alive ESP Buienradar MySensors WOL Winddelen counting RPi: Beta SMAspot RFlinkTest Domoticz ...Different backups
User avatar
EdwinK
Posts: 1820
Joined: Sunday 22 January 2017 21:46
Target OS: Raspberry Pi / ODroid
Domoticz version: BETA
Location: Rhoon
Contact:

Re: Import RIVM sensor data

Post by EdwinK »

Station:
from the RIVM is avalible to use (http://www.lml.rivm.nl/tabel/)

I created the script external, but I guess you can use the internal editor.
Running latest BETA on a Pi-3 | Toon® Thermostat (rooted) | Hue | Tuya | IKEA tradfri | Dashticz V3 on Lenovo Huawei Tablet | Conbee
Derik
Posts: 1601
Joined: Friday 18 October 2013 23:33
Target OS: Raspberry Pi / ODroid
Domoticz version: BETA
Location: Arnhem/Nijmegen Nederland
Contact:

Re: Import RIVM sensor data

Post by Derik »

mmm internal is working by me...

@ Developer is it perhaps possible to use the internal Python options?
Xu4: Beta Extreme antenna RFXcomE,WU Fi Ping ip P1 Gen5 PVOutput Harmony HUE SolarmanPv OTG Winddelen Alive ESP Buienradar MySensors WOL Winddelen counting RPi: Beta SMAspot RFlinkTest Domoticz ...Different backups
Derik
Posts: 1601
Joined: Friday 18 October 2013 23:33
Target OS: Raspberry Pi / ODroid
Domoticz version: BETA
Location: Arnhem/Nijmegen Nederland
Contact:

Re: Import RIVM sensor data

Post by Derik »

is it perhaps possible that the sensors send a notification??


So i can warn my kids there is smog or something??
What are the values that are good and what are bad values?
Xu4: Beta Extreme antenna RFXcomE,WU Fi Ping ip P1 Gen5 PVOutput Harmony HUE SolarmanPv OTG Winddelen Alive ESP Buienradar MySensors WOL Winddelen counting RPi: Beta SMAspot RFlinkTest Domoticz ...Different backups
Derik
Posts: 1601
Joined: Friday 18 October 2013 23:33
Target OS: Raspberry Pi / ODroid
Domoticz version: BETA
Location: Arnhem/Nijmegen Nederland
Contact:

Re: Import RIVM sensor data

Post by Derik »

There is also a Roet in the data... Is there a option to use this data to?
And the Nijmegen sensor are both not working...
Xu4: Beta Extreme antenna RFXcomE,WU Fi Ping ip P1 Gen5 PVOutput Harmony HUE SolarmanPv OTG Winddelen Alive ESP Buienradar MySensors WOL Winddelen counting RPi: Beta SMAspot RFlinkTest Domoticz ...Different backups
Derik
Posts: 1601
Joined: Friday 18 October 2013 23:33
Target OS: Raspberry Pi / ODroid
Domoticz version: BETA
Location: Arnhem/Nijmegen Nederland
Contact:

Re: Import RIVM sensor data

Post by Derik »

Looks like the sensors from Nijmegen are not in the list:
http://www.lml.rivm.nl/xml/actueel.xml

Perhaps the problem that the siet is getting to a other service?
luchtmeetnet?


Is there a other option?

Love to have to have this option in domoticz...
Xu4: Beta Extreme antenna RFXcomE,WU Fi Ping ip P1 Gen5 PVOutput Harmony HUE SolarmanPv OTG Winddelen Alive ESP Buienradar MySensors WOL Winddelen counting RPi: Beta SMAspot RFlinkTest Domoticz ...Different backups
antonlamers
Posts: 14
Joined: Wednesday 08 March 2017 14:46
Target OS: Raspberry Pi / ODroid
Domoticz version: 2021.1
Location: NL
Contact:

Re: Import RIVM sensor data

Post by antonlamers »

Hi,

It doesn't work in my domoticz.
The virtual text sensor say "hello world"

Anton
Greating,
Anton
Version: 2023.2
Build Hash: 19efd039c
Compile Date: 2023-07-21 17:23:44
dzVents Version: 3.1.8
Python Version: 3.9.2 (default, Feb 28 2021, 17:03:44) [GCC 10.2.1 20210110]
Derik
Posts: 1601
Joined: Friday 18 October 2013 23:33
Target OS: Raspberry Pi / ODroid
Domoticz version: BETA
Location: Arnhem/Nijmegen Nederland
Contact:

Re: Import RIVM sensor data

Post by Derik »

is there perhaps some one that have this script working with the new data...
Hope ...
I like this script for my longs.....
If...
Then i can use my medicine :(
Xu4: Beta Extreme antenna RFXcomE,WU Fi Ping ip P1 Gen5 PVOutput Harmony HUE SolarmanPv OTG Winddelen Alive ESP Buienradar MySensors WOL Winddelen counting RPi: Beta SMAspot RFlinkTest Domoticz ...Different backups
Derik
Posts: 1601
Joined: Friday 18 October 2013 23:33
Target OS: Raspberry Pi / ODroid
Domoticz version: BETA
Location: Arnhem/Nijmegen Nederland
Contact:

Re: Import RIVM sensor data

Post by Derik »

please
Is there a way to use the new data from RIVM?

Love to have this option working..

Thanks
Xu4: Beta Extreme antenna RFXcomE,WU Fi Ping ip P1 Gen5 PVOutput Harmony HUE SolarmanPv OTG Winddelen Alive ESP Buienradar MySensors WOL Winddelen counting RPi: Beta SMAspot RFlinkTest Domoticz ...Different backups
User avatar
phoenixblue
Posts: 76
Joined: Friday 25 November 2016 11:20
Target OS: Windows
Domoticz version:
Contact:

Re: Import RIVM sensor data

Post by phoenixblue »

Sorry for late response, was working on an other project ;-)
I will look into the new Rivm Data to use it.
Will be continu....
Derik
Posts: 1601
Joined: Friday 18 October 2013 23:33
Target OS: Raspberry Pi / ODroid
Domoticz version: BETA
Location: Arnhem/Nijmegen Nederland
Contact:

Re: Import RIVM sensor data

Post by Derik »

Great!
Thanks...

Other domoticz project??!
Xu4: Beta Extreme antenna RFXcomE,WU Fi Ping ip P1 Gen5 PVOutput Harmony HUE SolarmanPv OTG Winddelen Alive ESP Buienradar MySensors WOL Winddelen counting RPi: Beta SMAspot RFlinkTest Domoticz ...Different backups
User avatar
phoenixblue
Posts: 76
Joined: Friday 25 November 2016 11:20
Target OS: Windows
Domoticz version:
Contact:

Re: Import RIVM sensor data

Post by phoenixblue »

"Other domoticz project??!".... Not at all ;-)
Post Reply

Who is online

Users browsing this forum: Google [Bot] and 0 guests