KNMI-WeatherAlert-Domoticz

[Hansh]

Beste Domoticz-gebruikers,

(English text below)

Aangezien de oudere KNMI-feeds en XML-structuren verouderd raken en worden uitgefaseerd, ben ik achter de schermen bezig geweest met het ontwikkelen van een moderne, robuuste en toekomstbestendige oplossing om officiële KNMI-weerwaarschuwingen te integreren in Domoticz.

Het nieuwe script maakt gebruik van het officiële KNMI Data Platform en is gebouwd volgens de modernste standaarden.

Belangrijkste kenmerken van deze nieuwe opzet:

MQTT v5 over WebSockets (TLS): Maakt gebruik van de officiële KNMI Notification Service met ondersteuning voor Session Expiry Intervals om gemiste berichten na een herverbinding netjes op te halen.

GeoJSON & Ruimtelijke Filtering (shapely): Geen grove provincie-indelingen meer, maar een exacte polygoon-check op basis van de GPS-coördinaten van je eigen Domoticz-installatie.

Asynchrone Architectuur: Opgezet met een ontkoppelde achtergrond-thread (loop_forever) voor de netwerkverbinding en een veilige Queue worker voor de file-verwerking, zodat Domoticz of de MQTT-lus nooit vastloopt.

Tijdzone-bewust: Automatische omzetting van UTC-tijdstempels naar lokale Nederlandse tijd (Europe/Amsterdam) inclusief correcte dagweergave.

Robuuste Bewaking: Ingebouwde heartbeat-monitoring om verbindingsproblemen tijdig te signaleren zonder log-spam.

Status:
De code is inmiddels volledig geschreven, geoptimaliseerd en doorgrond (beschikbaar in zowel een Nederlandse als een Engelstalige variant). Zodra het KNMI de definitieve overstap maakt naar deze nieuwe opzet, zal ik het volledige script en de configuratie-instructies hier op het forum publiceren, zodat iedereen hiermee aan de slag kan!


Mocht je nu al ideeën, vragen of suggesties hebben, laat het hier vooral weten.

+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

Dear Domoticz users,

As the older KNMI feeds and XML structures are becoming obsolete and are being phased out, I have been working behind the scenes to develop a modern, robust, and future-proof solution for integrating official KNMI weather warnings into Domoticz.

The new script utilizes the official KNMI Data Platform and has been built according to the latest standards.

Key features of this new setup:

MQTT v5 over WebSockets (TLS): Leverages the official KNMI Notification Service, with support for "Session Expiry Intervals" to ensure that any messages missed during a disconnection are correctly retrieved upon reconnection.

GeoJSON & Spatial Filtering (using the 'shapely' library): Moves beyond broad, province-based categorization by performing precise polygon checks based on the GPS coordinates of your specific Domoticz installation.

Asynchronous Architecture: Designed with a decoupled background thread (`loop_forever`) for network connectivity and a secure queue worker for data processing, ensuring that neither Domoticz nor the MQTT loop ever freezes.

Timezone-Aware: Automatically converts UTC timestamps to local Dutch time (Europe/Amsterdam), including the correct display of the day.

Robust Monitoring: Includes built-in heartbeat monitoring to promptly detect connection issues without flooding your logs.

Status:
The code is now fully written, optimized, and thoroughly tested (available in both Dutch and English versions). As soon as the KNMI completes the transition to this new system, I will publish the full script and configuration instructions here on the forum, so that everyone can get started with it! If you already have any ideas, questions, or suggestions, please feel free to share them here.

[Hansh]

The project is now live on GitHub!

Following up on my previous message, I am happy to let you know that the script has been thoroughly tested and is running stable in production.

Since the transition to the new KNMI platform is progressing, the complete and future-proof package is now publicly available on GitHub!

You can find the repository here, complete with installation tips and configuration guides (available in English):

GitHub Repository: https://github.com/hotting/KNMI-WeatherAlert-Domoticz

What's included:

• The core Python script (v3.7.5) utilizing MQTT v5 over WebSockets and exact GeoJSON polygon filtering (shapely).

• A clean configuration template (WeatherAlertToDomoticz.cfg.example).

• Detailed installation and setup instructions in the README.

Feel free to check it out, test it with your own Domoticz installation, and let me know if you have any feedback, questions, or suggestions!

[Tukker]

The new script utilizes the official KNMI Data Platform
Which is?

[Hansh]

The new script utilizes the official KNMI Data Platform
Which is?

This is the official data platform

https://dataplatform.knmi.nl/dataset/k ... tended-1-0

[Tukker]

Your KNMI-WeatherAlert-Domoticz is interesting stuff because KNMI changes it's warning systematic recently.
No longer on provinces but rather on zones.

I requested an API key from developer.dataplatfor.knmi.nl and did an interesting finding.
Asked for an EDR API
The Environmental Data Retrieval (EDR) API allows you to query datasets on specific parameters in a spatio-temporal manner, reducing the need to download entire datasets. The EDR API adheres to Open Geospatial Consortium EDR standard, facilitating integration with other geospatial tools and systems.
I have two stations nearby, one at 12 km's and one at 21 km's
For sun radiation the 21 km station has better predictions than the nearby station.

KNMI has lots of data to share.

[Hansh]

The KNMI indeed provides a lot of data related to the weather. The KNMI Environmental Data Retrieval (EDR) API provides many different weather parameters, but weather warnings are not included here; these go through a different API.

Weather warnings relatively speaking don't change very much throughout the day. Often, refreshing once or twice a day is more than enough. However, to still remain up to date and not miss any new alerts, KNMI has an MQTT-based procedure to indicate when a new alert is available. So you only need to listen by subscribing to an MQTT channel. When you receive a signal there, you can retrieve the data (usually less than 1 MB) and process it. This way, you stay informed of the weather warnings using minimal resources.

The script I made (https://github.com/hotting/KNMI-WeatherAlert-Domoticz) does precisely that:

Listen on MQTT => Fetch data => Process data => Output to Domoticz

It uses hardly any network traffic or system processing power and runs smoothly in the background.

If you have any questions about the script or the procedures used, please let me know and I will answer them to the best of my ability.