Homeassistant to Domoticz MQTT Autodiscovery

[rwblinn]

Hi All,

Sharing my experience of receiving data from Home Assistant (HA) into Domoticz using MQTT Autodiscovery (MQTTAD).

Purpose
To send data from HA to Domoticz for historical data logging.
Example logging the HA entity Solar Battery Charge State (0-100%).

Background
After using Domoticz for almost 10 years, decided to migrate to HA mainly for its modern User Interface & Integration support (esp. for Homematic).
HA is running well since few month, but I am still a great Domoticz fan. Therefor have been seeking on how to use both HA and Domoticz.
Decided to use Domoticz

• as a HA historical data logger for selective entities (as described next) - used HighCharts in the past extensively.
• for special functions (in progress) - developed in dzVents/Lua.

Next is an outline how I have setup MQTTAD for HA historical data logging in Domoticz.
Please note that the basic setup of the HA MQTT Integration and Domoticz MQTTAD hardware is not explained here but referred to.

Be Aware
This is a first attempt using MQTTAD in both HA & Domoticz, so appreciate any improvement hints.

Solution
The solution is based upon HA MQTT Autodiscovery.
Setup HA
This is described here.
For every HA entity an Automation is created to publish an MQTT message with the entity state as payload.

Setup Domoticz
Add hardware MQTT Auto Discovery Client Gateway as outlined here.
The Auto Discovery Prefix is homeassistant. This means a MQTTAD config topic should start with homeassistant.
It is advised, that the state topic does not start with prefix homeassistant.

MQTT Explorer
The MQTT Explorer is used to check the MQTT messages and to delete topics.
Example Domoticz log deleting topic:

Code: Select all

2024-10-06 18:15:22.464 Debug: MQTTAD Client: topic: ha2domoticz/sensor/solar_bat_charge_state/state, message:
2024-10-06 18:15:22.565 Debug: MQTTAD Client: topic: homeassistant/sensor/solar_bat_charge_state/config, message:

MQTTAD Messages
The device configuration & state MQTT messages are submitted using Mosquitto client (mosquitto_pub) issued from a terminal (opened on the Domoticz system).
MQTTAD messages published are

• the device configuration (/config) and
• first value (/state) which creates the device in Domoticz.

Summary
For each device in Domoticz an

• HA Automation is created to MQTT publish the state of the HA entity.
• MQTTAD configuration message is published once to HA - Important: topic prefix is homeassistant and the device_class is the HA device_class as well as the associated unit_of_measurement.
• MQTT state message is initially published to autocreate the device in Domoticz - Important: topic prefix should not be homeassistant.
• MQTT state message is received from HA to update the device value.

Examples
Two examples are shared, the Solar Power Battery Charge State (%) and the Gas Meter (m3).

Device Power Battery Charge State
Get the value of the HA entity sensor.power_battery_charge_state into a Domoticz device (General/Percentage).
This device is a battery used to supply power to a house. It is fed by solar panels.

HA Automation
Create an HA automation to MQTT publish the state of the entity.
Topic:

Code: Select all

ha2domoticz/sensor/solar_bat_charge_state/state

Payload:

Code: Select all

'{{ states("sensor.power_battery_charge_state") }}'

Automation YAML:

Code: Select all

- id: '1727942232660'
  alias: MQTT Power Battery Charge State
  description: Publish solar battery charge state in pct 0-100.
  triggers:
  - entity_id:
    - sensor.power_battery_charge_state
    trigger: state
  conditions: []
  actions:
  - action: mqtt.publish
    metadata: {}
    data:
      evaluate_payload: false
      qos: 0
      retain: true
      topic: ha2domoticz/sensor/solar_bat_charge_state/state
      payload: '{{ states("sensor.power_battery_charge_state") }}'
  mode: single

MQTTAD Config Message
The MQTT configuration message to create the device:

Code: Select all

mosquitto_pub -r -h ha-ip -p 1883 -u USERNAME -P PASSWORD -t "homeassistant/sensor/solar_bat_charge_state/config" -m '{"value_template": "{{ value }}","device_class": "battery","state_topic": "ha2domoticz/sensor/solar_bat_charge_state/state","name": "SolarBatChargeState","unique_id": "HASOLAR1", "unit_of_measurement":"%" }'

Domoticz Log:

Code: Select all

2024-10-07 17:09:04.443 Debug: MQTTAD Client: topic: homeassistant/sensor/solar_bat_charge_state/config, message: {"value_template": "{{ value }}","device_class": "battery","state_topic": "ha2domoticz/sensor/solar_bat_charge_state/state","name": "SolarBatChargeState","unique_id": "HASOLAR1", "unit_of_measurement":"%" }
2024-10-07 17:09:04.445 Status: MQTTAD Client: Discovered: SolarBatChargeState/SolarBatChargeState (unique_id: HASOLAR1) 

MQTTAD State Message
The MQTT state message to set the initial device value:

Code: Select all

mosquitto_pub -r -h ha-ip -p 1883 -u USERNAME -P PASSWORD -t "ha2domoticz/sensor/solar_bat_charge_state/state" -m 100 

Domoticz Log:

Code: Select all

2024-10-06 18:17:31.610 Debug: MQTTAD Client: topic: ha2domoticz/sensor/solar_bat_charge_state/state, message: 100

Domoticz Devices List
The device is listed in the Domoticz devices list with Name SolarBatChargeState, Type General, Subtype Percentage.

Device Update
Domoticz log entry when receiving device update message from HA.

Code: Select all

2024-10-07 17:18:29.883 Debug: MQTTAD Client: topic: ha2domoticz/sensor/solar_bat_charge_state/state, message: 100
2024-10-07 17:18:29.890 MQTTAD Client: General/Percentage (SolarBatChargeState) 

Note
The device is not listed in the MQTTAD hardware setup because it is a sensor and not a number device.

Device Gas Meter
Get the value of the HA entity sensor.gas_meter into a Domoticz device (RFXMeter/RFXMeter Counter).

HA Automation
Create an HA automation to MQTT publish the state of the entity.
Topic:

Code: Select all

ha2domoticz/sensor/gas_meter/state

Payload:

Code: Select all

'{{ states("sensor.gas_meter") | float(0) }}'

Automation YAML:

Code: Select all

- id: '1711281225175'
  alias: MQTT Gas Meter
  description: Publish sensor gas meter
  trigger:
  - platform: state
    entity_id:
    - sensor.gas_meter
  condition: []
  action:
  - metadata: {}
    data:
      qos: 0
      retain: true
      payload: '{{states("sensor.gas_meter") | float(0) }}'
      topic: ha2domoticz/sensor/gas_meter/state
    action: mqtt.publish
  mode: single

MQTTAD Config Message
The MQTT configuration message to create the device:

Code: Select all

mosquitto_pub -r -h ha-ip -p 1883 -u USERNAME -P PASSWORD -t "homeassistant/sensor/gas_meter/config" -m '{"value_template": "{{ value }}","device_class": "gas","state_topic": "ha2domoticz/sensor/gas_meter/state","name": "GasMeter","unique_id": "HAGASMETER1", "unit_of_measurement":"m³" }'

Domoticz Log:

Code: Select all

 2024-10-07 17:16:31.600 Debug: MQTTAD Client: topic: homeassistant/sensor/gas_meter/config, message: {"value_template": "{{ value }}","device_class": "gas","state_topic": "ha2domoticz/sensor/gas_meter/state","name": "GasMeter","unique_id": "HAGASMETER1", "unit_of_measurement":"m³" }
2024-10-07 17:16:31.602 Status: MQTTAD Client: Discovered: GasMeter/GasMeter (unique_id: HAGASMETER1) 

MQTTAD State Message
The MQTT state message to set the initial device value:

Code: Select all

mosquitto_pub -r -h ha-ip -p 1883 -u USERNAME -P PASSWORD -t "ha2domoticz/sensor/gas_meter/state" -m 2900

Domoticz Log:

Code: Select all

2024-10-07 17:21:08.394 Debug: MQTTAD Client: topic: ha2domoticz/sensor/gas_meter/state, message: 2900

Domoticz Devices List
The device is listed in the Domoticz devices list Type RFXMeter, Subtype RFXMeter Counter kWh.

Device Update
Domoticz log entry when receiving device update message from HA.

Code: Select all

2024-10-07 17:24:25.920 Debug: MQTTAD Client: topic: ha2domoticz/sensor/gas_meter/state, message: 2963.25
2024-10-07 17:24:25.928 MQTTAD Client: RFXMeter/RFXMeter counter (GasMeter) 

Note
The device is not listed in the MQTTAD hardware setup because it is a sensor and not a number device.

Hints
Various hints
MQTT State Topic
It is advised (see post #2) not to use the word homeassistant as prefix for a state topic.

Unable to parse
After the device has been created in Domoticz, restart HA and check the log if there are any complains about the MQTT message config payload.
Examples:
Unable to parse JSON solar_bat_level:

Code: Select all

'{"value_template": "{{ value }}","device_class": "battery","state_topic": "ha2domoticz/sensor/solar_bat_level/state","name": "SolarBatteryLevel","unique_id": "HASOLAR1", "unit_of_measurement":"%" }}'

This is caused by double }} instead of a single } and the end of the payload.
The mosquitto client did not complain about this by the way.

Wrong device class
Ensure to use the right HA device class. These are listed in the error message.
In the example, do not use device class "number" but "battery".

Code: Select all

Error 'expected SensorDeviceClass or one of 'date', 'enum', 'timestamp', 'apparent_power', 'aqi', 'atmospheric_pressure', 'battery', 'carbon_monoxide', 'carbon_dioxide', 'conductivity', 'current', 'data_rate', 'data_size', 'distance', 'duration', 'energy', 'energy_storage', 'frequency', 'gas', 'humidity', 'illuminance', 'irradiance', 'moisture', 'monetary', 'nitrogen_dioxide', 'nitrogen_monoxide', 'nitrous_oxide', 'ozone', 'ph', 'pm1', 'pm10', 'pm25', 'power_factor', 'power', 'precipitation', 'precipitation_intensity', 'pressure', 'reactive_power', 'signal_strength', 'sound_pressure', 'speed', 'sulphur_dioxide', 'temperature', 'volatile_organic_compounds', 'volatile_organic_compounds_parts', 'voltage', 'volume', 'volume_storage', 'volume_flow_rate', 'water', 'weight', 'wind_speed' for dictionary value @ data['device_class']' when processing MQTT discovery message topic: 'homeassistant/sensor/gas_meter/config', message: '{'value_template': '{{ value }}', 'device_class': 'number', 'state_topic': 'ha2domoticz/sensor/gas_meter/state', 'unique_id': 'GASMETER1', 'unit_of_measurement': 'm³', 'name': 'GasMeter', 'platform': 'mqtt'}'

After HA restart, check the log if the MQTTAD components are created (first time only because retained flag):

Code: Select all

2024-10-06 13:14:53.067 INFO (MainThread) [homeassistant.components.mqtt.discovery] Found new component: sensor solar_bat_level
2024-10-06 13:14:53.074 INFO (MainThread) [homeassistant.components.mqtt.discovery] Found new component: sensor gas_meter
2024-10-06 13:14:59.989 INFO (MainThread) [homeassistant.components.mqtt.client] MQTT client initialized, birth message sent

Topic containing (reserved) word
If the topic contains the word battery, the device is not created in Domoticz although Domoticz discovered the device stated in the log.
Renaming in the configuration topic the word battery to bat did work. Have not tested with other words, like power.
It took a while to find out why Domoticz did not create the device and list in the Devices tab.

Domoticz Stop/Start
Helpful terminal (CLI) commands during testing:

Code: Select all

sudo systemctl start domoticz.service
sudo systemctl stop domoticz.service

[waltervl]

Hi All,

Sharing my experience of receiving data from Home Assistant (HA) into Domoticz using MQTT Autodiscovery (MQTTAD).
.......
Setup Domoticz
Add hardware MQTT Auto Discovery Client Gateway as outlined here.
The Auto Discovery Prefix is **homeassistant**. This means all MQTTAD topics must start with **homeassistant**.

This does NOT mean all MQTTAD topics must start with **homeassistant** but the AD config topics should start start with **homeassistant**.
It is better to have the state and set topics not in the same topic as the AD topic as it can cause confusion for Domoticz finding the config topics.

See also example in https://www.domoticz.com/wiki/MQTT_AD_R ... ms#Example

[rwblinn]

Thanks for further clarification. Have corrected the statement.

Not clear what you mean by:
"It is better to have the state and set topics not in the same topic as the AD topic as it can cause confusion for Domoticz finding the config topics."
Could you share another example?

What is advised to use for the shared example:
Config Message
Topic: "homeassistant/sensor/gas_meter/config"
Payload: '{"value_template": "{{ value }}","device_class": "gas","state_topic": "homeassistant/sensor/gas_meter/state","name": "GasMeter","unique_id": "GASMETER1", "unit_of_measurement":"m³" }'
State Message
Topic: "homeassistant/sensor/gas_meter/state"
Payload: 2958

[waltervl]

you should create something like below. Then domoticz looks for the config topucs and creates the devices as requested. It will look (and perhaps write) in the MQTT topics mentioned in the config topics for state and commands messages.
Config Message
Topic: "homeassistant/sensor/gas_meter/config"
Payload: '{"value_template": "{{ value }}","device_class": "gas","state_topic": "dataexchange/sensor/gas_meter/state","name": "GasMeter","unique_id": "GASMETER1", "unit_of_measurement":"m³" }'

State Message
Topic: "dataexchange/sensor/gas_meter/state"
Payload: 2958

Additional for batteries: Domoticz normally does not create battery status devices but adds the battery state to the device/sensor/switch it is connect to.

[rwblinn]

Thanks for help = appreciated. Now it is clear how to setup and use MQTTAD.
Have updated post #1 accordingly = used the prefix ha2domoticz for the state topics.

Please note that the battery device is a solar system battery used to power a house. The Domoticz device General/Percentage is fine for that.

What is left is how to get the devices shown up in the Domoticz Hardware MQTT Auto Discovery Client Gateway.
Not so important for now because the created devices can be used.

[waltervl]

What is left is how to get the devices shown up in the Domoticz Hardware MQTT Auto Discovery Client Gateway.
Not so important for now because the created devices can be used.

Please explain your issue in more detail...
If you mean that it is not clear how Domoticz decides how Domoticz decides how it converts AD config topics into Domoticz devices you should check the source code as it is not documented. Perhaps you could also check the Shelly teacher for Domoticz project that converts Shelly devices into AD config topics
https://github.com/enesbcs/shellyteacher4domo

[rwblinn]

Hi,
thanks again for further info.

What I meant is that the two devices created using MQTTAD are listed in the Domoticz Devices List (with hardware MQTTAD Client) but not in the Setup of the Hardware Controller MQTT Auto Discovery Client Gateway (named MQTTAD Client).

Exploring the shellyteacher4domo = good info.

Looked up the Domoticz source MQTTAutoDiscover.cpp: used "sensor" in the config & state topic from the allowed_components = { ... "sensor" ... }.
... but somehow missing a link between the MATTAD config topic and Domoticz MQTTAD Hardware Controller Configuration list of devices.

Nevertheless will continue adding more HA entities esp. to populate the Domoticz Energy Dashboard.

[waltervl]

Hi,
thanks again for further info.

What I meant is that the two devices created using MQTTAD are listed in the Domoticz Devices List (with hardware MQTTAD Client) but not in the Setup of the Hardware Controller MQTT Auto Discovery Client Gateway (named MQTTAD Client).

See wiki https://www.domoticz.com/wiki/MQTT#MQTT ... very_Setup Only used when the device is a number device and used to configure the behavior of discovered devices (eg motion sensor off delay).

Nevertheless will continue adding more HA entities esp. to populate the Domoticz Energy Dashboard.

I thought HA already has its own Energy Dashboard for a long time. I saw remarks the Domoticz dashboard was based on the HA dashboard...

[rwblinn]

I thought HA already has its own Energy Dashboard for a long time. I saw remarks the Domoticz dashboard was based on the HA dashboard...

Yes and using ... but for my Domoticz as Datalogger setup want to store same data, so that is why will use Domoticz Energy Dashboard as well.

Think we are good for now = thanks for help and will keep posted on the progress.