Page 11 of 17

Re: Python Plugin: MqttMapper

Posted: Friday 23 August 2024 15:36
by thaui
FlyingDomotic wrote: Thursday 22 August 2024 16:11 They're 2 classical things that may cause issues:
1) published data is not marked as "retain". In this case, MqttMapper won't be updated until value changes. It's generally a good idea to flag topics "retain", as readers staring after last update could see last value.
2) you made changes into device type/subtype/switchtype/options not allowed by Domoticz. In this case, you have to delete device and restart plug-in in order for a new one to be crated with last settings.
I need to come back on this issue. I have good experiance with some othe devices delivering data to MqttMapper. But the tasmota didn't and I do not find the trick.
Unbenannt.JPG
Unbenannt.JPG (14.33 KiB) Viewed 1518 times

Re: Python Plugin: MqttMapper

Posted: Friday 23 August 2024 16:27
by FlyingDomotic
To help, i need (part of) configuration file showing the incriminated device, and content of MQTT topic used in configuration file.

Re: Python Plugin: MqttMapper

Posted: Friday 23 August 2024 21:02
by thaui
FlyingDomotic wrote: Friday 23 August 2024 16:27 To help, i need (part of) configuration file showing the incriminated device, and content of MQTT topic used in configuration file.
this can be seen in the console of the Tasmota unit
19:38:45.955 MQT: tele/Hichi_F42ED0/SENSOR = {"Time":"2024-08-23T19:38:45","SML":{"server_id":"0a01484c590200155d58","export_total_kwh":0.0000,"total_kwh":1760.8927,"curr_w":532,

this is shown in the MQTT Explorer (no idea why twice) tasmota and tele
Hichi topic.JPG
Hichi topic.JPG (53.89 KiB) Viewed 1501 times
And this is what I added to the MqttMapper.json
"Verbrauch current":{
"topic": "tele/Hichi_F42ED0/SENSOR",
"type": "248", "subtype":"1",
"mapping":{"item":"curr_w"}
}
and here we have the result
Unbenannt.JPG
Unbenannt.JPG (14.34 KiB) Viewed 1501 times

Re: Python Plugin: MqttMapper

Posted: Saturday 24 August 2024 22:27
by FlyingDomotic
I just wrote a small tool aimed at dumping parameters, API and database data, as well as MQTT related topics.

To get it, just update to last version by

Code: Select all

git pull
then restart plugin (or Domoticz) and run dumpMqttMapperValues.py (either by

Code: Select all

./dumpMqttMapperValues.py
or

Code: Select all

python3 dumpMqttMapperValues.py
from plugin folder.

If asked (read Warnings at script startup), you may need to install paho-mqtt and/or sqlite3 python module, using

Code: Select all

pip3 install paho-mqtt
or

Code: Select all

pip3 install sqlite3
Send me dumpMqttMapperValues result by PM (or here).

Re: Python Plugin: MqttMapper

Posted: Sunday 25 August 2024 9:54
by thaui
this is the error log
2024-08-25 12:33:46.063 Error: Mqtt Mapper: Can't find >curr_w< in >{'Time': '2024-08-25T11:33:45', 'SML': {'server_id': '0a01484c590200155d58', 'export_total_kwh': 0.0, 'total_kwh': 1771.478, 'curr_w': 953, 'volt_p1': 236.1, 'volt_p2': 236.5, 'volt_p3': 234.7, 'amp_p1': 1.9, 'amp_p2': 2.8, 'amp_p3': 0.1, 'phase_angle_l2_l1': 120.0, 'phase_angle_l3_l1': 240.0, 'phase_angle_p1': 339.0, 'phase_angle_p2': 36.0, 'phase_angle_p3': 285.0, 'freq': 50}}<

Re: Python Plugin: MqttMapper

Posted: Sunday 25 August 2024 14:41
by FlyingDomotic
I should have been seen that before, it was in your screen dump, sorry...

Value "curr_w" is not as root level in the topic, but in the "SML" branch. So its name is "SML/curr_w" instead of "curr_w".

Try to change:

Code: Select all

"mapping":{"item":"curr_w"}
by

Code: Select all

"mapping":{"item":"SML/curr_w"}

Re: Python Plugin: MqttMapper

Posted: Sunday 25 August 2024 15:47
by thaui
Who the hell is so experianced to understand this details. This did the trick. Thanks a lot.

Re: Python Plugin: MqttMapper

Posted: Saturday 31 August 2024 17:47
by thaui
@FlyingDomoticz
I need another time your help. After you helped me to get the "curr_w" added to devices, I added two new devices for "total_kwh" and "export_total_kwh". I found no chance to add this both and the python file checker told me

Checking MqttMapper.json
Duplicate tele/Hichi_F42ED0/SENSOR node/key found
Duplicate tele/Hichi_F42ED0/SENSOR node/key found

I added "Strom Import kWh" and "Strom Export kWh".

"Strom current W":{
"topic": "tele/Hichi_F42ED0/SENSOR",
"type": "248", "subtype":"1",
"mapping":{"item":"SML/curr_w"}
},
"Strom Import kWh":{
"topic": "tele/Hichi_F42ED0/SENSOR",
"type": "243", "subtype":"28",
"mapping":{"item":"SML/total_kwh"}
},
"Strom Export kWh":{
"topic": "tele/Hichi_F42ED0/SENSOR",
"type": "243", "subtype":"28",
"mapping":{"item":"SML/export_total_kwh"}
}

Re: Python Plugin: MqttMapper

Posted: Saturday 31 August 2024 21:45
by FlyingDomotic
thaui wrote: Saturday 31 August 2024 17:47 @FlyingDomoticz
I need another time your help. After you helped me to get the "curr_w" added to devices, I added two new devices for "total_kwh" and "export_total_kwh". I found no chance to add this both and the python file checker told me

Checking MqttMapper.json
Duplicate tele/Hichi_F42ED0/SENSOR node/key found
Duplicate tele/Hichi_F42ED0/SENSOR node/key found

I added "Strom Import kWh" and "Strom Export kWh".

"Strom current W":{
"topic": "tele/Hichi_F42ED0/SENSOR",
"type": "248", "subtype":"1",
"mapping":{"item":"SML/curr_w"}
},
"Strom Import kWh":{
"topic": "tele/Hichi_F42ED0/SENSOR",
"type": "243", "subtype":"28",
"mapping":{"item":"SML/total_kwh"}
},
"Strom Export kWh":{
"topic": "tele/Hichi_F42ED0/SENSOR",
"type": "243", "subtype":"28",
"mapping":{"item":"SML/export_total_kwh"}
}
This means that multiple devices are connected to the same topic. And as topic is used as key into Domoticz, and Domoticz ask for a unique key, there's a problem.

To solve this, MqttMapper implements a "key" item, that will be used instead of "topic" by Domoticz. there's still the same limitation (key should be unique for each MqttMapper instance), but as you can set the key as you want, this shouldn't be a problem.

Here's an example on how you can modify this configuration file:

Code: Select all

        "Strom current W":{
                "topic": "tele/Hichi_F42ED0/SENSOR",
                "key": "current",
                "type": "248", "subtype":"1",
                "mapping":{"item":"SML/curr_w"}
        },
         "Strom Import kWh":{
                "topic": "tele/Hichi_F42ED0/SENSOR",
                "key": "total",
                "type": "243", "subtype":"28",
                "mapping":{"item":"SML/total_kwh"}
        },
        "Strom Export kWh":{
                "topic": "tele/Hichi_F42ED0/SENSOR",
                "key": "export total",
                "type": "243", "subtype":"28",
                "mapping":{"item":"SML/export_total_kwh"}
As you can put whatever you want into key, as long it's unique, you may also write:

Code: Select all

        "Strom current W":{
                "topic": "tele/Hichi_F42ED0/SENSOR",
                "key": "tele/Hichi_F42ED0/SENSOR/SML/curr_w",
                "type": "248", "subtype":"1",
                "mapping":{"item":"SML/curr_w"}
        },
         "Strom Import kWh":{
                "topic": "tele/Hichi_F42ED0/SENSOR",
                "key": "tele/Hichi_F42ED0/SENSOR/SML/total_kwh",
                "type": "243", "subtype":"28",
                "mapping":{"item":"SML/total_kwh"}
        },
        "Strom Export kWh":{
                "topic": "tele/Hichi_F42ED0/SENSOR",
                "key": "tele/Hichi_F42ED0/SENSOR/SML/export_total_kwh,
                "type": "243", "subtype":"28",
                "mapping":{"item":"SML/export_total_kwh"}
... keeping in mind that:
  • Important things are in "topic" and "mapping":{"item", not in "key"
  • But key, being used as ... key, is linked to device. If you change it, you'll create a new device.

Re: Python Plugin: MqttMapper

Posted: Saturday 31 August 2024 23:00
by HvdW
Why not use

Code: Select all

mosquitto_sub -v -h broker_ip -p 1883 -t '#'   -v 
to catch all the messages?

Re: Python Plugin: MqttMapper

Posted: Saturday 31 August 2024 23:46
by FlyingDomotic
HvdW wrote: Saturday 31 August 2024 23:00 Why not use

Code: Select all

mosquitto_sub -v -h broker_ip -p 1883 -t '#'   -v 
to catch all the messages?
This is more or less what happens in dumpMqttMapperValues.py, adding also MqttMapper configuration file, Domoticz API and database view, only for needed topics.

However, issue here is linked to the fact that same topic is used by multiple devices.

Re: Python Plugin: MqttMapper

Posted: Sunday 01 September 2024 10:54
by thaui
FlyingDomotic wrote: Saturday 31 August 2024 21:45
thaui wrote: Saturday 31 August 2024 17:47 @FlyingDomoticz
I need another time your help. After you helped me to get the "curr_w" added to devices, I added two new devices for "total_kwh" and "export_total_kwh". I found no chance to add this both and the python file checker told me

Checking MqttMapper.json
Duplicate tele/Hichi_F42ED0/SENSOR node/key found
Duplicate tele/Hichi_F42ED0/SENSOR node/key found

I added "Strom Import kWh" and "Strom Export kWh".

"Strom current W":{
"topic": "tele/Hichi_F42ED0/SENSOR",
"type": "248", "subtype":"1",
"mapping":{"item":"SML/curr_w"}
},
"Strom Import kWh":{
"topic": "tele/Hichi_F42ED0/SENSOR",
"type": "243", "subtype":"28",
"mapping":{"item":"SML/total_kwh"}
},
"Strom Export kWh":{
"topic": "tele/Hichi_F42ED0/SENSOR",
"type": "243", "subtype":"28",
"mapping":{"item":"SML/export_total_kwh"}
}
This means that multiple devices are connected to the same topic. And as topic is used as key into Domoticz, and Domoticz ask for a unique key, there's a problem.

To solve this, MqttMapper implements a "key" item, that will be used instead of "topic" by Domoticz. there's still the same limitation (key should be unique for each MqttMapper instance), but as you can set the key as you want, this shouldn't be a problem.

Here's an example on how you can modify this configuration file:

Code: Select all

        "Strom current W":{
                "topic": "tele/Hichi_F42ED0/SENSOR",
                "key": "current",
                "type": "248", "subtype":"1",
                "mapping":{"item":"SML/curr_w"}
        },
         "Strom Import kWh":{
                "topic": "tele/Hichi_F42ED0/SENSOR",
                "key": "total",
                "type": "243", "subtype":"28",
                "mapping":{"item":"SML/total_kwh"}
        },
        "Strom Export kWh":{
                "topic": "tele/Hichi_F42ED0/SENSOR",
                "key": "export total",
                "type": "243", "subtype":"28",
                "mapping":{"item":"SML/export_total_kwh"}
As you can put whatever you want into key, as long it's unique, you may also write:

Code: Select all

        "Strom current W":{
                "topic": "tele/Hichi_F42ED0/SENSOR",
                "key": "tele/Hichi_F42ED0/SENSOR/SML/curr_w",
                "type": "248", "subtype":"1",
                "mapping":{"item":"SML/curr_w"}
        },
         "Strom Import kWh":{
                "topic": "tele/Hichi_F42ED0/SENSOR",
                "key": "tele/Hichi_F42ED0/SENSOR/SML/total_kwh",
                "type": "243", "subtype":"28",
                "mapping":{"item":"SML/total_kwh"}
        },
        "Strom Export kWh":{
                "topic": "tele/Hichi_F42ED0/SENSOR",
                "key": "tele/Hichi_F42ED0/SENSOR/SML/export_total_kwh,
                "type": "243", "subtype":"28",
                "mapping":{"item":"SML/export_total_kwh"}
... keeping in mind that:
  • Important things are in "topic" and "mapping":{"item", not in "key"
  • But key, being used as ... key, is linked to device. If you change it, you'll create a new device.
Thanks a lot, this did it.

Re: Python Plugin: MqttMapper

Posted: Tuesday 03 September 2024 9:10
by TiXav
I use it since yesterday, super plugin!
but for those that try to move a swith where the topic value is "1" corresponding to "ON" in domoticz. You must write "1.0" instead of "1" (otherwaise the swalue stay at 1 and the swith does not change)
example:

"Portillon BLU":{
"topic": "shelliesX/PortillonX",
"key":"Door",
"type": "244", "subtype": "73", "switchtype": "0",
"mapping": {"item": "Window","default": "0", "values": {"1.0":"1","0":"0"}}
},
I find out for 2 days the solution
Someone can confirm this ?

Re: Python Plugin: MqttMapper

Posted: Tuesday 03 September 2024 11:26
by FlyingDomotic
TiXav wrote: Tuesday 03 September 2024 9:10 I use it since yesterday, super plugin!
but for those that try to move a swith where the topic value is "1" corresponding to "ON" in domoticz. You must write "1.0" instead of "1" (otherwaise the swalue stay at 1 and the swith does not change)
example:

"Portillon BLU":{
"topic": "shelliesX/PortillonX",
"key":"Door",
"type": "244", "subtype": "73", "switchtype": "0",
"mapping": {"item": "Window","default": "0", "values": {"1.0":"1","0":"0"}}
},
I find out for 2 days the solution
Someone can confirm this ?
Mapping is used to associate a value extracted from MQTT topic to Domoticz value. As MQTT topic depends on application sending data, every value can be found for a particular case ("On", "ON", "on", 1, "Open", "Active"...)

To see values written in MQTT, you can use tools like MQTT Explorer. Unfold (sub) topics until you find the right one.

You may also use dumpMqttMapperValues.py, in plugin folder, it will show (almost) everything related to MqttMapper devices (MqttMapper definition, Domoticz API and database values, MQTT topic value. Use "--showchanges" to wait for changes during one minute (use "--wait=<n> to wait <n> minutes), letting you time to force a change on MQTT.

Re: Python Plugin: MqttMapper

Posted: Wednesday 04 September 2024 20:22
by rocknsalsa
I need some help. :roll:
I am trying the MqttMapper plugin since several days with little success. Certainly I miss something ...

I am using domoticz 2024.7 with MQTT. MQTT receives data from a Balboa spa / MQTT interface (using "https://github.com/ccutrer/balboa_worldwide_app/wiki" applications). Note: the interface is working: I am able to control the Balboa interface through "Mqtt Explorer".

One the of Balboa devices (a LED switch) has the followig HomeAssistant definition :

{"name":"Hot Tub Lights",
"availability_topic":"homie/bwa/$state",
"payload_available":"ready",
"payload_not_available":"lost",
"qos":1,
"icon":"mdi:car-parking-lights",
"device":{
"manufacturer":"Balboa Water Group",
"sw_version":"2.1.7",
"model":"CN6013X",
"name":"BWA Link",
"identifiers":"bwa"},
"state_topic":"homie/bwa/spa/light1",
"command_topic":"homie/bwa/spa/light1/set",
"payload_off":"false",
"payload_on":"true",
"unique_id":"bwa_spa_light1"
}

So the important inputs are (in line with what I see in MQTT Explorer):
"state_topic":"homie/bwa/spa/light1",
"command_topic":"homie/bwa/spa/light1/set",
"payload_off":"false",
"payload_on":"true",

As a consequence, I wrote the MqttMapper config file:

{
"Spa LED test": {"topic": "homie/bwa/spa/light1",
"type": "244", "subtype": "73", "switchtype": "0",
"set": {"topic": "homie/bwa/spa/light1/set", "payload": "#"},
"mapping": {"item": "", "default": "false", "values": {"true": "100", "false": "0"}}
}
}
Note: after some search in this discussion, I found that "100" is the true state of a domoticz switch.

My issue:
The switch is well created in domoticz. But:
- the current switch status is not in line with MQTT explorer status
- the control seems to work (at least MQTT explorer shows in the history that a command is received).

In particular, when I use Mqtt Explorer to manually change the values in Mqtt (command_topic":"homie/bwa/spa/light1/set) and the attached balboa interface, the balboa interface is working, but .... nothing is happening on domoticz side.

The domoticz logs :

when activating the domoticz switch: it seems to work :)
2024-09-04 20:10:40.034 Status: User: admin (IP: xxxxxx) initiated a switch command (176/MM_brax_test - Spa LED test/On)
2024-09-04 20:10:40.076 MM_brax_test: 001/MM_brax_test - Spa LED test, homie/bwa/spa/light1: Command: 'On', Level: 0, Color:
2024-09-04 20:10:40.077 MM_brax_test: Setting homie/bwa/spa/light1 to >true<

putting "false" through MQTT explorer: putting false
2024-09-04 20:11:44.480 MM_brax_test: testValue="true" (<class 'str'>), readValue="0.0" (<class 'str'>)
2024-09-04 20:11:44.480 MM_brax_test: testValue="false" (<class 'str'>), readValue="0.0" (<class 'str'>)
2024-09-04 20:11:44.481 MM_brax_test: Setting MM_brax_test - Spa LED test to >false<

putting "true" through MQTT explorer: putting again false (and not true as expected) ;)
2024-09-04 20:13:42.680 MM_brax_test: testValue="true" (<class 'str'>), readValue="1.0" (<class 'str'>)
2024-09-04 20:13:42.680 MM_brax_test: testValue="false" (<class 'str'>), readValue="1.0" (<class 'str'>)
2024-09-04 20:13:42.681 MM_brax_test: Setting MM_brax_test - Spa LED test to >false<

So, what is wrong in my config file ?
Thanks for your advice.

Re: Python Plugin: MqttMapper

Posted: Wednesday 04 September 2024 23:38
by waltervl
If it is home assistant autodiscovery compatible (it seems so) you do not need mqtt mapper plugin but can use the Domoticz mqtt Autodiscovery gateway https://www.domoticz.com/wiki/MQTT#Add_ ... Gateway.22
Fill in the right parameters and Domoticz will find the right mqtt config topics and create the devices as needed.

Re: Python Plugin: MqttMapper

Posted: Wednesday 04 September 2024 23:49
by FlyingDomotic
In fact, there are 2 reasons:
1) I made a change to have specific test when value is boolean (as Python convert them to 0 and 1, as shown in the log you copied here).
2) Configuration file should be modified as:

Code: Select all

{
	"Spa LED test": {
	"topic": "homie/bwa/spa/light1",
	"type": "244", "subtype": "73", "switchtype": "0",
	"set": {"topic": "homie/bwa/spa/light1/set", "payload": "#"},
	"mapping": {"item": "", "default": "0", "values": {"True": "100", "False": "0"}}
	}
}

Re: Python Plugin: MqttMapper

Posted: Thursday 05 September 2024 9:35
by rocknsalsa
First of all, thanks for your quick answer and help. :D :D

I tried out your proposed change (with a small adaptation from "True" to "true" and "False" to "false").

But there is no improvement. When switching "manually" with "MQTT Explorer" the "true" state is still "0/0". See the domoticz log below:

setting to "true":
2024-09-05 09:25:45.357 MM_brax_test: testValue="true" (<class 'str'>), readValue="1.0" (<class 'str'>)
2024-09-05 09:25:45.368 MM_brax_test: testValue="false" (<class 'str'>), readValue="1.0" (<class 'str'>)
2024-09-05 09:25:45.368 MM_brax_test: Setting MM_brax_test - Spa LED test to 0/0

setting to "false":
2024-09-05 09:26:51.468 MM_brax_test: testValue="true" (<class 'str'>), readValue="0.0" (<class 'str'>)
2024-09-05 09:26:51.468 MM_brax_test: testValue="false" (<class 'str'>), readValue="0.0" (<class 'str'>)
2024-09-05 09:26:51.470 MM_brax_test: Setting MM_brax_test - Spa LED test to 0/0

Could there be an other issue ?
Thanks.

Re: Python Plugin: MqttMapper

Posted: Thursday 05 September 2024 9:46
by rocknsalsa
@waltervl

Thanks for your comment. :D

Yes you are right, I have effectively all the associated devices created in domoticz and they are reflecting in real-time the real state of the interface.
But the problem is that domoticz sent the device command action (On/Off) in a json format ( "{value : true}" ), whereas the Balboa interface expects a raw value ( "true" ). So it does not work from domoticz to MQTT (Balboa interface).
Therefor I try the Mqtt Mapper.
If you know how to change domoticz behaviour sending a command to MQTT, you are welcome.

Re: Python Plugin: MqttMapper

Posted: Thursday 05 September 2024 16:54
by waltervl
rocknsalsa wrote: Thursday 05 September 2024 9:46 @waltervl

Thanks for your comment. :D

Yes you are right, I have effectively all the associated devices created in domoticz and they are reflecting in real-time the real state of the interface.
But the problem is that domoticz sent the device command action (On/Off) in a json format ( "{value : true}" ), whereas the Balboa interface expects a raw value ( "true" ). So it does not work from domoticz to MQTT (Balboa interface).
Therefor I try the Mqtt Mapper.
If you know how to change domoticz behaviour sending a command to MQTT, you are welcome.
Or change Balboa interface that it accepts the JSON format. Perhaps a value template has to be set in the config topic