Python Plugin: MqttMapper

[FlyingDomotic]

It seems that file /opt/domoticz/userdata/plugins/MqttMapper/MqttMapper.json don't exist, is not readable or contains invalid JSON.

Anyway, errors following first message are fixed in last version.

[Hencor]

I‘m trying to get a connection between my smartmeter read out by Iobroker and domoticz with mqtt messages. My question is if I can map two different topics to one device?
For example the counter [kWh] is the topic

Code: Select all

alias/0/main/Counter

and the power [W]

Code: Select all

alias/0/main/power

The topic message looks like this:

Code: Select all

{
  "val": 8249.2442,
  "ack": true,
  "ts": 1679233073032,
  "q": 0,
  "from": "system.adapter.smartmeter.0",
  "user": "system.user.admin",
  "lc": 1679233073032
}

[FlyingDomotic]

If I correctly understood, idea is to have a KWh device, which will read instant power in topic alias/0/main/power and total KWh counter in alias/0/main/Counter. Is this correct ? Values to read are in "val" item of json mesage. Right?

If this is the case, the easiest way today is to create 2 devices, and an LUA or dzVents script to concatenate the 2 values into sValue of a KWh device.

But it's not impossible that in a near future (let's say tomorrow), a new version allows to change only part of a sValue (today, only full sValues could be set).

[FlyingDomotic]

Future is here!

The new version now allows to write something like:

Code: Select all

{
      "My KWh counter" : {
        "topic": "plug/power",
        "key": "My KWh counter",
        "type": "248", "subtype": "1", "switchtype": "0",
        "mapping": {"item": "val;~"}
      },
      "My KWh counter2" : {
        "topic": "plug/counter",
        "key": "My KWh counter",
        "type": "248", "subtype": "1", "switchtype": "0",
        "mapping": {"item": "~;val"}
      }
}

[Hencor]

Okay. Thanks for the fast response and solution. That was exactly what I was looking for.

Unfortunately I'm struggeling with two issues right now.
1. All my plugins now changed the type to MQTT Mapper as I tried to add the hardware... See IDX 11 and 12.

Hardware.PNG (64.78 KiB) Viewed 1564 times

For now they are working still as expected, but I cannot change any settings now. Also when I click on MQTT Mapper the Hardware type is not selected. I can only change setting when I'm deleting the hardware and add again.
There seems to be something completely gone wrong.

2. The MQTT Mapper doesn't find or can't read the config file:

Code: Select all

 2023-03-20 14:23:12.460 MQTT Mapper: 'Language':'de'
2023-03-20 14:23:12.460 MQTT Mapper: 'Version':'1.0.9'
2023-03-20 14:23:12.460 MQTT Mapper: 'Author':'Flying Domotic'
2023-03-20 14:23:12.460 MQTT Mapper: 'Name':'MQTT Mapper'
2023-03-20 14:23:12.460 MQTT Mapper: 'Address':'127.0.0.1'
2023-03-20 14:23:12.460 MQTT Mapper: 'Port':'1883'
2023-03-20 14:23:12.460 MQTT Mapper: 'Key':'MqttMapper'
2023-03-20 14:23:12.460 MQTT Mapper: 'Mode1':'MqttMapper.json'
2023-03-20 14:23:12.460 MQTT Mapper: 'Mode6':'Normal'
2023-03-20 14:23:12.460 MQTT Mapper: 'DomoticzVersion':'2023.1'
2023-03-20 14:23:12.460 MQTT Mapper: 'DomoticzHash':'f9b9ac774'
2023-03-20 14:23:12.460 MQTT Mapper: 'DomoticzBuildTime':'2023-02-14 15:06:40'
2023-03-20 14:23:12.460 MQTT Mapper: Device count: 0
2023-03-20 14:23:12.460 MQTT Mapper: Debug logging mask set to: PYTHON PLUGIN
2023-03-20 14:23:12.278 Status: MQTT Mapper: Started.
2023-03-20 14:23:12.278 Status: MQTT Mapper: Entering work loop.
2023-03-20 14:23:12.457 Status: MQTT Mapper: Initialized version 1.0.9, author 'Flying Domotic'
2023-03-20 14:23:12.461 Error: MQTT Mapper: Error reading JSON file /home/pi/domoticz/plugins/MqttMapper/MqttMapper.json

The config file is this:

Code: Select all

  }
	"My KWh counter" : {
        "topic": "smartmeter/0/1-0:16_7_0__255/value",
        "key": "My KWh counter",
        "type": "248", "subtype": "1", "switchtype": "0",
        "mapping": {"item": "val;~"}
      },
	  
      "My KWh counter2" : {
        "topic": "smartmeter/0/1-0:1_8_0__255/value",
        "key": "My KWh counter",
        "type": "248", "subtype": "1", "switchtype": "0",
        "mapping": {"item": "~;val"}
      }
	}

[Kedi]

Change the first char.

[Hencor]

Oh wow.

But problem 1 persists.

[FlyingDomotic]

Domoticz seems, in certain cases, not liking any change in Python plugins. The easiest way I found to fix this is to restart Domoticz when updating a plug-in code file. That's what I do with both 2020.2 and 2021.1 and should probably works with other versions.

[Hencor]

The easiest way I found to fix this is to restart Domoticz when updating a plug-in code file.

Well, several restarts, reboots and also database restores seems not to solve this mess. Any further ideas how to solve, I think of this display error? Sorry for hijack this thread a bit.

And another thing is that this kWh counter does not show the kWh but only the current power (watts). But in the past it was also pretty confusing to me with the many different counter types available in dzvents, virtual switch hardware, etc.

[FlyingDomotic]

Concerning bad naming of plug-in that are not MqttMapper (lines 11 and 12), I gave all my knowledge with Domoticz restart. If this persists, this is probably a Domoticz bug... Could you try removing MqttMapper and see what happens to lines 11 & 12? Are they taking name of last plug-in?

Just for my understanding, what version of Domoticz are you using?

Concerning KWh counters (at least for those of type "P1 smart Meter, Energy), the "main" data is instant power. However, looking at text inside the widget, you'll find something like today's consumption (in KWh), and total consumption. Is this the case?

[Hencor]

Concerning bad naming of plug-in that are not MqttMapper (lines 11 and 12), I gave all my knowledge with Domoticz restart. If this persists, this is probably a Domoticz bug... Could you try removing MqttMapper and see what happens to lines 11 & 12? Are they taking name of last plug-in?

I investigated a bit more. When I delete the MqttMapper plugin folder inside the domoticz/plugins directory the DeCONZ plugin (or the display) looks as it should. But even when I clone the folder from github again, without adding the MqttMapper hardware it changes.
- I tried also to clone the folder with a different name - Mqtt_Mapper, but it changes also the DeCONZ hardware entry.
- I tried with a different plugin (PP-Manager). When I clone this folder nothing happens to the DeCONZ hardware entry.
I'm running the newest version 2023.1 - Python Version: 3.7.3

I will open a bug report in the forum section.


Back to the actual topic:
I changed the type of the meter now to 243 and subtype to 29. This is according to the table https://www.domoticz.com/wiki/Developin ... vice_Types the instant + counter device and it is working

Very very very nice! Thanks for the quick help!

[Hencor]

I'm a little bit hooked to this plugin right now

I have two more questions.
1. It seems that the multiplier is not applied.
The value is 8264.8734 kWh and should be multiplied with 1000 for a Wh value.

Code: Select all

 "Haus" : {
        "topic": "smartmeter/0/1-0:16_7_0__255/value",
        "key": "Haus",
        "type": "243", "subtype": "29", "switchtype": "0",
        "mapping": {"item": "val;~"}
      },
      "Haus 2" : {
        "topic": "smartmeter/0/1-0:1_8_0__255/value",
        "key": "Haus",
        "type": "243", "subtype": "29", "switchtype": "0",
        "mapping": {"item": "~;val", "multiplier": 1000}
       },

I already germanized my json

Code: Select all

2023-03-21 13:45:19.331 MQTT: onMQTTConnected found smartmeter/0/1-0:16_7_0__255/value, Device 'MQTT - Haus'
2023-03-21 13:45:19.332 MQTT: Setting MQTT - Haus to >215;8264.8734<

2. The next step was to have a different type of counter with 2 tarrifs, a P1 Smart Meter. But it seems not to function correctly. This is the log and the device only shows invalid. May this just disappear with the next update of values? I have a low consumption and a low resolution so this may take a while and as I said before I'm hooked right now and a little impatient

Code: Select all

2023-03-21 13:44:39.381 Status: Warning: Expecting svalue with at least 5 elements separated by semicolon, 0 elements received (""), notification not sent (Hardware: 19 - MQTT, ID: Scheune, Unit: 2, Type: FA - P1 Smart Meter, SubType: 1 - Energy)
2023-03-21 13:44:39.408 Status: Warning: Expecting svalue with at least 5 elements separated by semicolon, 0 elements received (""), notification not sent (Hardware: 19 - MQTT, ID: Scheune, Unit: 2, Type: FA - P1 Smart Meter, SubType: 1 - Energy)

[FlyingDomotic]

1. It seems that the multiplier is not applied.

You're right. As of now, multiplier is only applied when value to set is numeric (loaded in nValue). Here, we're loading a string with 2 numerical value. This clearly is a bug on my side It has been fixed in 1.0.10. Thanks for your feedback!

2. The next step was to have a different type of counter with 2 tarrifs, a P1 Smart Meter. But it seems not to function correctly.

Electricity (instant and counter) needs 2 values: POWER;ENERGY

Electricity P1 smart meter needs up to 6 values: USAGE1;USAGE2;RETURN1;RETURN2;CONS;PROD
▪ USAGE1= total energy usage meter tariff 1 in Wh.
▪ USAGE2= total energy usage meter tariff 2 in Wh.
▪ RETURN1= total energy return meter tariff 1 in Wh.
▪ RETURN2= total energy return meter tariff 2 in Wh.
▪ CONS= actual usage power (Watt)
▪ PROD= actual return power (Watt)

Should you have all values directly available in MQTT, you may load them using the same technic as POWER;ENERGY, but with 6 values instead of 2 (and if my memory good serves me, only the 5 first are mandatory).

If not (for example, you have only POWER; ENERGY and a flag indicating the current tariff), you may consider using a small script to conditionally load USAGE1 or USAGE2 depending on tariff flag. I can provide you some example if needed.

[Hencor]

You're right. As of now, multiplier is only applied when value to set is numeric (loaded in nValue). Here, we're loading a string with 2 numerical value. This clearly is a bug on my side It has been fixed in 1.0.10. Thanks for your feedback!

Thanks for the fast modification, but sorry the multiplier still seems not be taken into account.

Code: Select all

 2023-03-23 13:12:58.448 (MQTT - Haus) Updating device from 0:'220;8279.7436' to have values 0:'220;8279.7456'.
2023-03-23 13:12:58.507 MQTT: onMQTTConnected found smartmeter/0/1-0:16_7_0__255/value, Device 'MQTT - Haus' 

For the P1 smartmeter I just set USAGE1 und USAGE2. There seems to be a initialization issue. When the P1 meter is created its showing "invalid". I can fix when I just send 0;0;0;0;0 with a JSON URL command. Then the counter works as expected (beside the multiplier issue)
There are also these warning at device initialization:

Code: Select all

 2023-03-23 13:08:12.501 Status: MQTT: Initialized version 1.0.10, author 'Flying Domotic'
2023-03-23 13:08:12.547 Status: Warning: Expecting svalue with at least 5 elements separated by semicolon, 0 elements received (""), notification not sent (Hardware: 3 - MQTT, ID: Scheune, Unit: 2, Type: FA - P1 Smart Meter, SubType: 1 - Energy)
2023-03-23 13:08:12.562 Status: Warning: Expecting svalue with at least 5 elements separated by semicolon, 0 elements received (""), notification not sent (Hardware: 3 - MQTT, ID: Scheune, Unit: 2, Type: FA - P1 Smart Meter, SubType: 1 - Energy) 

[FlyingDomotic]

Thanks for the fast modification, but sorry the multiplier still seems not be taken into account.

Oups ... My fault, I just tried with an integer value, and not a float one. Last version (1.0.11) now test for both...

[Hencor]

Thanks! It is working now. Very nice.

Just the problem with the changing hardware type to fix.

[FlyingDomotic]

Just the problem with the changing hardware type to fix.

What do you mean?

[Hencor]

This one: viewtopic.php?t=40131

I faced some issues when I played around with the MqttMapper plugin.

Domoticz version 2023.1 stable - Python Version: 3.7.3

Issue:
The typ of my other python plugins changed to MqttMapper and the settings aren't available. (DeCONZ - idx 12 and Home Connect - idx 11)
The settings from the Mqtt Mapper hardware itself are also not available. I can only change settings when I remove and add the hardware again.
Also activation or deactivation is not possible anymore.


Hardware.PNG

Attempts to solve the problem:
1. Removed the Mqtt Mapper Hardware - no success
2. Did a full database restore - no success
3. 1. + removed the Mqtt Mapper plugin folder - success. The deconz plugin name and settings are available as expected
But after clone the Mqtt Mapper plugin from github the problem reappears.
I also tried to clone another plugin (PP-Manager) for testing but there the problem did not appear

My problem was also discussed in the plugin thread: viewtopic.php?t=39279&start=20

[FlyingDomotic]

I'll answer to original message, I don't saw it ...

[WBeulink]

Domoticz: Version: latest stable
RPi4, Bullseye

Plugin version from today: 29-06-2023

Very interested in plugin but can't get it working.
If I add the plugin in hardware in Domoticz and come back to check settings chosen hardware field (Type: ) is empty!
Part of config is lost.

See attachment.

Willem