Python Plugin: MqttMapper

[peterchef]

Sadly I am a neanderthal that runs windows everywhere.

The mqtt broker machine is running windows 11 pro 23h2 with eclipse mosquitto 2.0.19.
The client machine is Windows 10 22H2 running -- domoticz 2024.7 build 16268 -- mqttmapper 0.48 -- python 3.11.9 32bit.

I think the problem is likely to be related to the plugin framework build for windows.

[FlyingDomotic]

IMHO, Windows may eventually be good for GUI (even if you now have the same/better GUI with Linux), but I should recognize that some softs only exists in Windows version. However, when speaking about servers and IOT, this is really not a good idea at all.

Buying a RPi 4 will cost you 50/60€, and make a better job for Domoticz. System update is now automatic (and patches well tested Don't known when power consumption difference will pay the RPi, but it may be interesting to compute it.

[peterchef]

After 45yrs with computers - unlikely to change horses. Guess I will find another solution.

[FlyingDomotic]

I have a bit more with computers (since 1976, when PC didn't exist yet), leaning Linux at 50+ was not a big deal ...

In addition if you connect a screen to your RPi, you won't see a big difference with Windows...

[rron]

I ḿ still struggling with on issue I cannot find about type, subtype

Code: Select all

    "SmartEVSE evtotalenergy charged": {
        "topic": "SmartEVSE-5572/EVTotalEnergyCharged",
        "type": "243", "subtype": "29", "switchtype": "0",
        "mapping": {"item": "", "multiplier": 0.001}

When I am using type 248 and subtype 1 the outcome is in Watts
When I am using type 243 and subtype 29 the outcome is in instant+counter in kWh
I only need kWh, what kind of type and subtype do I need?

[waltervl]

That would be Counter type 113 and select the right switchtype
The device types are listed here: https://wiki.domoticz.com/Developing_a_ ... vice_Types

[rron]

I will try that and let you know.

[rron]

That would be Counter type 113 and select the right switchtype
The device types are listed here: https://wiki.domoticz.com/Developing_a_ ... vice_Types

Hello Walter,

Counter is OK now. Thanks.

[hjzwiers]

I can't seem to get this right ..........

I changed my MQTT file:

My own rework of the outside temperature:

{
"T_Outside": {
"topic": "ebusd/Broadcast/Outsidetemp",
"type": "80", "subtype": "5", "switchtype": "0",
"mapping": {"iten":{"item": ""}, "multiplier": 1}
},

"T_Living": {
"topic": "ebusd/ctlv2/Z1RoomTemp",
"type": "80", "subtype": "5", "switchtype": "0",
"mapping": {"item": "", "multiplier": 1}
},

"MQTTrunning": {
"topic": "ebusd/global/running",
"type": "244", "subtype": "73", "switchtype": "0",
"mapping": {"item": "", "default": "0"}
},
"MQTTsignal": {
"topic": "ebusd/global/signal",
"type": "244", "subtype": "73", "switchtype": "0",
"mapping": {"item": "", "default": "0"}
}
}

[waltervl]

topic "ebusd/Broadcast/Outsidetemp" has message
{"value":{"value": 6.562"}}

then the mapping should be
"mapping": {"item": "value/value"}

[hjzwiers]

Thanks that was it!!

I was a bit confused as MQTT made two new files, but I deleted the old ones.

[winner61]

Hello, sometimes I take a dive and try to accomplish getting growatt inverter data into Domoticz.
I have partially succes. With MQTTmapper I get values with a json file I found in this thread. They appear just for a second and then reset to '0'
I used this code from viewtopic.php?p=314130&hilit=growatt#p314130 by pwhooftman. Thanks for this example.

I attach a piece of the log file as an image because it shows more clearly.

p2.png (236.1 KiB) Viewed 2925 times

p3.png (76.79 KiB) Viewed 2925 times

Just for a second!

If anyone knows what's happening I would be very grateful.

[FlyingDomotic]

It seems that data returned by your device is slightly (very?) different of those given in other topic you mentioned.

To get precise idea of topic content, cut received message from log (from line onMqttPublish foud xxx, Device 'yyy', message 'zzz', extract zzz part, without surrounding quotes), and paste it in any json formatter (for example https://jsonformatter.curiousconcept.com, or even better NotePad++ with json plugin).

You'll clearly find full path to use, in your case, probably something like "value/voltage_l3" (instead of "value/pvgridvoltage3") or "value/act_power_l2" (instead of "value/pv2watt"). Obviously, it seems that temperature is not returned by your device.

Note that it could be easier to debug if you cut and past text (including log) in your message instead of an image.

Note also that you have negative values in power, meaning either that you produce energy and re-inject it into grid, or you mounted tore up side down.

[winner61]

Thank you FlyingDomotic. I'm not really into programming so this is not really transparent for me. I hope to get some good advice here, like from you. The reason that I used an image is because of the instant colors distinguishing between the error- and normal log. Tried to do what you said and removed the quotes bur notepadd++ won't json format it. Also not transparent what to leave out or keep in. Mayby a little pointer . Thank you again.

2025-02-02 12:15:31.099 GrowattMOD: onMQTTPublish found energy/growatt, Device 'GrowattMOD - Growatt PV2 voltage', message '{'device': 'QBL1CL305R', 'time': '2025-02-02T12:14:35', 'buffered': 'no', 'values': {'datalogserial': 'XGD6CCN0JV', 'pvserial': 'QBL1CL305R', 'pvstatus': 1, 'pvpowerin': 48842, 'pv1voltage': 3604, 'pv1current': 67, 'pv1watt': 24146, 'pv2voltage': 3528, 'pv2current': 70, 'pv2watt': 24696, 'pv3voltage': 0, 'pv3current': 0, 'pv3watt': 0, 'pv4voltage': 0, 'pv4current': 0, 'pv4watt': 0, 'pvpowerout': 24982, 'qac': 337, 'pac': 24011, 'pvfrequency': 4998, 'pvgridvoltage': 2393, 'pvgridcurrent': 34, 'pvgridpower': 8136, 'pvgridvoltage2': 2375, 'pvgridcurrent2': 34, 'pvgridpower2': 8075, 'pvgridvoltage3': 2325, 'pvgridcurrent3': 34, 'pvgridpower3': 7905, 'vacrs': 4125, 'vacst': 4084, 'vactr': 4075, 'ptousertotal': 0, 'ptogridtotal': 0, 'ptoloadtotal': 24982, 'totworktime': 61038737, 'eactoday': 66, 'pvenergytoday': 66, 'eactotal': 97914, 'pvenergytotal': 97914, 'epvtotal': 83108, 'epv1today': 29, 'epv1total': 40919, 'epv2today': 37, 'epv2total': 42189, 'epv3today': 0, 'epv3total': 0, 'etousertoday': 113, 'etousertotal': 37264, 'etogridtoday': 0, 'etogridtotal': 37664, 'eloadtoday': 184, 'eloadtotal': 82629, 'epv4today': 0, 'epv4total': 0, 'epvtoday': 66, 'deratingmode': 0, 'iso': -6, 'dcir': 53, 'dcis': 4, 'dcit': 132, 'gfci': 1, 'busvoltage': 6493, 'pvtemperature': 291, 'pvimptemperature': 428, 'boosttemperature': 283, 'temp4': 0, 'comboardtemperature': 303, 'pbusvoltage': 3272, 'nbusvoltage': 3219, 'ipf': 20000, 'realoppercent': 29, 'opfullwatt': 80000, 'standbyflag': 0, 'faultmaincode': 0, 'warnmaincode': 0, 'faultsubcode': 0, 'warnsubcode': 0, 'uwpresentfftvaxxxlue[channela]': 0, 'bafcistatus': 0, 'uwstrength[channela]': 0, 'uwselfcheckvalue[channela]': 0, 'invstartdelaytime': 60, 'bdconoffstate': 1, 'drycontactstate': 0, 'pself': 24982, 'esystoday': 70, 'edischrtoday': 12, 'edischrtotal': 39391, 'echrtoday': 7, 'echrtotal': 39470, 'eacchrtoday': 0, 'eacchrtotal': 18615, 'esystotal': 101274, 'eselftoday': 71, 'eselftotal': 63607, 'reserved3143': 0, 'priority': 0, 'epsfac': 0, 'epsvac1': 0, 'epsiac1': 0, 'epspac1': 0, 'epsvac2': 0, 'epsiac2': 0, 'epspac2': 0, 'epsvac3': 0, 'epsiac3': 0, 'epspac3': 0, 'epspac': 0, 'loadpercent': 0, 'pf': 10000, 'dcv': 0, 'newbdcflag': 0, 'bdcderatingmode': 0, 'sysstatemode': 257, 'faultcode': 0, 'warncode': 0, 'vbat': -28619, 'ibat': 63, 'soc': 13, 'vbus1': 6481, 'vbus2': 3189, 'ibb': 31, 'illc': 31, 'tempa': 233, 'tempb': 0, 'pdischr': 0, 'pchr': 23580, 'pchrxxxl': 23580, 'edischrtotalstor': 39391, 'echrtotalstor': 39470, 'bdc1flag': 3, 'vbus2low': 3292, 'bmsmaxvoltcellno': 108, 'bmsminvoltcellno': 38, 'bmsbatteryavgtemp': 0, 'bmsmaxcelltemp': 144, 'bmsbatteryavgtemp2': 122, 'bmsmaxcelltemp2': 1, 'bmsbatteryavgtemp3': 26, 'bmsmaxsoc': 0, 'bmsminsoc': 0, 'parallelbatterynum': 1, 'bmsderatereason': 0, 'bmsgaugefcc(ah)': 48, 'bmsgaugerm(ah)': 48, 'bmserror': 0, 'bmswarn': 0, 'bmsfault': 0, 'bmsfault2': 0, 'batisostatus': 0, 'battneedchargerequestflag': 0, 'bmsstatus': 1, 'bmserror2': 0, 'bmswarn2': 0, 'bmssoc': 13, 'bmsbatteryvolt': -28666, 'bmsbatterycurr': 610, 'bmsbatterytemp': 310, 'bmsmaxcurr': 2500, 'bmsmaxdischrcurr': 2500, 'bmscyclecnt': 0, 'bmssoh': 98, 'bmschargevoltlimit': -25776, 'bmsdischargevoltlimit': -32496, 'bmswarn3': 0, 'bmserror3': 0, 'bmssinglevoltmax': 3303, 'bmssinglevoltmin': 3276, 'batloadvolt': -28666}}'

[FlyingDomotic]

Thank you FlyingDomotic. I'm not really into programming so this is not really transparent for me. I hope to get some good advice here, like from you. The reason that I used an image is because of the instant colors distinguishing between the error- and normal log. Tried to do what you said and removed the quotes bur notepadd++ won't json format it. Also not transparent what to leave out or keep in. Mayby a little pointer . Thank you again.

You have to select everything starting after "message='" and last quote ', giving:

Code: Select all

{'device': 'QBL1CL305R', 'time': '2025-02-02T12:14:35', 'buffered': 'no', 'values': {'datalogserial': 'XGD6CCN0JV', 'pvserial': 'QBL1CL305R', 'pvstatus': 1, 'pvpowerin': 48842, 'pv1voltage': 3604, 'pv1current': 67, 'pv1watt': 24146, 'pv2voltage': 3528, 'pv2current': 70, 'pv2watt': 24696, 'pv3voltage': 0, 'pv3current': 0, 'pv3watt': 0, 'pv4voltage': 0, 'pv4current': 0, 'pv4watt': 0, 'pvpowerout': 24982, 'qac': 337, 'pac': 24011, 'pvfrequency': 4998, 'pvgridvoltage': 2393, 'pvgridcurrent': 34, 'pvgridpower': 8136, 'pvgridvoltage2': 2375, 'pvgridcurrent2': 34, 'pvgridpower2': 8075, 'pvgridvoltage3': 2325, 'pvgridcurrent3': 34, 'pvgridpower3': 7905, 'vacrs': 4125, 'vacst': 4084, 'vactr': 4075, 'ptousertotal': 0, 'ptogridtotal': 0, 'ptoloadtotal': 24982, 'totworktime': 61038737, 'eactoday': 66, 'pvenergytoday': 66, 'eactotal': 97914, 'pvenergytotal': 97914, 'epvtotal': 83108, 'epv1today': 29, 'epv1total': 40919, 'epv2today': 37, 'epv2total': 42189, 'epv3today': 0, 'epv3total': 0, 'etousertoday': 113, 'etousertotal': 37264, 'etogridtoday': 0, 'etogridtotal': 37664, 'eloadtoday': 184, 'eloadtotal': 82629, 'epv4today': 0, 'epv4total': 0, 'epvtoday': 66, 'deratingmode': 0, 'iso': -6, 'dcir': 53, 'dcis': 4, 'dcit': 132, 'gfci': 1, 'busvoltage': 6493, 'pvtemperature': 291, 'pvimptemperature': 428, 'boosttemperature': 283, 'temp4': 0, 'comboardtemperature': 303, 'pbusvoltage': 3272, 'nbusvoltage': 3219, 'ipf': 20000, 'realoppercent': 29, 'opfullwatt': 80000, 'standbyflag': 0, 'faultmaincode': 0, 'warnmaincode': 0, 'faultsubcode': 0, 'warnsubcode': 0, 'uwpresentfftvaxxxlue[channela]': 0, 'bafcistatus': 0, 'uwstrength[channela]': 0, 'uwselfcheckvalue[channela]': 0, 'invstartdelaytime': 60, 'bdconoffstate': 1, 'drycontactstate': 0, 'pself': 24982, 'esystoday': 70, 'edischrtoday': 12, 'edischrtotal': 39391, 'echrtoday': 7, 'echrtotal': 39470, 'eacchrtoday': 0, 'eacchrtotal': 18615, 'esystotal': 101274, 'eselftoday': 71, 'eselftotal': 63607, 'reserved3143': 0, 'priority': 0, 'epsfac': 0, 'epsvac1': 0, 'epsiac1': 0, 'epspac1': 0, 'epsvac2': 0, 'epsiac2': 0, 'epspac2': 0, 'epsvac3': 0, 'epsiac3': 0, 'epspac3': 0, 'epspac': 0, 'loadpercent': 0, 'pf': 10000, 'dcv': 0, 'newbdcflag': 0, 'bdcderatingmode': 0, 'sysstatemode': 257, 'faultcode': 0, 'warncode': 0, 'vbat': -28619, 'ibat': 63, 'soc': 13, 'vbus1': 6481, 'vbus2': 3189, 'ibb': 31, 'illc': 31, 'tempa': 233, 'tempb': 0, 'pdischr': 0, 'pchr': 23580, 'pchrxxxl': 23580, 'edischrtotalstor': 39391, 'echrtotalstor': 39470, 'bdc1flag': 3, 'vbus2low': 3292, 'bmsmaxvoltcellno': 108, 'bmsminvoltcellno': 38, 'bmsbatteryavgtemp': 0, 'bmsmaxcelltemp': 144, 'bmsbatteryavgtemp2': 122, 'bmsmaxcelltemp2': 1, 'bmsbatteryavgtemp3': 26, 'bmsmaxsoc': 0, 'bmsminsoc': 0, 'parallelbatterynum': 1, 'bmsderatereason': 0, 'bmsgaugefcc(ah)': 48, 'bmsgaugerm(ah)': 48, 'bmserror': 0, 'bmswarn': 0, 'bmsfault': 0, 'bmsfault2': 0, 'batisostatus': 0, 'battneedchargerequestflag': 0, 'bmsstatus': 1, 'bmserror2': 0, 'bmswarn2': 0, 'bmssoc': 13, 'bmsbatteryvolt': -28666, 'bmsbatterycurr': 610, 'bmsbatterytemp': 310, 'bmsmaxcurr': 2500, 'bmsmaxdischrcurr': 2500, 'bmscyclecnt': 0, 'bmssoh': 98, 'bmschargevoltlimit': -25776, 'bmsdischargevoltlimit': -32496, 'bmswarn3': 0, 'bmserror3': 0, 'bmssinglevoltmax': 3303, 'bmssinglevoltmin': 3276, 'batloadvolt': -28666}}

Ensuite, remplacer les quotes simples ' par des quotes doubles ".

Après ça, Notepad++ va fonctionner ...

Par contre, on est assez loin du format du post précédent ...

[winner61]

Hello FlyingDomotic. It worked I have a list now, but the names are like in the original json from pchooftman. I attach both.
So, I get values each minute but they don't stick and after a second all is empty again.
The temperature is there but on the temperature tab.

Greetings, Winfryd.

[winner61]

DomoLogGrowattMOD.txt

(88.21 KiB) Downloaded 5 times

List.json.txt

(3.75 KiB) Downloaded 8 times

The first is a longer peace of the logfile.

[FlyingDomotic]

I got it!
It seems you have 2 devices speaking on the same topic. First is QBL1CL305R:

{'device': 'QBL1CL305R', 'time': '2025-02-02T07:11:14', 'buffered': 'no', 'values': {'datalogserial': 'XGD6CCN0JV', 'pvserial': 'QBL1CL305R', 'pvstatus': 1, 'pvpowerin': 31263, 'pv1voltage': 3544, 'pv1current': 43, 'pv1watt': 15239, 'pv2voltage': 3561, 'pv2current': 45, 'pv2watt': 16024, 'pv3voltage': 0, 'pv3current': 0, 'pv3watt': 0, 'pv4voltage': 0, 'pv4current': 0, 'pv4watt': 0, 'pvpowerout': 13423, 'qac': 127, 'pac': 12186, 'pvfrequency': 5000, 'pvgridvoltage': 2366, 'pvgridcurrent': 18, 'pvgridpower': 4258, 'pvgridvoltage2': 2352, 'pvgridcurrent2': 18, 'pvgridpower2': 4233, 'pvgridvoltage3': 2354, 'pvgridcurrent3': 18, 'pvgridpower3': 4237, 'vacrs': 4093, 'vacst': 4074, 'vactr': 4082, 'ptousertotal': 0, 'ptogridtotal': 0, 'ptoloadtotal': 13423, 'totworktime': 61054268, 'eactoday': 96, 'pvenergytoday': 96, 'eactotal': 97944, 'pvenergytotal': 97944, 'epvtotal': 83207, 'epv1today': 78, 'epv1total': 40968, 'epv2today': 87, 'epv2total': 42239, 'epv3today': 0, 'epv3total': 0, 'etousertoday': 113, 'etousertotal': 37264, 'etogridtoday': 0, 'etogridtotal': 37664, 'eloadtoday': 216, 'eloadtotal': 82661, 'epv4today': 0, 'epv4total': 0, 'epvtoday': 165, 'deratingmode': 0, 'iso': -6, 'dcir': 29, 'dcis': 11, 'dcit': 240, 'gfci': 0, 'busvoltage': 6446, 'pvtemperature': 293, 'pvimptemperature': 441, 'boosttemperature': 283, 'temp4': 0, 'comboardtemperature': 319, 'pbusvoltage': 3245, 'nbusvoltage': 3201, 'ipf': 20000, 'realoppercent': 15, 'opfullwatt': 80000, 'standbyflag': 0, 'faultmaincode': 0, 'warnmaincode': 0, 'faultsubcode': 0, 'warnsubcode': 0, 'uwpresentfftvaxxxlue[channela]': 0, 'bafcistatus': 0, 'uwstrength[channela]': 0, 'uwselfcheckvalue[channela]': 0, 'invstartdelaytime': 60, 'bdconoffstate': 1, 'drycontactstate': 0, 'pself': 13423, 'esystoday': 102, 'edischrtoday': 12, 'edischrtotal': 39391, 'echrtoday': 74, 'echrtotal': 39537, 'eacchrtoday': 0, 'eacchrtotal': 18615, 'esystotal': 101306, 'eselftoday': 103, 'eselftotal': 63639, 'reserved3143': 0, 'priority': 0, 'epsfac': 0, 'epsvac1': 0, 'epsiac1': 0, 'epspac1': 0, 'epsvac2': 0, 'epsiac2': 0, 'epspac2': 0, 'epsvac3': 0, 'epsiac3': 0, 'epspac3': 0, 'epspac': 0, 'loadpercent': 0, 'pf': 10000, 'dcv': 0, 'newbdcflag': 0, 'bdcderatingmode': 0, 'sysstatemode': 257, 'faultcode': 0, 'warncode': 0, 'vbat': -28082, 'ibat': 47, 'soc': 52, 'vbus1': 6437, 'vbus2': 3168, 'ibb': 23, 'illc': 23, 'tempa': 270, 'tempb': 0, 'pdischr': 0, 'pchr': 17980, 'pchrxxxl': 17980, 'edischrtotalstor': 39391, 'echrtotalstor': 39537, 'bdc1flag': 3, 'vbus2low': 3269, 'bmsmaxvoltcellno': 108, 'bmsminvoltcellno': 16, 'bmsbatteryavgtemp': 0, 'bmsmaxcelltemp': 171, 'bmsbatteryavgtemp2': 148, 'bmsmaxcelltemp2': 2, 'bmsbatteryavgtemp3': 28, 'bmsmaxsoc': 0, 'bmsminsoc': 0, 'parallelbatterynum': 1, 'bmsderatereason': 0, 'bmsgaugefcc(ah)': 48, 'bmsgaugerm(ah)': 48, 'bmserror': 0, 'bmswarn': 0, 'bmsfault': 0, 'bmsfault2': 0, 'batisostatus': 0, 'battneedchargerequestflag': 0, 'bmsstatus': 1, 'bmserror2': 0, 'bmswarn2': 0, 'bmssoc': 52, 'bmsbatteryvolt': -28116, 'bmsbatterycurr': 470, 'bmsbatterytemp': 334, 'bmsmaxcurr': 2500, 'bmsmaxdischrcurr': 2500, 'bmscyclecnt': 0, 'bmssoh': 98, 'bmschargevoltlimit': -25776, 'bmsdischargevoltlimit': -32496, 'bmswarn3': 0, 'bmserror3': 0, 'bmssinglevoltmax': 3344, 'bmssinglevoltmin': 3336, 'batloadvolt': -28136}}

... while second is XGD6CCN0JV:

{'device': 'XGD6CCN0JV', 'time': '2025-02-02T07:11:16', 'buffered': 'no', 'values': {'datalogserial': 'XGD6CCN0JV', 'pvserial': 'QBL1CL305R', 'voltage_l1': 2355, 'voltage_l2': 2347, 'voltage_l3': 2353, 'Current_l1': 15, 'Current_l2': 12, 'Current_l3': 26, 'act_power_l1': -2849, 'act_power_l2': -1472, 'act_power_l3': 4068, 'app_power_l1': 3599, 'app_power_l2': 3029, 'app_power_l3': 6254, 'react_power_l1': -2010, 'react_power_l2': -2148, 'react_power_l3': 427, 'powerfactor_l1': -792, 'powerfactor_l2': -527, 'powerfactor_l3': 649, 'pos_rev_act_power': -253, 'pos_act_power': -253, 'rev_act_power': -253, 'app_power': 12787, 'react_power': -3678, 'powerfactor': -14, 'frequency': 499, 'L1-2_voltage': 4072, 'L2-3_voltage': 4070, 'L3-1_voltage': 4078, 'pos_act_energy': 78553, 'rev_act_energy': 105727}}

When receiving the first one, data is properly loaded as items are found, but when second is received, they're set to zero as not found.

As of now, MqttMapper was designed to have dedicated topics, so has difficulties to with variable content.

I'll made a change in order to be able to select message only if it contains a field with specific value (here, selection could by done on "device" = "QBL1CL305R".

Stay tuned!

[FlyingDomotic]

Last version includes now 2 changes :
1) if MqttMapper can't find item data in message, it won't be reset anymore.
2) you can now ignore messages that don't have a specific value in a given item. In our case, you may add

{"select": {"item": "device", "value": "QBL1CL305R"}}

You can add it into your configuration file, on each line related to device QBL1CL305R.

[winner61]

Hello FlyingDomotic, that sounds very good and did a git pull.
Yes there are 2 devices, the inverter and the datalogger. The QB.. is the inverter and there is the data to be found.

I tried several things with {"select": {"item": "device", "value": "QBL1CL305R"}} but can't find the proper place for it.

I presume I have to place it in the json file in each device I want to see. Could you guide me where to out it, for example in:

"PV2 watt": {
"topic": "energy/growatt",
"key": "energy/growatt/pv2watt",
"type": "248", "subtype": "1",
"mapping": {"item": "values/pv2watt", "multiplier": 0.1}

Thank you so much.