MQTT domoticz/out "flooding" Topic is solved

[AlessioLorusso]

Hello

My setup is a
Version: 2023.2
Build Hash: 19efd039c
Compile Date: 2023-07-21 17:23:44
"Dockerized" version in a ubuntu docker-compose configuration.

I noticed that any device change status is ecohed in domoticz/out topic messages.
For example the meteo ("Open weather map" hardware) every now and then flood the topic with dozens of messages for each meteo value: forecasted temperature for each xx hours, rain ...etc. This is really "flooding" the topic.

Not an issue unless I stated to notice that the poorest hardware connected to domoticz via MQTT is having bad time to cope with this unexpeced dump of data, causing, now and then disconnection, timeouts or simple JSON parse error (buffer too small?).
This seems to affect the small sonoff basic mostly.

Is there a way to limit the amout of MQTT data that domoticz "dumps" on the out topic?

I saw in other trhead the "prevent loop option" that I already have set to "true": I'm not sure this is the case.

Thanks for any suggestion!

[heggink]

Easiest is to remove domoticz/out. That disabled output completely.

[AlessioLorusso]

Yes ...it will solve the issue, but all the devices won't receive any command at all from domoticz....
I'm afraid this don't work.
Thanks

[waltervl]

If Prevent Loop is set to True (default) then there is no other option. But then the output in domticz/out is just as designed...
These are small messages so it will not flood your system.


You can set MQTT Publish to type "index". then it will publish the messages per domoticz device. Makes it more readable and configurable for the devices that use it.

[AlessioLorusso]

Yes I was afraid there is not a "quick fix" for this.

Definitely too much MQTT traffic caused the disconnection / reconnection, with varius dealys in between of not responsiveness, mostly for all the Sonoff BASIC based switchs.
With Espurna I got an "parsing error" and after some minutes the connections and parsing restart.
With Tasmota no errors on log, but only a MQTT reconnection in the log.

I applied this "fix":
- cahnged mqtt topic out for domoticz (like domoticz/outoriginal)
- create a simple flow that read messages from the new topic (domoticz/outoriginal) and write everything to the old one (domoticz/out) keeping only the "essentials" fields on the json output (idx, svalue, nvalue...).

This quickily solved the reconnection on Tasmota (I had one every 2 minutes in some cases...).

Let's see in the coming days!

Thanks1

[waltervl]

If you have really flooding of data also look at the update frequency of some of your domoticz hardware gateways or scripts. It is mostly not needed to update weather or energy measuring devices every couple of seconds. Perhaps every 30 sec or every minute is enough.

[claude2666]

Hi,

I created some rooms and added them to a floor plan. In the room plan, I added the devices I want to have the updates published.
Then I set used published topic Floor/Room. In that case, I have control over which sensors/devices are published.

[AlessioLorusso]

Hi,

I created some rooms and added them to a floor plan. In the room plan, I added the devices I want to have the updates published.
Then I set used published topic Floor/Room. In that case, I have control over which sensors/devices are published.

Hi

Seems a good solution.
My point actually was also about the lengty messages: Domoticz sent not only the status (idx, nvalue, svalue1) but also a lot of other information, like description, that could cause harm to the "poor" sonoff basic (which I have a lot). A couple of messages, or only one, can force the device to detach from MQTT and became irresponsive for a while.

Yes I could avoid to put descriptions on the property pages of devices, but I'd rather use the quickiest solution to use a MQTT Nodered flow to filter everything out.

Flow diagram

Cattura.PNG (13.11 KiB) Viewed 3039 times

Function script:

Code: Select all

if ( msg.payload && "idx" in msg.payload)
{
//    if ( msg.payload.idx==46)  
    {
        msgnew={};
        msgnew.payload={ "idx":msg.payload.idx}
        if ( msg.payload && "nvalue" in msg.payload) msgnew.payload.nvalue=msg.payload.nvalue
        if ( msg.payload && "svalue" in msg.payload) msgnew.payload.svalue=msg.payload.svalue
        if ( msg.payload && "svalue1" in msg.payload) msgnew.payload.svalue1=msg.payload.svalue1
        if ( msg.payload && "svalue2" in msg.payload) msgnew.payload.svalue2=msg.payload.svalue2
        
        return msgnew
    }
//    else
//        return null;
}
    else
        return null;

A little "cheap and dirty", but so far (one week) is working fine.

Alex

[roblom]

I don't know if it was already in when this topic started but for people reding this topic again, the MQTT wiki says:

From 2024.1 it is possible to select the devices that need to send data to the topic out (default domoticz/out). The rest of the devices will not send MQTT data.

Default the list is empty and all devices will send data.

To filter the devices go to menu Setup - Hardware and click on the Setup button of the MQTT Client gateway. A new dialog page will be shown were you can select the devices.

[Eucli090]

I don't know if it was already in when this topic started but for people reding this topic again, the MQTT wiki says:

From 2024.1 it is possible to select the devices that need to send data to the topic out (default domoticz/out). The rest of the devices will not send MQTT data.

Default the list is empty and all devices will send data.

To filter the devices go to menu Setup - Hardware and click on the Setup button of the MQTT Client gateway. A new dialog page will be shown were you can select the devices.

Did you find a solution, the filter option does not work for me, I don't know why.

[waltervl]

Did you find a solution, the filter option does not work for me, I don't know why.

You have a different issue. You do not have anything in domoticz/out....