Page 1 of 1
MQTT domoticz/out "flooding"
Posted: Monday 07 August 2023 23:38
by AlessioLorusso
Hello
My setup is a
Version: 2023.2
Build Hash: 19efd039c
Compile Date: 2023-07-21 17:23:44
"Dockerized" version in a ubuntu docker-compose configuration.
I noticed that any device change status is ecohed in domoticz/out topic messages.
For example the meteo ("Open weather map" hardware) every now and then flood the topic with dozens of messages for each meteo value: forecasted temperature for each xx hours, rain ...etc. This is really "flooding" the topic.
Not an issue unless I stated to notice that the poorest hardware connected to domoticz via MQTT is having bad time to cope with this unexpeced dump of data, causing, now and then disconnection, timeouts or simple JSON parse error (buffer too small?).
This seems to affect the small sonoff basic mostly.
Is there a way to limit the amout of MQTT data that domoticz "dumps" on the out topic?
I saw in other trhead the "prevent loop option" that I already have set to "true": I'm not sure this is the case.
Thanks for any suggestion!
Re: MQTT domoticz/out "flooding"
Posted: Monday 07 August 2023 23:42
by heggink
Easiest is to remove domoticz/out. That disabled output completely.
Re: MQTT domoticz/out "flooding"
Posted: Tuesday 08 August 2023 0:06
by AlessioLorusso
Yes ...it will solve the issue, but all the devices won't receive any command at all from domoticz....
I'm afraid this don't work.
Thanks
Re: MQTT domoticz/out "flooding"
Posted: Tuesday 08 August 2023 9:24
by waltervl
If Prevent Loop is set to True (default) then there is no other option. But then the output in domticz/out is just as designed...
These are small messages so it will not flood your system.
You can set MQTT Publish to type "index". then it will publish the messages per domoticz device. Makes it more readable and configurable for the devices that use it.
Re: MQTT domoticz/out "flooding"
Posted: Tuesday 08 August 2023 12:36
by AlessioLorusso
Yes I was afraid there is not a "quick fix" for this.
Definitely too much MQTT traffic caused the disconnection / reconnection, with varius dealys in between of not responsiveness, mostly for all the Sonoff BASIC based switchs.
With Espurna I got an "parsing error" and after some minutes the connections and parsing restart.
With Tasmota no errors on log, but only a MQTT reconnection in the log.
I applied this "fix":
- cahnged mqtt topic out for domoticz (like domoticz/outoriginal)
- create a simple flow that read messages from the new topic (domoticz/outoriginal) and write everything to the old one (domoticz/out) keeping only the "essentials" fields on the json output (idx, svalue, nvalue...).
This quickily solved the reconnection on Tasmota (I had one every 2 minutes in some cases...).
Let's see in the coming days!
Thanks1
Re: MQTT domoticz/out "flooding"
Posted: Tuesday 08 August 2023 14:07
by waltervl
If you have really flooding of data also look at the update frequency of some of your domoticz hardware gateways or scripts. It is mostly not needed to update weather or energy measuring devices every couple of seconds. Perhaps every 30 sec or every minute is enough.
Re: MQTT domoticz/out "flooding"
Posted: Sunday 13 August 2023 13:47
by claude2666
Hi,
I created some rooms and added them to a floor plan. In the room plan, I added the devices I want to have the updates published.
Then I set used published topic Floor/Room. In that case, I have control over which sensors/devices are published.
Re: MQTT domoticz/out "flooding"
Posted: Sunday 13 August 2023 16:03
by AlessioLorusso
claude2666 wrote: ↑Sunday 13 August 2023 13:47
Hi,
I created some rooms and added them to a floor plan. In the room plan, I added the devices I want to have the updates published.
Then I set used published topic Floor/Room. In that case, I have control over which sensors/devices are published.
Hi
Seems a good solution.
My point actually was also about the lengty messages: Domoticz sent not only the status (idx, nvalue, svalue1) but also a lot of other information, like description, that could cause harm to the "poor" sonoff basic (which I have a lot). A couple of messages, or only one, can force the device to detach from MQTT and became irresponsive for a while.
Yes I could avoid to put descriptions on the property pages of devices, but I'd rather use the quickiest solution to use a MQTT Nodered flow to filter everything out.
- Flow diagram
- Cattura.PNG (13.11 KiB) Viewed 3376 times
Function script:
Code: Select all
if ( msg.payload && "idx" in msg.payload)
{
// if ( msg.payload.idx==46)
{
msgnew={};
msgnew.payload={ "idx":msg.payload.idx}
if ( msg.payload && "nvalue" in msg.payload) msgnew.payload.nvalue=msg.payload.nvalue
if ( msg.payload && "svalue" in msg.payload) msgnew.payload.svalue=msg.payload.svalue
if ( msg.payload && "svalue1" in msg.payload) msgnew.payload.svalue1=msg.payload.svalue1
if ( msg.payload && "svalue2" in msg.payload) msgnew.payload.svalue2=msg.payload.svalue2
return msgnew
}
// else
// return null;
}
else
return null;
A little "cheap and dirty", but so far (one week) is working fine.
Alex
Re: MQTT domoticz/out "flooding"
Posted: Sunday 30 June 2024 12:29
by roblom
I don't know if it was already in when this topic started but for people reding this topic again, the MQTT wiki says:
From 2024.1 it is possible to select the devices that need to send data to the topic out (default domoticz/out). The rest of the devices will not send MQTT data.
Default the list is empty and all devices will send data.
To filter the devices go to menu Setup - Hardware and click on the Setup button of the MQTT Client gateway. A new dialog page will be shown were you can select the devices.
Re: MQTT domoticz/out "flooding"
Posted: Monday 16 September 2024 8:17
by Eucli090
roblom wrote: ↑Sunday 30 June 2024 12:29
I don't know if it was already in when this topic started but for people reding this topic again, the MQTT wiki says:
From 2024.1 it is possible to select the devices that need to send data to the topic out (default domoticz/out). The rest of the devices will not send MQTT data.
Default the list is empty and all devices will send data.
To filter the devices go to menu Setup - Hardware and click on the Setup button of the MQTT Client gateway. A new dialog page will be shown were you can select the devices.
Did you find a solution, the filter option does not work for me, I don't know why.
Re: MQTT domoticz/out "flooding"
Posted: Monday 16 September 2024 9:20
by waltervl
Eucli090 wrote: ↑Monday 16 September 2024 8:17
Did you find a solution, the filter option does not work for me, I don't know why.
You have a different issue. You do not have anything in domoticz/out....