Page 1 of 2
MQTTAD seems to loose track on MQTT broker ???
Posted: Saturday 18 May 2024 7:29
by eddieb
Hi,
I have this strange problem.
I used to had Z2M plugin and never seen this but with MQTT AD I notice following :
When zigbee switches are added, all seems to work, I can switch the device from domoticz.
After a while, some of the switches stop responding from domoticz. I can NOT switch them anymore
I have to go to the Zigbee2MGTT screen (on port 8888 here) to switch the device and see that it is working from there...
walking over the device settings makes domoticz "remembering" that device again and all seems working again (for a couple of days)
Now I notice the same problem with some Tasmota devices.
They work OK but sending a switch command just stops working after a while. (Though the #idx is correctly and the tasmota device is working correctly from tasmoadmin and even in MQTTexplorer it looks ok.
It seems in domoticz MQTTAD looses track on what is happening in the MQTT broker ...
Anyone seen this ?
Any suggestions in how to debug this in domoticz ?
Re: MQTTAD seems to loose track on MQTT broker ???
Posted: Saturday 18 May 2024 9:27
by heggink
I would start by chasing the problem origin. Since you can control switches from the zb2mqtt interface, the problem either sits in your mqtt broker or at the client side interfacing with it. Use mqtt explorer to see what messages are being sent from/to the broker, try to manually insert a message using mosquitto_pub, things like that.
Re: MQTTAD seems to loose track on MQTT broker ???
Posted: Tuesday 18 June 2024 8:45
by eddieb
Hi,
I am really loosing it now.
still after a restart of domoticz about half of my zigbee devices loose connection.
In Zigbee2MQTT (port 8888) everything works fine, all devices are updated and switch commands work
In Domoticz some devices work, some do not.
can someone please point me in the direction of how to debug this ?
this is one of the switch devices
- Screenshot 2024-06-18 at 08.34.28.png (47.53 KiB) Viewed 2802 times
this is the error in domoticz
- Screenshot 2024-06-18 at 08.33.53.png (23.24 KiB) Viewed 2802 times
these are all the devices on that device
- Screenshot 2024-06-18 at 08.43.36.png (282.68 KiB) Viewed 2802 times
I have several other devices not working right now, really annoying
the connection between Z2M and MQTTAD is not working properly here
Re: MQTTAD seems to loose track on MQTT broker ???
Posted: Tuesday 18 June 2024 8:51
by eddieb
for now, I am unable to trigger any Zigbee switch from domoticz. Other sensors from the same device seem to work
my Ikea vindstyrka is not reporting at all right now, untal latest update/reboot that worked fine for weeks
Re: MQTTAD seems to loose track on MQTT broker ???
Posted: Tuesday 18 June 2024 9:54
by Kedi
do you have in Zigbee2mqtt the retain enabled?
Re: MQTTAD seems to loose track on MQTT broker ???
Posted: Tuesday 18 June 2024 10:14
by eddieb
Where do I find that option in the :8888 interface ?
Re: MQTTAD seems to loose track on MQTT broker ???
Posted: Tuesday 18 June 2024 10:23
by Kedi
No idea where it it is in the GUI, I have to search like you have to.
Here it is in the manual explained:
https://www.zigbee2mqtt.io/guide/config ... ce-options
You can set it per device under Device -> Settings
The one I mentioned above is for all devices set in configuration.yaml file, like I did.
Minimum is to set all battery devices to true, this way mosquitto will keep the last state in case of power down or reboot.
Re: MQTTAD seems to loose track on MQTT broker ???
Posted: Tuesday 18 June 2024 10:25
by eddieb
All devices do not have retain: true
but, like I said, in Z2M all works fine, all devices and all sensors are updated and working.
Re: MQTTAD seems to loose track on MQTT broker ???
Posted: Tuesday 18 June 2024 11:31
by eddieb
I found this error in mosquitto logging ... what does that mean ?
Code: Select all
1718701578: New client connected from 127.0.0.1 as Domoticz-MQTT-AutoDiscover740c5e22-22a4-473b-84d4-f90e87bd4c1334 (c1, k120).
1718701578: Outgoing messages are being dropped for client Domoticz-MQTT-AutoDiscover740c5e22-22a4-473b-84d4-f90e87bd4c1334.
Re: MQTTAD seems to loose track on MQTT broker ???
Posted: Tuesday 18 June 2024 11:34
by Kedi
So set (all) devices to retain: true and see if that solves your problem.
Retain has nothing to do with the fact that devices are working in Z2M.
It is worth a shot, it does not hurt anything.
Re: MQTTAD seems to loose track on MQTT broker ???
Posted: Tuesday 18 June 2024 11:35
by Kedi
"Outgoing messages are being dropped for client" ---> Google it.
Your mosquitto might be mal configured.
Re: MQTTAD seems to loose track on MQTT broker ???
Posted: Tuesday 18 June 2024 11:48
by eddieb
Ok, that might have been the issue ...
Code: Select all
cat /etc/mosquitto/conf.d/local.conf
max_queued_messages 0
restarted mosquitto and it seems to work fine now, tnx for the sugegstion
I will monitor this for the next couple of days closely
Re: MQTTAD seems to loose track on MQTT broker ???
Posted: Tuesday 18 June 2024 18:49
by Kedi
"max_queued_messages 0" is not a good idea. The mosquitto default is 1000 and should be enough for Z2M and Domoticz.
You have to find out why there are so much messages queued.
If you add this for some time to the local.conf you might find out what is sending so much messages or is not responding.
Code: Select all
log_dest file /var/log/mosquitto.log
log_type all
Re: MQTTAD seems to loose track on MQTT broker ???
Posted: Wednesday 19 June 2024 6:36
by eddieb
Morning,
I added the log_type all to the local.conf and now I get really a lot of logging.
So much I really have no clue where to look at.
Besides zigbee devices there are also a bunch of tasmota devices using the broker.
for now I did set max_queue_messages 5000 and that seems to work well
Re: MQTTAD seems to loose track on MQTT broker ???
Posted: Wednesday 19 June 2024 6:52
by gizmocuz
instead of log_type all you can add these two lines
log_type error
log_type warning
So only errors and warnings are logged
Your broker and your Domoticz system are both in your local network? And the speed between them is fast enough?
(They are connected with a cable instead of Wifi)
Also have a look at the CPU usage of both Domoticz and MQTT broker's machines
Do you connect via TLS (secure)? Does it make a difference when you make a plain connection (cpu wise)
Re: MQTTAD seems to loose track on MQTT broker ???
Posted: Wednesday 19 June 2024 7:03
by eddieb
Broker and domoticz are both on the same RPI4
RPI connected by eth to local 1GB lan
no TLS on local machines
cpu load is normally not higher than 2, mostly under 1
Re: MQTTAD seems to loose track on MQTT broker ???
Posted: Wednesday 19 June 2024 7:20
by eddieb
Code: Select all
2024-06-19 07:08:43 New client connected from 192.168.0.146 as DVES_452C98 (c1, k30, u'DVES_USER').
2024-06-19 07:08:43 New client connected from 192.168.0.153 as DVES_33112A (c1, k30, u'DVES_USER').
2024-06-19 07:08:44 New client connected from 192.168.0.147 as DVES_500C91 (c1, k30, u'DVES_USER').
2024-06-19 07:08:44 New client connected from 192.168.0.150 as DVES_4FCE23 (c1, k30, u'DVES_USER').
2024-06-19 07:08:44 New client connected from 192.168.0.149 as DVES_4BF3A0 (c1, k30, u'DVES_USER').
2024-06-19 07:08:44 New client connected from 192.168.0.151 as DVES_554241 (c1, k30, u'DVES_USER').
2024-06-19 07:08:44 New client connected from 192.168.0.145 as DVES_C80778 (c1, k30, u'DVES_USER').
2024-06-19 07:08:44 New client connected from 192.168.0.152 as DVES_DAAD37 (c1, k30, u'DVES_USER').
2024-06-19 07:08:44 New client connected from 192.168.0.148 as DVES_E7AF5D (c1, k30, u'DVES_USER').
2024-06-19 07:08:44 New client connected from 127.0.0.1 as mqttjs_06a0b7a1 (c1, k60).
2024-06-19 07:08:47 New client connected from 192.168.0.231 as Domoticz-MQTT-AutoDiscover04acb960-e88c-422e-9e9f-e140e27ae48b3 (c1, k120).
2024-06-19 07:08:47 New client connected from 127.0.0.1 as Domoticz-MQTT-AutoDiscover740c5e22-22a4-473b-84d4-f90e87bd4c1334 (c1, k120).
2024-06-19 07:08:48 New client connected from 192.168.0.247 as conditioner (c1, k15).
a total of 13 connections to the broker.
9 DVES users (Tasmota devices)
1 mqttjs (NodeRed ?)
2 MQTT-AutoDiscovery
1 conditioner (Airco unit)
Re: MQTTAD seems to loose track on MQTT broker ???
Posted: Wednesday 19 June 2024 7:45
by Kedi
I have a busy RPi4 but have no problems on port 1883 with the default setting on mosquitto.
It might be that some device is bursting.
Install iptraf-ng and look for traffic on port 1883
Re: MQTTAD seems to loose track on MQTT broker ???
Posted: Wednesday 19 June 2024 8:50
by eddieb
these are the top communicators ...
Code: Select all
TCP Connections (Source Host:Port) ───────────────────────────────────────────────────────────────
│┌localhost:1883 > 41420 12365610 --A- lo
│ localhost:37828 > 29423 1652544 -PA- lo
│ localhost:57372 > 41134 13173332 --A- lo
│ localhost:1883 > 29236 2363740 -PA- lo
Code: Select all
root@domoticz:~# netstat -aptn | grep 37828
tcp 0 0 127.0.0.1:37828 127.0.0.1:1883 ESTABLISHED 401/node-red
tcp 0 0 127.0.0.1:1883 127.0.0.1:37828 ESTABLISHED 26904/mosquitto
root@domoticz:~# netstat -aptn | grep 57372
tcp6 0 4 ::1:1883 ::1:57372 ESTABLISHED 26904/mosquitto
tcp6 28 0 ::1:57372 ::1:1883 ESTABLISHED 5848/domoticz
all other connections to 1883 are less than 10% of these
Re: MQTTAD seems to loose track on MQTT broker ???
Posted: Wednesday 19 June 2024 9:14
by Kedi
That's a lot.
Do you have in Domoticz Hardware the MQTT Gateway "prevent loop" setting set to true?
You can investigate the Node-red flows with debug if a flow is sending too much.
With MQTT Explorer you could investigate the $SYS topic.