Can the MQTT Client Gateway post "retained" messages?

Others (MiLight, Hue, Toon etc...)

Moderator: leecollings

Bikey
Posts: 331
Joined: Sunday 22 February 2015 12:19
Target OS: Linux
Domoticz version: 2020.x
Location: Netherlands
Contact:

Can the MQTT Client Gateway post "retained" messages?

Post by Bikey »

Hi,

I'm using the MQTT Client gateway to have Domoticz publish the status of sensors and switches on MQTT so that other software can use this.

This works fine, except that if the other tool restarts, it is not until and update is send from Domoticz about a sensor before it is informed of the last state. This would be solved if Domoticz would publish it's messages on MQTT using the "retain" flag (see explanation)
However, I don't see how this can be configured in the MQTT Client gateway.

Does anybody know an elegant way to do this?
User avatar
waltervl
Posts: 5711
Joined: Monday 28 January 2019 18:48
Target OS: Linux
Domoticz version: 2024.7
Location: NL
Contact:

Re: Can the MQTT Client Gateway post "retained" messages?

Post by waltervl »

From what I know about retained messages this is not what they are meant for. And also you will have to find a way to remove them again.

Better is to have the other tool request the device data on startup with MQTT command "getdeviceinfo".
Domoticz running on Udoo X86 (on Ubuntu)
Devices/plugins: ZigbeeforDomoticz (with Xiaomi, Ikea, Tuya devices), Nefit Easy, Midea Airco, Omnik Solar, Goodwe Solar
User avatar
waaren
Posts: 6028
Joined: Tuesday 03 January 2017 14:18
Target OS: Linux
Domoticz version: Beta
Location: Netherlands
Contact:

Re: Can the MQTT Client Gateway post "retained" messages?

Post by waaren »

waltervl wrote: Monday 29 March 2021 15:51 From what I know about retained messages this is not what they are meant for. And also you will have to find a way to remove them again.
This is exactly what retained Mosquitto message are meant for.

Removal of retained messages is an implicit process. As soon as the next message arrives for a topic the previous message is removed.
Debian buster, bullseye on RPI-4, Intel NUC.
dz Beta, Z-Wave, RFLink, RFXtrx433e, P1, Youless, Hue, Yeelight, Xiaomi, MQTT
==>> dzVents wiki
User avatar
waltervl
Posts: 5711
Joined: Monday 28 January 2019 18:48
Target OS: Linux
Domoticz version: 2024.7
Location: NL
Contact:

Re: Can the MQTT Client Gateway post "retained" messages?

Post by waltervl »

Then I read the wrong article or mis interpretated it.....
Domoticz running on Udoo X86 (on Ubuntu)
Devices/plugins: ZigbeeforDomoticz (with Xiaomi, Ikea, Tuya devices), Nefit Easy, Midea Airco, Omnik Solar, Goodwe Solar
Bikey
Posts: 331
Joined: Sunday 22 February 2015 12:19
Target OS: Linux
Domoticz version: 2020.x
Location: Netherlands
Contact:

Re: Can the MQTT Client Gateway post "retained" messages?

Post by Bikey »

Yes, this is exactly what the use is for retained messages.

From the link I have referred to:
Retained messages help newly-subscribed clients get a status update immediately after they subscribe to a topic. The retained message eliminates the wait for the publishing clients to send the next update.

To me this is so common sense, I wonder if/why nobody else seems to have ran into this question with Domoticz?
User avatar
waltervl
Posts: 5711
Joined: Monday 28 January 2019 18:48
Target OS: Linux
Domoticz version: 2024.7
Location: NL
Contact:

Re: Can the MQTT Client Gateway post "retained" messages?

Post by waltervl »

From what I see in on the forum most people use MQTT to communicate from Domoticz with single devices (or hubs with devices) and those devices (or hubs) know their own status. They only get data from Domoticz to do something, switch on/off etc. So in that case no need to send retained messages from Domoticz. Domoticz is receiving device status data and sending commands.

To what other systems are you sending MQTT messages with Domoticz device info?
Domoticz running on Udoo X86 (on Ubuntu)
Devices/plugins: ZigbeeforDomoticz (with Xiaomi, Ikea, Tuya devices), Nefit Easy, Midea Airco, Omnik Solar, Goodwe Solar
Bikey
Posts: 331
Joined: Sunday 22 February 2015 12:19
Target OS: Linux
Domoticz version: 2020.x
Location: Netherlands
Contact:

Re: Can the MQTT Client Gateway post "retained" messages?

Post by Bikey »

I'm using Domoticz on a Raspberry Pi to read the status of GPIO-ports that are connected to my alarm system and be able to switch it on/of.

Domoticz acts as a hub to forward this information to HomeAssistant that does logic with it (to which I am migrating to) ;-)
User avatar
waltervl
Posts: 5711
Joined: Monday 28 January 2019 18:48
Target OS: Linux
Domoticz version: 2024.7
Location: NL
Contact:

Re: Can the MQTT Client Gateway post "retained" messages?

Post by waltervl »

You better use a separate application for that instead of Domoticz. Google for raspberry GPIO mqtt and a new world will arise :-) Probably easier to setup retained messages too.....
Domoticz running on Udoo X86 (on Ubuntu)
Devices/plugins: ZigbeeforDomoticz (with Xiaomi, Ikea, Tuya devices), Nefit Easy, Midea Airco, Omnik Solar, Goodwe Solar
User avatar
waaren
Posts: 6028
Joined: Tuesday 03 January 2017 14:18
Target OS: Linux
Domoticz version: Beta
Location: Netherlands
Contact:

Re: Can the MQTT Client Gateway post "retained" messages?

Post by waaren »

Bikey wrote: Monday 29 March 2021 18:03 Domoticz acts as a hub to forward this information to HomeAssistant that does logic with it.
Thx to @kiddigital, from 2021.1 build 13232 onwards domoticz MQTT supports retained messages.

To list all retained messages use following cmd on the command line

Code: Select all

sudo mosquitto_sub --retained-only -F "%U - %t - %p" -v -p 1883 -t "#" |  paste -sd '' | sed 's/\t/ /g' | sed 's/  / /g' | sed 's/\}1/\}\n1/g'
nb. because of buffering it might take until a next message before the command will show results.
Debian buster, bullseye on RPI-4, Intel NUC.
dz Beta, Z-Wave, RFLink, RFXtrx433e, P1, Youless, Hue, Yeelight, Xiaomi, MQTT
==>> dzVents wiki
Andrex
Posts: 92
Joined: Thursday 18 February 2016 9:11
Target OS: Linux
Domoticz version:
Contact:

Re: Can the MQTT Client Gateway post "retained" messages?

Post by Andrex »

waaren wrote: Monday 26 April 2021 14:00from 2021.1 build 13232 onwards domoticz MQTT supports retained messages.
Where do I set the retain flag? In the hardware section I can't find it.
User avatar
jvdz
Posts: 2265
Joined: Tuesday 30 December 2014 19:25
Target OS: Raspberry Pi / ODroid
Domoticz version: 4.107
Location: Netherlands
Contact:

Re: Can the MQTT Client Gateway post "retained" messages?

Post by jvdz »

Andrex wrote: Saturday 02 October 2021 15:49
waaren wrote: Monday 26 April 2021 14:00from 2021.1 build 13232 onwards domoticz MQTT supports retained messages.
Where do I set the retain flag? In the hardware section I can't find it.
In the Hardware setup, but this is only valid for certain formats :
Publish Topic:
Flat

Select the Topic('s) Domoticz will use to publish outgoing messages.
Flat - publish outgoing messagen on topic {domoticz/out}.
Hierarchical - publish outgoing messagen on topic {domoticz/out}/{$floorplan name}/{$plan name}.
Combined - Use both Flat and Hierarchical topic schemes.
Index - publish outgoing messagen on topic {domoticz/out}/{$idx}. (with or without Retain bit)
Name - publish outgoing messagen on topic {domoticz/out}/{$name}. (with or without Retain bit)
None - disable outgoing messages.
Last edited by jvdz on Saturday 02 October 2021 19:17, edited 1 time in total.
New Garbage collection scripts: https://github.com/jvanderzande/GarbageCalendar
Andrex
Posts: 92
Joined: Thursday 18 February 2016 9:11
Target OS: Linux
Domoticz version:
Contact:

Re: Can the MQTT Client Gateway post "retained" messages?

Post by Andrex »

jvdz wrote: Saturday 02 October 2021 16:35
In the Hardware setup, but this is only valid for certain formats (?) :
Publish Topic:
Flat

Select the Topic('s) Domoticz will use to publish outgoing messages.
Flat - publish outgoing messagen on topic {domoticz/out}.
Hierarchical - publish outgoing messagen on topic {domoticz/out}/{$floorplan name}/{$plan name}.
Combined - Use both Flat and Hierarchical topic schemes.
Index - publish outgoing messagen on topic {domoticz/out}/{$idx}. (with or without Retain bit)
Name - publish outgoing messagen on topic {domoticz/out}/{$name}. (with or without Retain bit)
None - disable outgoing messages.
I'm sorry, but I'm lost.
What you reported is what I see in the hardware setup, but where o I set the retain bit?
User avatar
jvdz
Posts: 2265
Joined: Tuesday 30 December 2014 19:25
Target OS: Raspberry Pi / ODroid
Domoticz version: 4.107
Location: Netherlands
Contact:

Re: Can the MQTT Client Gateway post "retained" messages?

Post by jvdz »

Andrex wrote: Saturday 02 October 2021 16:41 I'm sorry, but I'm lost.
What you reported is what I see in the hardware setup, but where o I set the retain bit?
... by setting the Publish topic to one of the items that has (with Retain).

I have no idea why the other message types do not have the option to set the retain option, but that seems to be the choice made in hardware.html:

Code: Select all

				<td>
					<select id="combotopicselect" style="width:200px" class="combobox ui-corner-all">
						<option value="1">Flat</option>
						<option value="2">Floor/Room</option>
						<option value="3">Flat + Floor/Room</option>
						<option value="4">Index</option>
						<option value="132">Index (with Retain)</option>
						<option value="8">Name</option>
						<option value="136">Name (with Retain)</option>
						<option value="0">None</option>
					</select>
					<br>
					<span>
						Select the Topic('s) Domoticz will use to publish outgoing messages. <br>
						<b>Flat</b> - publish outgoing messagen on topic <i>{domoticz/out}</i>.<br>
						<b>Hierarchical</b> - publish outgoing messagen on topic <i>{domoticz/out}/{$floorplan name}/{$plan name}</i>.<br>
						<b>Combined</b> - Use both <b>Flat</b> and <b>Hierarchical</b> topic schemes.<br>
						<b>Index</b> - publish outgoing messagen on topic <i>{domoticz/out}/{$idx}</i>.&nbsp;(with or without Retain bit)<br>
						<b>Name</b> - publish outgoing messagen on topic <i>{domoticz/out}/{$name}</i>.&nbsp;(with or without Retain bit)<br>
						<b>None</b> - disable outgoing messages.<br>
						<br>
						Note that <b>Hierarchical</b> only reports sensor updates for sensors that are placed on a floorplan/plan.<br>
						<br>
						Alternative topic prefixes for {domoticz/in} and {domoticz/out} can be entered below.<br>  
				</td>
Last edited by jvdz on Saturday 02 October 2021 19:17, edited 1 time in total.
New Garbage collection scripts: https://github.com/jvanderzande/GarbageCalendar
Andrex
Posts: 92
Joined: Thursday 18 February 2016 9:11
Target OS: Linux
Domoticz version:
Contact:

Re: Can the MQTT Client Gateway post "retained" messages?

Post by Andrex »

[quote=jvdz post_id=280344 time=1633185856 user_id=3281]
... by setting the Publish topic to one of the items that has [b](with Retain)[/b].
[/quote]

I'm sorry again...but in those items how do I specify that I need the retain bit?
This is what I see:
[attachment=0]chrome_KcZGCRAw60.png[/attachment]

P.s. I don't know how I broke the quote here... :?
Attachments
chrome_KcZGCRAw60.png
chrome_KcZGCRAw60.png (58.12 KiB) Viewed 1791 times
User avatar
jvdz
Posts: 2265
Joined: Tuesday 30 December 2014 19:25
Target OS: Raspberry Pi / ODroid
Domoticz version: 4.107
Location: Netherlands
Contact:

Re: Can the MQTT Client Gateway post "retained" messages?

Post by jvdz »

Click the Dropdownbox that contains flat and you understand what I mean..... but: I do not see the option to set the retain for the FLAT format in the html page, but assume it is possible when that option would be added or we know what the JSON call looks like.
New Garbage collection scripts: https://github.com/jvanderzande/GarbageCalendar
User avatar
jvdz
Posts: 2265
Joined: Tuesday 30 December 2014 19:25
Target OS: Raspberry Pi / ODroid
Domoticz version: 4.107
Location: Netherlands
Contact:

Re: Can the MQTT Client Gateway post "retained" messages?

Post by jvdz »

I also understand why the retain option is made only available to those 2 option as retain means is saves the last message for a specific topic and saving the last message for the other topics doesn't make sense when you like to save the last status per device!.
New Garbage collection scripts: https://github.com/jvanderzande/GarbageCalendar
User avatar
kiddigital
Posts: 438
Joined: Thursday 10 August 2017 6:52
Target OS: Raspberry Pi / ODroid
Domoticz version: Beta
Location: Netherlands
Contact:

Re: Can the MQTT Client Gateway post "retained" messages?

Post by kiddigital »

Andrex wrote:
jvdz wrote: Saturday 02 October 2021 16:44 ... by setting the Publish topic to one of the items that has (with Retain).
I'm sorry again...but in those items how do I specify that I need the retain bit?
This is what I see:
chrome_KcZGCRAw60.png
P.s. I don't know how I broke the quote here... :?
Are you sure you are at the right Beta version? Make sure you clear your browser cache after updating Domoticz.

When at the right version, you should see in the dropdown list also options that say that the retain bit will be set as well.
One RPi with Domoticz, RFX433e, aeon labs z-wave plus stick GEN5, ha-bridge 5.4.0 for Alexa, Philips Hue Bridge, Pimoroni Automation Hat
One RPi with Pi foundation standard touch screen to display Dashticz
User avatar
waltervl
Posts: 5711
Joined: Monday 28 January 2019 18:48
Target OS: Linux
Domoticz version: 2024.7
Location: NL
Contact:

Re: Can the MQTT Client Gateway post "retained" messages?

Post by waltervl »

It should be in the Publish Topic pulldown menu
Screenshot_20211003-000028~2.png
Screenshot_20211003-000028~2.png (97.69 KiB) Viewed 1757 times
Domoticz running on Udoo X86 (on Ubuntu)
Devices/plugins: ZigbeeforDomoticz (with Xiaomi, Ikea, Tuya devices), Nefit Easy, Midea Airco, Omnik Solar, Goodwe Solar
Andrex
Posts: 92
Joined: Thursday 18 February 2016 9:11
Target OS: Linux
Domoticz version:
Contact:

Re: Can the MQTT Client Gateway post "retained" messages?

Post by Andrex »

waltervl wrote: Sunday 03 October 2021 0:05 It should be in the Publish Topic pulldown menu
I don't have those 2 options in the pulldown menu:
firefox_2EpTc4v5Es.png
firefox_2EpTc4v5Es.png (5.96 KiB) Viewed 1744 times
I'm on version 2021.1 Compile Date: 2021-04-17 (I don't see the build number).
Last edited by Andrex on Sunday 03 October 2021 15:05, edited 2 times in total.
Andrex
Posts: 92
Joined: Thursday 18 February 2016 9:11
Target OS: Linux
Domoticz version:
Contact:

Re: Can the MQTT Client Gateway post

Post by Andrex »

kiddigital wrote: Saturday 02 October 2021 23:08 Are you sure you are at the right Beta version? Make sure you clear your browser cache after updating Domoticz.
When at the right version, you should see in the dropdown list also options that say that the retain bit will be set as well.
I've just update to the latest beta (Version: 2021.1 (build 13682)) and I still don't have those 2 options...

Edit: never mind, it was the same cache problem, now I see those 2 options.
Thanks!!!
Last edited by Andrex on Sunday 03 October 2021 15:05, edited 1 time in total.
Post Reply

Who is online

Users browsing this forum: No registered users and 1 guest