[90] MQTT Support

[Kouseri]

Yes, I think that might help. And you can link to the message from there.

Bug report submitted. The bug report can be found from here.

[Kouseri]

I noticed a strange thing when added RFXCOM transceiver (RFXtrx433E) to system today. Domoticz changed Hardware names of all my MQTT protocol based devices
from "Unknown?" to "RFXtrx433E". More information can be found via the bug report (link included to my previous post).

[gizmocuz]

Finally i think i have build a native mqtt client in domoticz

Who can PM me a broker to connect to ?

Is there a standard for home automation messages?

[arendst]

I use mosquitto as broker and mqtt-spy (https://code.google.com/p/mqtt-spy/) as visual tool.

[pepijn]

Hi Rob,

Broker:
For testing you could use MQTT Dashboard (broker.mqttdashboard.com:1883 / www.mqttdashboard.com) or test.mosquitto.org to setup your local broker, mosquitto will be a good choice.

Protocol:
MQTT describes the MQTT-SN standard (SN stands for Sensor Network) which is aimed for non IP based networks like zwave.
The most flexible way (and maybee te most complicated one) is to create some kind of mechanism to map mqtt topics to domoticz devices.

[Banjer]

Hi Rob,
MQTT describes the MQTT-SN standard (SN stands for Sensor Network) which is aimed for non IP based networks like zwave.

MQTT-SN uses a gateway in-between MQTT-SN devices and the MQTT broker, and afaik it only describes the way it communicaties with that broker, not how the content is structured. Since there already gateways that can run on a Pi (https://github.com/ty4tw/MQTT-SN) i wouldn't put much effort in that.

[Kouseri]

Like!

Finally i think i have build a native mqtt client in domoticz

Who can PM me a broker to connect to ?

Is there a standard for home automation messages?

[gizmocuz]

Hi Rob,

Broker:
For testing you could use MQTT Dashboard (broker.mqttdashboard.com:1883 / http://www.mqttdashboard.com) or test.mosquitto.org to setup your local broker, mosquitto will be a good choice.

Protocol:
MQTT describes the MQTT-SN standard (SN stands for Sensor Network) which is aimed for non IP based networks like zwave.
The most flexible way (and maybee te most complicated one) is to create some kind of mechanism to map mqtt topics to domoticz devices.

I know how to setup a broker and work with it, but the mechanism to push messages is the problem.
Like i think we should just json for the messages, we need a SenserID, numeric sensor id (maybe also child id), a sensor description (like temp/hum/baro or temp+hum) and the value

i like the 'keep it simple' approach, so letting users define custom messages to sensors is a bit complicated

[y1ann1s]

Hi
So what does your implementation of the client do?
how can we test /config?

is it two way? you should at least offer configuration for default posting and subscription locations
Json is fine or you could follow a tree like the below

"default listen topic"/SensorID/numeric sensor id (maybe also child id)/ a sensor description (like temp/hum/baro or temp+hum)
____________________________________________________________/the value

and the same way posting to the default post topic

[gizmocuz]

yes two way, thats why we need a sender_id, so domoticz will know it does have to react on messages posted by itself.

i thought of something like (pseudo)

Code: Select all

json {
 sender_id: domoticz (for incoming it does not mather)
 node_id: 1
 child_id: 1
 type: temp+hum+baro
 value: 21.5;69;1018.3
}

json {
 sender_id: your client
 node_id: 2
 child_id: 1
 type: light
 value: on/off
}

json {
 sender_id: your client
 node_id: 2
 child_id: 1
 type: dimmer
 value: 73
}

json {
 sender_id: your client
 node_id: 3
 child_id: 1
 type: temp
 value: 21.3
}

[gerardvs]

Ideally you could make it compatible with https://iotdb.org/

[y1ann1s]

I am not sure I follow gismocuz

the topic you post to the one you subscribe can be different
Why do you care who posted it. It is an event in the mqtt that you either subscribe, so you do something about it cause someone posted something for you
or post to some other location for everyone else to do whatever they like.
I would suggest following a topic structure similar to the devices page
so HW would be MQTT,lets assume that in mqtt tha base topic would be "ToDomoticz"

so an example post from a sensor would be
Topic , data
/todomoticz/1(id of sensor)/unit /Name,"Test device"
/todomoticz/1(id of sensor)/unit /Type,"Temp + Humidity + Baro"
/todomoticz/1(id of sensor)/unit /subtype,"1"
/todomoticz/1(id of sensor)/unit /Data ....etc

another one could be

/todomoticz/2(id of sensor)/unit /Name,"Temp device"
/todomoticz/2(id of sensor)/unit /Type,"Switch"
/todomoticz/2(id of sensor)/unit /subtype,"contact"
/todomoticz/2(id of sensor)/unit /Data ,"1"

so this creates an input tree

in the same format you can post to an output tree

based on mqtt retain policy and the device will testament you might have or not all necessary info.
If you do add the device as an unused one,if not wait until it becomes valid device

The advantage over json is that the sensor/device post only the value to a predefined topic for it,or have another engine like Node-red doing the remmaping ,rather that creating a full json message everytime.

I have already setup a connection to domoticz from node-red through the node.js lua script out there.

So I am more than happy to test any implementation and next steps.I can send you a visualization of my mqtt tree and the node-red mqtt manipulations to repost data is you find it helpful

I think this feature is one of the most important thing happening for Domoticz.It'll give the ability to interface with anything easily.

[Dnpwwo]

Is forcing a 'base topic' a good idea? I haven't used MQTT specifically but I have used other publish and subscript mechanisms.

I assumed that this would be implemented by allowing people to specify the topic at the virtual sensor level so that Domoticz can pick up existing messages that are on the network, not just ones custom crafted for it. That way the only topic specified would be the one with the data in it, in the example below: /todomoticz/2(id of sensor)/unit /Data ,"1", or /any/old/topic/value, "true".

The other fields are only required to auto create sensors which I think would be of limited value given the additional complexity it introduces.

[Sicco28]

For the items published by domoticz, this should remain simple: similar to the simple lua script for publishing mqtt messages at the beginning of this thread, the structure should be:
- topic name should include the device name + the attribute
- payload includes the value

Messages from Domoticz are status/changes in the domoticz system which are published for anyone to subscribe. Any other device (e.g. an ardhuino), which has interest in a specific topic (read: device, or virtual device in domoticz), can subscribe to the topic, and act upon changes published.

E.g. please see example verbose messages from my mqtt broker:
events/domoticz/Woonkamer/Humidity 45
events/domoticz/Woonkamer/Dewpoint 7.250669002533
events/domoticz/Woonkamer/Temperature 19.5
events/domoticz/Elektriek/Utility 350
events/domoticz/Gasmeter/Utility 2.8959999084473
events/domoticz/Woonkamer-livingcolor-red On
actions/domoticz/152 19.9;43;0
events/domoticz/Etherten/Temperature 19.89999961853
events/domoticz/Etherten/Humidity 43
events/domoticz/Etherten/Dewpoint 6.9491577148438
events/domoticz/Woonkamer-livingcolor-red Off

In this case, my ethernet-connected arduino is subscribed to the topic 'events/domoticz/Woonkamer-livingcolor-red'. In case the virtual device in domoticz is switched On, the arduino acts upon it (and turns the living room lamp on ...duh ).

The device ID, device type is not interesting in this context -> if you intend to sync systems (including syncing device types and ID's), the JSON api is available, and provides you with all necessary information.

Currently, the only thing missing from the simple lua publishing script, is that changes in the dimmer level of a virtual switch do not trigger the "devicechanged" lua script. If we can ensure that all of these domoticz-system status updates are publised to the mqtt broker, that would be perfect.

As for subscribing by domoticz to a specific topic, ...that's more complicated (because of the many supported device types in domoticz). Currently, I've copied the node.js script (also from this thread), to a update dummy temperature device (see actions/domoticz/152 19.9;43;0 in above example).

[gizmocuz]

If someone wants to finish the MQTT class, please do so

[tommysharp]

Greetings guys, I'm just delving into this and trying to get Domoticz to "talk" to the mosquitto MQTT server.

So far I think I'm almost there.... At least on the MQTT server side I can see ping requests coming from domoticz and the MQTT server is responding to them. This is using the new MQTT hardware option in Domoticz.

Anyone able to give some simple bullet points on what I need to do now to publish a message whenever a certain contact switch is changed?

Also being able to subscribe to messages to update power usage in Domoticz would be great!

[pepijn]

@tommysharp

If someone wants to finish the MQTT class, please do so

Uhh ... Some coding

[A1AD]

Greetings guys, I'm just delving into this and trying to get Domoticz to "talk" to the mosquitto MQTT server.
So far I think I'm almost there....

Yes you are almost there, we just need to wait until someone finish the code
See http://www.domoticz.com/forum/viewtopic.php?f=6&t=5451 for more info.

[gskout]

Hi,

Gizmocus in an older post talked about a 'keep it simple' approach, and is probably the best route to take (as in most things anyway ).

A little practical view of MQTT may help clear up things:
1. MQTT relies on tcpip, so almost always a Gateway is needed between sensors and the MQTT broker, to translate the sensors data. Exceptions are sensors that have hardware capable of tcpip comms, but these are rare and often not battery powered. A Gateway may have many sensors attached to it, and the GatewayID should be handled by the domoticz MQTT mechanism.
2. Essentially, the domoticz MQTT mechanism needs to subscribe to a well defined topic in order to get messages from sensors gateway(s), and also publish to dynamic topics based on GatewayID, so relays, dimmers, actuators etc can receive commands from domoticz and do some work.
3. MQTT topics and messages are strings.

In order to 'keep things simple' here is a proposition:

1. Domoticz may subscribe to fixed topic 'domoticzIn/#', so it will receive all messages to topics domoticzIn/.../... etc. Sensors gateways will publish messages to topic 'domoticzIn/gatewayID', so domoticz will gather the gatewayID from the subtopic.
2. Gateway should publish single formatted message (sensor data) to MQTT, that will start with sensorID and all other sensor essential data.

Example:
Domoticz has subscribed to topic 'domoticzIn/#'.
Gateway with ID=1001 will publish to topic 'domoticzIn/1001', another with ID=2001 will publish to 'domoticzIn/2001' and so on.
Let's say a temperature sensor is attached to gateway with ID=1001.
This gateway will publish on topic 'domoticzIn/1001' single formatted messages with all temperature sensor data, eg: 'sensorID;sensorType;temperatureData;BatteryLevel;SignalLevel'.
Depending on sensorType (that will always be at the 2nd field), domoticz will know how to split the data, and handle them as already done for several other hardware it supports.

3. In the case of relays, dimmers, actuators etc, the gateway will have to publish an informative message first (as above), so domoticz will register the device. It will also subscribe to topic 'domoticzOut/gatewayID', and will receive all messages on this topic.

Example:
Gateway with ID=3005 has a dimmer device attached.
When dimmer is powered on, the gateway publishes to topic 'domoticzIn/3005' the informative message, so domoticz know the gatewayID, sensorID, sensorType and registers the device.
The gateway also subscribes to topic 'domoticzOut/3005', and domoticz will publish all single formatted messages (commands) for the dimmer on this topic.


To summarise, Domoticz will subscribe to a fixed (probably hardcoded?) topic 'domoticzIn/#', and will be able to publish to dynamic topics 'domoticzOut/gatewayID' where gatewayID has been already gathered from its subscription subtopic.

Of course domoticz needs to have well defined device structures (in string format, as MQTT messages are strings) for any possible device type that needs to handle (I guess they are already there for other hardware, but probably in binary format).

This is my 'keep things simple' approach, and would be nice if you could share your thoughts (please forgive me if there is any confusion, I tried to be simple ).

[stevejbauer]

Would it be possible to use a schema for the mqtt stuff so that it matches what other home automation software like openhab is using? This would potentially increase the number of things supported with less software having to be written.

Steve