[90] MQTT Support

[wildfire]

I still think the mqtt module should work without IDX aswell, programming nodes everytime you change someting in domoticz is a drag if there somewer build in to your house, in a top corner of a room somewhere.
so you should be able to make a virtual device listen to /raw/garage/temp or /raw/garage/humi.
this would make my life easyer at least
that way i can also skip using node red for rerouting every post and adding an IDX

[nayr]

Wrote a NodeJS module for interacting directly w/Domoticz, forked discussion regarding it into its own thread: viewtopic.php?f=21&t=10190

[wildfire]

this looks great, im gonne try this tonight!

thanks

[kylegordon]

Looks good, thank you!

Annoyingly my continuing strace of domoticz is doing just that. It hasn't crashed at all now!

[nayr]

I noticed when writing my module I sent in some data that crashed domoticz durring testing, I think more error checking needs to be done in the MQTT hardware.

but I added error checking to my node module to prevent Strings (or NaN) being pushed when expecting an integer and vice versa, I have not witnessed it happen recently.

I suspect your problem is data related, I would log both whats being sent and received and then focus on the last thing sent before everything goes pear shaped.

Another thing I found when writing my Node Module was that the MQTT API was missing the addlogmessage command that exists in the URL API... so I fixed that, hopefully it'll be merged soon: https://github.com/domoticz/domoticz/pull/382

[kylegordon]

It is quite possibly data related, and I have quite a lot of it flying across the MQTT broker.

I've already raised the issue with the authors about the JS string 'NaN' crashing Domoticz, and I was told that I should ensure that such a string is never sent to Domoticz. So much for defensive programming. It's also destroyed the wife acceptance factor.

meh

[Toni]

I've been logging my domoticz/in topic into a file, and didn't notice any problems with the data though.

[Toni]

Today I also switched from feeding sensor data through MQTT to feeding in the data using the REST API. Let's see...

A very minor change in my automation, originally I was feeding the data through the API, and still had the old versions in version control. It's good to use Git also for your event scripts...

[Toni]

And it looks good so far, with MQTT it would already have crashed a number of times.

So, don't rely on MQTT, but use the REST API instead to feed in external data to Domoticz.

[nayr]

Ive been pushing alot of data through MQTT for a while now without issue, at pretty high frequency to boot.. I think with a good library thats sanitizing inputs, such as my node-domoticz-js module, its plenty reliable..

I think if I tried to fallback to the URL API right now things would slow way down, when I turn my volume knob Ive got it throttled back to 1 command per 100ms and it still generates a ton of traffic for a few moments of movement (10 updates a second).. establishing http connections and closing them would suck up alot of that time.

[Toni]

I wish it would have been working for me. But it's not just about the awful reliability. My CPU use skyrocketed on Oct 14th 2015, and went back down yesterday when I switched from MQTT to the REST API. I think it was because of a Domoticz change, as I've been running the same MQTT scripts since last summer.

cpu.PNG (113.44 KiB) Viewed 5155 times

[nayr]

thats pretty dramatic, was that your script eating up cpu or domoticz?

ive seen no increase in cpu load since using the MQTT interface, quite the opposite actually.. my memory usage was pegged @ 100% and settled down once I started moving old scripts over to it.. but I also started cleaning up things and using a common library.. removed a ton of event scripts since I dont have to make calls to my software.

It just goes to show, it all comes down to your code.. its going to be hard to peg this on MQTT specifically, yeah the URL API is able to cope better.. that could be for lots of reasons.. MQTT is definitely a more raw interface, and less forgiving than the URL API.

[Toni]

I still sent the same data to Domoticz, and Domoticz still listens to domoticz/in. There's just no traffic in there.

So all I did was to change the publishing of my sensor values from mqtt to the rest api. I use a perl script, and the change was literally a one-liner, from mqtt publish to http get. So all the same functionality is still there, only the publishing method changed.

[SchattenMann]

Is it possible to include the Hardware ID/Name/Type on default MQTT messages? This would be a great help!
right now we only have IDX, sType and dType...

on a different note is it possible to send the following command via MQTT?

Code: Select all

/json.htm?type=setused&idx=<idx>&setpoint=<setpoint>&mode=<mode>&used=true 

I've tried:

Code: Select all

{"type": "setused", "idx": "22" , "setpoint": "13.07", "mode": "TemporaryOverride", "used": "true"}
{"command": "setused", "idx": "22" , "setpoint": "13.07", "mode": "TemporaryOverride", "used": "true"}

without success so I take not all HTTP commands can be "translated" to MQTT?

[nayr]

I get all that info out MQTT

Code: Select all

{"Battery":255,"RSSI":12,"dtype":"Light/Switch","id":"000140E0","idx":145,"name":"TV","nvalue":0,"stype":"Selector Switch","svalue1":"0","switchType":"Selector","unit":1}

and yeah not all HTTP commands are in MQTT, ive been trying to add the missing ones as I find them.. (addlogmessage for example)

you might see if you can use the udevice command to get what you want

[SchattenMann]

I get all that info out MQTT

Code: Select all

{"Battery":255,"RSSI":12,"dtype":"Light/Switch","id":"000140E0","idx":145,"name":"TV","nvalue":0,"stype":"Selector Switch","svalue1":"0","switchType":"Selector","unit":1}

So fo I but that doesn't give you the HardwareName for instance. It's really useful to know if it's a Z-Wave/Nest/Honeywell/Hue/Oregon upfront since I don't want to have the same treatment for a Nest Thermostat that I have for a Honeywell TRV for instance - makes sense?
For now I have an HTTP GET request to get a list of all devices and store the HardwareName (along side other things) in a JS Object to be later used by node-red but this step would be unnecessary if the MQTT Message could push this as well.

and yeah not all HTTP commands are in MQTT, ive been trying to add the missing ones as I find them.. (addlogmessage for example)

Yeah i've found out the same thing now. It would be great to add Thermostart/Honeywell support

you might see if you can use the udevice command to get what you want

for now I'm using /json.htm?type=devices& to get a full list of all devices on the startup code

[nayr]

oic, yeah I think that would be really nice..

for example my Home Theatre controller uses alot of dummy switches.. right now you have to configure the idx of each switch manually, with you could create a dummy hardware device just for my software, and my software could detect new switches as they were created.

I'd also like to get the selector level name in output, for example svalue=20 = PlayStation3 in the UI but I cant get the level name via API like I can the switch name.

I also believe were missing the ability in MQTT to create devices, with all these exposed in the API I could likely make my software smart enough it sets its self up and creates all the hardware like a native device would.

[SchattenMann]

yeah, it would be great to see Domoticz MQTT support to keep moving forward as I'm planning to ditch my Vera and use Domoticz + node-red for all my logic

[jeanvdr]

Hi,

I try to connect a nodeMCU-esp12 to a RasberryPi Domoticz server through mqtt.
I use C to program the esp12 with Arduino IDE.
No problem to transmit some info to Domoticz from the nodeMCU (acting as a sensor node) using mqtt and publishing in domoticz/in.
On the contrary, no way to get any data when subscribing on domoticz/out ?
I get the data on my PC with python/paho so I know data are passing by.
When I use mosquitto_pub on my RaspberryPi to publish in domoticz/out, then my nodeMCU receive the data.
So for now, I use a lua-script in Domoticz to republish some data in domoticz/out to get it to my nodeMCU (I know I could use node-red)
I've tried all mqtt librairies existing on Arduino, same result

Any clue, anybody ?

Thanks a lot

Jean

[nayr]

@jeanvdr, Can you try using another client to see if domoticz is sending data to domoticz/out?

to anyone else concerned, I worked with @gaudryrc and he expanded the new Selector Switches output in MQTT: https://github.com/domoticz/domoticz/issues/459

Code: Select all

{
	"Battery":255,
	"LevelNames":"Off|TV|PS3|PS4|NVR|FM",
	"LevelOffHidden":"false",
	"RSSI":12,
	"SelectorStyle":"0",
	"dtype":"Light/Switch","
	id":"000140E0",
	"idx":145,
	"name":"TV",
	"nvalue":2,
	"stype":"Selector Switch",
	"svalue1":"10",
	"switchType":"Selector",
	"unit":1
}