MQTT to virtual sensors Topic is solved

Moderator: leecollings

Post Reply
PieterS
Posts: 196
Joined: Wednesday 31 May 2017 16:06
Target OS: NAS (Synology & others)
Domoticz version: 2024.7
Location: NL
Contact:

MQTT to virtual sensors

Post by PieterS »

Not very familiar to MQTT and converting data to virtual sensors.
I am running Mosquitto on Synology and running SBFspot and Domoticz Beta V 2021.1 (build 13243) on Raspberry.
SBFspot dumps every 5 minutes its data to the broker and Domoticz/in tells me in the log this information:

Code: Select all

2021-05-04 12:10:03.896 Debug: MQTT: MQTT: Topic: domoticz/in, Message: {"Timestamp": "04/05/2021 12:10:03","SunRise": "04/05/2021 06:05:00","SunSet": "04/05/2021 21:04:00","InvSerial": 2100086206,"InvName": "SN: 2100086206","InvTime": "04/05/2021 12:09:36","InvStatus": "Ok","InvTemperature": 34.600,"InvGridRelay": "Closed","EToday": 2.566,"ETotal": 40273.693,"PACTot": 773.000,"UDC1": 371.760,"UDC2": 371.130,"IDC1": 1.067,"IDC2": 1.077,"PDC1": 396.000,"PDC2": 399.000}
So far so good!
I made some virtual sensors like
IDC1 (idx 180)
IDC2 (idx 181)
UDC1 (idx 182)
UDC2 (idx 183)
but how do I filter and update these sensors in Domoticz? Could not find an examplescript. I read the wiki about JSON but I don't know if the information is stored in the domoticz.db.
I Looked in my other script named Luftdaten but that is based on a query to the device. This is just the opposite, I think...
Lot's of questions. Maybe can give somebody a hint?
Synology with Domoticz build (V2024.7) in Docker
mgugu
Posts: 217
Joined: Friday 04 November 2016 12:33
Target OS: Raspberry Pi / ODroid
Domoticz version: Beta
Location: France
Contact:

Re: MQTT to virtual sensors

Post by mgugu »

You have to send mqtt messages accepted by Domoticz (check https://www.domoticz.com/wiki/MQTT#MQTT_to_Domoticz).
No possibility in Domoticz to trigger other messages (AFAIK)
So I see 2 solutions:
1- Change the SBF output format according to the Domoticz requirements
2- Use a server like node js or Node Red to intercept the SBF message and re route it in the correct format.
PieterS
Posts: 196
Joined: Wednesday 31 May 2017 16:06
Target OS: NAS (Synology & others)
Domoticz version: 2024.7
Location: NL
Contact:

Re: MQTT to virtual sensors

Post by PieterS »

Little disapointed with your suggestions. Of course I read your first hint to the Wiki about MQTT to Domoticz. But that is too technical for me without examples. And there was a reference to JSON
I was glad to get much more information of my Solar Inverter with MQTT then I got before. I hoped that there was a way to pick the right info to update the virtual devices... And to send it further on to InfluxDB and Grafana.
To rearrange the format in SBFspot is a nogo for me.
I have running Node Red on my Raspberry. Without any flow at this moment. Another goal to go. :roll:

Thanks for your answer anyway.
Synology with Domoticz build (V2024.7) in Docker
User avatar
waaren
Posts: 6028
Joined: Tuesday 03 January 2017 14:18
Target OS: Linux
Domoticz version: Beta
Location: Netherlands
Contact:

Re: MQTT to virtual sensors

Post by waaren »

PieterS wrote: Tuesday 04 May 2021 13:02 Lot's of questions. Maybe can give somebody a hint?
The MQTT implementation of domoticz requires the MQTT messages to be in a defined format. If you have no influence on the message format itself you will need something like nodered to convert the messages into something that can be send to- and interpreted by domoticz.

I will move the topic to the nodered subforum.
Debian buster, bullseye on RPI-4, Intel NUC.
dz Beta, Z-Wave, RFLink, RFXtrx433e, P1, Youless, Hue, Yeelight, Xiaomi, MQTT
==>> dzVents wiki
mgugu
Posts: 217
Joined: Friday 04 November 2016 12:33
Target OS: Raspberry Pi / ODroid
Domoticz version: Beta
Location: France
Contact:

Re: MQTT to virtual sensors

Post by mgugu »

PieterS wrote: Tuesday 04 May 2021 15:15 I have running Node Red on my Raspberry. Without any flow at this moment. Another goal to go. :roll:
Thanks for your answer anyway.
This is an excellent solution if you have Node Red already running. You will find a lot of examples to create nodes for mqtt listening, processing and then sending formated mqtt messages to Domoticz.
User avatar
FireWizard
Posts: 1770
Joined: Tuesday 25 December 2018 12:11
Target OS: Raspberry Pi / ODroid
Domoticz version: Beta
Location: Voorthuizen (NL)
Contact:

Re: MQTT to virtual sensors

Post by FireWizard »

Hello Pieter, @PieterS

@waaren wrote:
The MQTT implementation of domoticz requires the MQTT messages to be in a defined format. If you have no influence on the message format itself you will need something like nodered to convert the messages into something that can be send to- and interpreted by domoticz.
I would like to add to that statement the following:

It is a bad idea to publish a value to domoticz/in, which is not properly formatted for the sensor where the IDX is referring to.
Maybe and hopefully the message you send to domoticz/in is rejected by Domoticz as the part "command:udevice" is missing.
The command you send to Domoticz does not comply with the defined format, as needed for Domoticz and probably you will find in the Domoticz log, a message indicating something as "MQTT bad command received".

So I suggest to change the topic to something like sbfspot/data instead of domoticz/in.

In the example below I have used that topic (sbfspot/data)

Screenshot_SBFspot1.png
Screenshot_SBFspot1.png (56.48 KiB) Viewed 8280 times

As you can see I have published the data under the topic sbfspot/data, but you can take any name you like, except a name with domoticz in it.
The values, you want to see in Domoticz are (your example) IDC1, IDC2, UDC1 and UDC2.

The "function" node converts the received message to 4 messages, correctly defined for Domoticz' Current (Ampere 1 phase) and Voltage sensors.

See the contents below:

Code: Select all

msg1 = {};
msg2 = {};
msg3 = {};
msg4 = {};

msg1.payload = {"command":"udevice","idx":181,"nvalue":0,"svalue":msg.payload.IDC1.toString()}; 
msg2.payload = {"command":"udevice","idx":182,"nvalue":0,"svalue":msg.payload.IDC2.toString()};
msg3.payload = {"command":"udevice","idx":183,"nvalue":0,"svalue":msg.payload.UDC1.toString()};
msg4.payload = {"command":"udevice","idx":184,"nvalue":0,"svalue":msg.payload.UDC2.toString()};

return [[msg1,msg2,msg3,msg4]];
This will result in the following output:

Screenshot_SBFspot2.png
Screenshot_SBFspot2.png (49.7 KiB) Viewed 8274 times
The complete flow you will find below:

Code: Select all

[{"id":"793b1ee2.03615","type":"mqtt in","z":"5051a917.d70fa8","name":"SBFspot In","topic":"sbfspot/data","qos":"2","datatype":"json","broker":"f9f13036.e28b58","nl":false,"rap":true,"rh":0,"x":880,"y":260,"wires":[["a49a4867.0f0b08"]]},{"id":"a49a4867.0f0b08","type":"function","z":"5051a917.d70fa8","name":"","func":"msg1 = {};\nmsg2 = {};\nmsg3 = {};\nmsg4 = {};\n\nmsg1.payload = {\"command\":\"udevice\",\"idx\":181,\"nvalue\":0,\"svalue\":msg.payload.IDC1.toString()}; \nmsg2.payload = {\"command\":\"udevice\",\"idx\":182,\"nvalue\":0,\"svalue\":msg.payload.IDC2.toString()};\nmsg3.payload = {\"command\":\"udevice\",\"idx\":183,\"nvalue\":0,\"svalue\":msg.payload.UDC1.toString()};\nmsg4.payload = {\"command\":\"udevice\",\"idx\":184,\"nvalue\":0,\"svalue\":msg.payload.UDC2.toString()};\n\nreturn [[msg1,msg2,msg3,msg4]];","outputs":1,"noerr":0,"initialize":"","finalize":"","libs":[],"x":1060,"y":260,"wires":[["66e48cc4.fe1b74","33baa9b3.8628ce"]]},{"id":"66e48cc4.fe1b74","type":"debug","z":"5051a917.d70fa8","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","statusVal":"","statusType":"auto","x":1250,"y":320,"wires":[]},{"id":"33baa9b3.8628ce","type":"mqtt out","z":"5051a917.d70fa8","name":"To Domoticz","topic":"domoticz/in","qos":"0","retain":"false","respTopic":"","contentType":"","userProps":"","correl":"","expiry":"","broker":"f9f13036.e28b58","x":1250,"y":260,"wires":[]},{"id":"f9f13036.e28b58","type":"mqtt-broker","name":"localhost","broker":"127.0.0.1","port":"1883","clientid":"","usetls":false,"compatmode":true,"keepalive":"60","cleansession":true,"birthTopic":"","birthQos":"0","birthRetain":"false","birthPayload":"","closeTopic":"","closeQos":"0","closePayload":"","willTopic":"","willQos":"0","willPayload":""}]
Do not forget to configure your MQTT server (broker) in the MQTT nodes.
These are currently configured for localhost.

Try it and let us know.

Regards
PieterS
Posts: 196
Joined: Wednesday 31 May 2017 16:06
Target OS: NAS (Synology & others)
Domoticz version: 2024.7
Location: NL
Contact:

Re: MQTT to virtual sensors

Post by PieterS »

H Firewizard,

Thank you for your reply! Glad to get an example for a flow. Did not make any progress till now. I am an early beginner.

I got no errors about JSON in the latest beta version of Domoticz with topic domoticz/in. Not in any tab. And then I got at least some input from SBFspot...
It is a bad idea to publish a value to domoticz/in, which is not properly formatted for the sensor where the IDX is referring to.
No message found.

When i change the topic in the configurationfile of SBFspot to what you suggest than there are no readings in Domoticz.
This is what I see:
Image
So MQTT seems to work.
This is the part of the config-file about MQTT in SBFspot. I set the host to 192.168.1.18 and topic to sbfspot/data

Code: Select all

# Full path to mosquitto_pub executable
MQTT_Publisher=/usr/bin/mosquitto_pub

# IP address or DNS name of MQTT Broker
# Don't use test broker for production environments
MQTT_Host=192.168.1.18

# Port (Optional)
#MQTT_Port=

# Topic Name
# When using public broker like test.mosquitto.org make sure to define a unique topic name
# by appending the serial number of the inverter to avoid you receive data of another SBFspot user.
MQTT_Topic=sbfspot/data

# Format of message items to be sent
# JSON: MQTT_ItemFormat="{key}": {value}
# TEXT: MQTT_ItemFormat={key}:{value}
# XML:  MQTT_ItemFormat=<item name="{key}" value="{value}" />
MQTT_ItemFormat="{key}": {value}

# MQTT_ItemDelimiter (none|blank|comma|semicolon default comma)
# JSON: MQTT_ItemDelimiter=comma
# TEXT: MQTT_ItemDelimiter=semicolon
# XML : MQTT_ItemDelimiter=none

MQTT_ItemDelimiter=comma

# Mandatory arguments for mosquitto_pub executable
# JSON: MQTT_PublisherArgs=-h {host} -t {topic} -m "{{message}}"
# TEXT: MQTT_PublisherArgs=-h {host} -t {topic} -m "{message}"
# XML : MQTT_PublisherArgs=-h {host} -t {topic} -m "<mqtt_message>{message}</mqtt_message>"
MQTT_PublisherArgs=-h {host} -t {topic} -m "{{message}}"

# Data to be published (comma delimited)
MQTT_Data=Timestamp,SunRise,SunSet,InvSerial,InvName,InvTime,InvStatus,InvTemperature,InvGridRelay,BTSignal,GridFreq,EToday,ETotal,PACTot,PDCTot,FeedTm,UAC1,UDC1,UDC2,IDC1,IDC2,PDC1,PDC2

# Possible keywords are (if supported by your inverter):
# SBFspot Alias   Code                Description
# =======================================================================
# Timestamp                           Current date/time
# InvTime                             Inverter date/time
# SunRise                             Sunrise time
# SunSet                              Sunset time
# InvSerial                           Serial Number
# InvName         NameplateLocation   Device name
# InvClass        NameplateMainModel  Device class
# InvType         NameplateModel      Device type
# InvSwVer        NameplatePkgRev     Software package
# InvStatus       OperationHealth     Condition
# InvTemperature  CoolsysTmpNom       Operating condition temperature
# InvGridRelay    OperationGriSwStt   Grid relay/contactor
# ETotal          MeteringTotWhOut    Total yield
# EToday          MeteringDyWhOut     Day yield
# PACTot          GridMsTotW          Power
# PDC1/PDC2       DcMsWatt            DC power input String 1/2
# UDC1/UDC2       DcMsVol             DC voltage input String 1/2
# IDC1/IDC2       DcMsAmp             DC current input String 1/2
# OperTm          MeteringTotOpTms    Operating time
# FeedTm          MeteringTotFeedTms  Feed-in time
# PAC1            GridMsWphsA         Power L1
# PAC2            GridMsWphsB         Power L2
# PAC3            GridMsWphsC         Power L3
# UAC1            GridMsPhVphsA       Grid voltage phase L1
# UAC2            GridMsPhVphsB       Grid voltage phase L2
# UAC3            GridMsPhVphsC       Grid voltage phase L3
# IAC1            GridMsAphsA_1       Grid current phase L1
# IAC2            GridMsAphsB_1       Grid current phase L2
# IAC3            GridMsAphsC_1       Grid current phase L3
# GridFreq        GridMsHz            Grid frequency
# BatTmpVal       BatTmpVal           Battery temperature
# BatVol          BatVol              Battery voltage
# BatAmp          BatAmp              Battery current
# BatChaStt       BatChaStt           Current battery charge status

# End of Config
In MQTT Explorer there is a new topic with a stream of data on the broker which runs on my Synology.
Image

On the Raspberry (192.168.1.15) I made a new flow by making an import of your example.
Image

In Node-RED: The MQTT out node has as server localhost and Topic domoticz/in because Node-REED and Domoticz run both on this machine.

In Domoticz I made all virtual sensors
Image

But there is no input in logfile in Domoticz (192.168.1.15) from Node-red...
Seems there is no data from SBFspot to Nodered..
On Raspberry I did the command

Code: Select all

sudo systemctl daemon-reload
sudo service domoticz stop
sudo service domoticz start
How can I check that there is traffic from broker to Node-RED?
I hope you have the answer?

Node-RED is doing well with the other flow.. Not the one I really need :roll:
Image
Synology with Domoticz build (V2024.7) in Docker
User avatar
FireWizard
Posts: 1770
Joined: Tuesday 25 December 2018 12:11
Target OS: Raspberry Pi / ODroid
Domoticz version: Beta
Location: Voorthuizen (NL)
Contact:

Re: MQTT to virtual sensors

Post by FireWizard »

Hello Pieter, @PieterS

You wrote:
Thank you for your reply! Glad to get an example for a flow. Did not make any progress till now. I am an early beginner.
You are very close to the end result, but first I like to explain a little.

As we already discussed earlier and it was already mentioned earlier, SBFspot is not able to communicate directly with Domoticz, as Domoticz requires its own data format So we have to convert the data from, what is send by SBFspot, to what Domoticz understand.
This "data conversion" is done, in this case, by Node Red.

We have chosen to communicate by means of MQTT.
This means that SBFspot publishes (sending) its data to a MQTT server (broker. e.g. Mosquitto). Node Red subscribes (receiving) to that data. Therefore SBFspot should publish the data under a topic, in this case sbfspot/data and Node Red subscribes to the same topic.

Next step is that Node Red converts the data to the format, required by Domoticz and publishes again to a topic that Domoticz can read.
Default Domoticz subscribes to domoticz/in (and publishes to domoticz/out). So Node Red publishes the data to domoticz/in.

I assume you have 1 MQTT server (broker) on your network.

Let us step through your post.
I got no errors about JSON in the latest beta version of Domoticz with topic domoticz/in. Not in any tab. And then I got at least some input from SBFspot...
That is okay, do not worry if you do not receive errors :D
When i change the topic in the configuration file of SBFspot to what you suggest than there are no readings in Domoticz.
Okay that is correct. If I look to your screenshots. I see that it picks one value for each IDX in svalue.
No idea what it presents, but it are not the IDX's that SBFspot will use.
So this is okay again
So MQTT seems to work.
Correct. This is okay.
This is the part of the config-file about MQTT in SBFspot. I set the host to 192.168.1.18 and topic to sbfspot/data
Conclusion:

SBFspot will publish its data to a MQTT server (broker) with IP address 192.168.1.18 and the default port 1883 and the topic sbfspot/data.
I also see it will publish the data in JSON format. Perfect.
In MQTT Explorer there is a new topic with a stream of data on the broker which runs on my Synology.
If you look to the message in MQTT Explorer, you see the following:
IP address: 192.168.1.18. REMEMBER that IP Address.

You also see that Domoticz publishes on domoticz/out (not important for now) and that "something" publishes to domoticz/in (which writes a value of 16.1 to IDX 179. That looks very good, although I do not know, what it represents.

Then the internal message $SYS from the broker. This has to be there.
And then the important one:
Data published under sbfspot/data in JSON format.
Perfect, this means that your communication between SBFspot and the MQTT server (broker) is okay.
On the Raspberry (192.168.1.15) I made a new flow by making an import of your example.
Okay and try to REMEMBER the IP Adress of the Pi (192.168.1.15)
In Node-RED: The MQTT out node has as server localhost and Topic domoticz/in because Node-REED and Domoticz run both on this machine.
Okay, so you have Node Red and Domoticz running on the same device, with IP Address of 192.168.1.15, but your MQTT server runs on 192.168.1.18.
(Do you remember!)
Node Red has to publish its data to the MQTT server and not to localhost, even if both Node Red and Domoticz runs on the same device.
On localhost you do not have a MQTT server installed. So impossible to communicate with the non-existing server.

So change to server in the MQTT out node to 192.168.1.18 (the same as the SBFspot In node) and it will start to work. I am quite sure.
In Domoticz I made all virtual sensors
Be aware that different virtual sensors require different formats.

See:

https://www.domoticz.com/wiki/MQTT#Upda ... .2Fsensors

For the different formats of the sensors, you can have look at:
https://piandmore.wordpress.com/2019/02 ... -domoticz/ and
https://www.domoticz.com/wiki/Domoticz_ ... re_3_Phase
But there is no input in logfile in Domoticz (192.168.1.15) from Node-red...
That is the default behavior in the latest versions (from 13179), So nothing to worry. Search the forum,if you need it.
See: viewtopic.php?f=6&t=36171&p=274250&hilit=MQTT#p274250
Seems there is no data from SBFspot to Nodered..
There is, and if you want to see it, then you should connect a "Debug" node to the output of the MQTT input node (SBFspot In)

One warning:

Be careful with the data you send to domoticz/in on 192.168.1.18.
For the voltage and current I am sure it correct, but the other you should check the format.

Let me know.

Regards
Last edited by FireWizard on Wednesday 05 May 2021 21:15, edited 4 times in total.
User avatar
waltervl
Posts: 5396
Joined: Monday 28 January 2019 18:48
Target OS: Linux
Domoticz version: 2024.7
Location: NL
Contact:

Re: MQTT to virtual sensors

Post by waltervl »

FireWizard wrote: Wednesday 05 May 2021 18:49
But there is no input in logfile in Domoticz (192.168.1.15) from Node-red...
That is the default behavior in the latest versions (from 13179), So nothing to worry. Search to forum,if you need it.
See: viewtopic.php?f=6&t=36171&p=274250&hilit=MQTT#p274250
What do you think of this new default behavior? Can you share your thoughts in that topic?
Domoticz running on Udoo X86 (on Ubuntu)
Devices/plugins: ZigbeeforDomoticz (with Xiaomi, Ikea, Tuya devices), Nefit Easy, Midea Airco, Omnik Solar, Goodwe Solar
User avatar
FireWizard
Posts: 1770
Joined: Tuesday 25 December 2018 12:11
Target OS: Raspberry Pi / ODroid
Domoticz version: Beta
Location: Voorthuizen (NL)
Contact:

Re: MQTT to virtual sensors

Post by FireWizard »

Hi,@waltervl

Walter I suggest not to hijack this topic.
This is a principal discussion, we should have in another thread.

Regards
PieterS
Posts: 196
Joined: Wednesday 31 May 2017 16:06
Target OS: NAS (Synology & others)
Domoticz version: 2024.7
Location: NL
Contact:

Re: MQTT to virtual sensors

Post by PieterS »

@ Firewizard,
Thank you for your kind and patient explanation! As I try to learn about a new protocol and its behaviour you are very clear to me. I apreciate that very much.

As I may summarise is the way of the traffic: SBFspot -> Broker -> Node-RED -> Broker -> Domoticz....
I missed one step.
And had in mind that output of Node-RED after translation was ready for Domoticz. And could been send to that application if it is assigned to Topic domoticz/in in de MQTT out node..

But now we made some progress:
Image

Now I receive the traffic of SBFspot in Domoticz!!
But I guess some information is not in the proper format...

One reason is that I added some extra values to the export of SBFspot... so it mixed up.

This is what I see in the DEBUG-LOG:

Code: Select all

 2021-05-05 19:19:28.915 Debug: MQTT: MQTT: Topic: domoticz/in, Message: {"idx":175,"RSSI":7,"nvalue":0,"svalue":"21.1"}
2021-05-05 19:19:29.045 Debug: MQTT: MQTT: Topic: domoticz/in, Message: {"idx":179,"RSSI":7,"nvalue":0,"svalue":"20.9"}
2021-05-05 19:30:04.212 Debug: MQTT: MQTT: Topic: domoticz/in, Message: {"command":"udevice","idx":181,"nvalue":0,"svalue":"0.137"}
2021-05-05 19:30:04.318 Debug: MQTT: MQTT: Topic: domoticz/in, Message: {"command":"udevice","idx":182,"nvalue":0,"svalue":"0.138"}
2021-05-05 19:30:04.423 Debug: MQTT: MQTT: Topic: domoticz/in, Message: {"command":"udevice","idx":183,"nvalue":0,"svalue":"353.26"}
2021-05-05 19:30:04.529 Debug: MQTT: MQTT: Topic: domoticz/in, Message: {"command":"udevice","idx":184,"nvalue":0,"svalue":"352.79"} 
For your clarity:
There is another device (an ESP8266 with ESPeasy) who sends info for idx: 175 and idx 179 to the broker and Domoticz.

Next homework for me is to extend the Function-node in the flow...

By the way:
I run the latest beta of Domoticz and I changed the configfile of Domoticz to get a debug-tab.

One again: Very many thanks for your lessons! Now I understand a bit more how the traffic runs.
Another goal is to understand the right functions..

I let you know when all values can be read in the dashboard. :D
Synology with Domoticz build (V2024.7) in Docker
PieterS
Posts: 196
Joined: Wednesday 31 May 2017 16:06
Target OS: NAS (Synology & others)
Domoticz version: 2024.7
Location: NL
Contact:

Re: MQTT to virtual sensors

Post by PieterS »

@FireWizard: Thanks again for your step by step explanation. I succeeded in getting all the data of the MQTT-stream via Node-RED to Domoticz. :D Just create some more virtual sensors and rebuild the function-node in Node-RED.
Image

Only thing is to figure out why I lose some decimals in the exposed values in Domoticz... While the data is provided by Node-RED.. :roll:

Code: Select all

2021-05-06 13:00:04.064 Debug: MQTT: MQTT: Topic: domoticz/in, Message: {"command":"udevice","idx":201,"nvalue":0,"svalue":"45392.918"}
2021-05-06 13:00:04.176 Debug: MQTT: MQTT: Topic: domoticz/in, Message: {"command":"udevice","idx":206,"nvalue":0,"svalue":"Ok"}
2021-05-06 13:00:04.285 Debug: MQTT: MQTT: Topic: domoticz/in, Message: {"command":"udevice","idx":195,"nvalue":0,"svalue":"36.91"}
2021-05-06 13:00:04.392 Debug: MQTT: MQTT: Topic: domoticz/in, Message: {"command":"udevice","idx":207,"nvalue":0,"svalue":"Closed"}
2021-05-06 13:00:04.499 Debug: MQTT: MQTT: Topic: domoticz/in, Message: {"command":"udevice","idx":193,"nvalue":0,"svalue":"72.157"}
2021-05-06 13:00:04.606 Debug: MQTT: MQTT: Topic: domoticz/in, Message: {"command":"udevice","idx":202,"nvalue":0,"svalue":"49.96"}
2021-05-06 13:00:04.714 Debug: MQTT: MQTT: Topic: domoticz/in, Message: {"command":"udevice","idx":205,"nvalue":0,"svalue":"6.642"}
2021-05-06 13:00:04.820 Debug: MQTT: MQTT: Topic: domoticz/in, Message: {"command":"udevice","idx":190,"nvalue":0,"svalue":"40301.134"}
2021-05-06 13:00:04.927 Debug: MQTT: MQTT: Topic: domoticz/in, Message: {"command":"udevice","idx":189,"nvalue":0,"svalue":"3194"}
2021-05-06 13:00:05.035 Debug: MQTT: MQTT: Topic: domoticz/in, Message: {"command":"udevice","idx":204,"nvalue":0,"svalue":"3301"}
2021-05-06 13:00:05.143 Debug: MQTT: MQTT: Topic: domoticz/in, Message: {"command":"udevice","idx":187,"nvalue":0,"svalue":"43912.751"}
2021-05-06 13:00:05.249 Debug: MQTT: MQTT: Topic: domoticz/in, Message: {"command":"udevice","idx":198,"nvalue":0,"svalue":"3194"}
2021-05-06 13:00:05.355 Debug: MQTT: MQTT: Topic: domoticz/in, Message: {"command":"udevice","idx":186,"nvalue":0,"svalue":"247.93"}
2021-05-06 13:00:05.462 Debug: MQTT: MQTT: Topic: domoticz/in, Message: {"command":"udevice","idx":199,"nvalue":0,"svalue":"12.882"}
2021-05-06 13:00:05.568 Debug: MQTT: MQTT: Topic: domoticz/in, Message: {"command":"udevice","idx":184,"nvalue":0,"svalue":"369.13"}
2021-05-06 13:00:05.674 Debug: MQTT: MQTT: Topic: domoticz/in, Message: {"command":"udevice","idx":185,"nvalue":0,"svalue":"368.02"}
2021-05-06 13:00:05.781 Debug: MQTT: MQTT: Topic: domoticz/in, Message: {"command":"udevice","idx":180,"nvalue":0,"svalue":"4.477"}
2021-05-06 13:00:05.887 Debug: MQTT: MQTT: Topic: domoticz/in, Message: {"command":"udevice","idx":181,"nvalue":0,"svalue":"4.482"}
2021-05-06 13:00:05.993 Debug: MQTT: MQTT: Topic: domoticz/in, Message: {"command":"udevice","idx":182,"nvalue":0,"svalue":"1652"}
2021-05-06 13:00:06.100 Debug: MQTT: MQTT: Topic: domoticz/in, Message: {"command":"udevice","idx":183,"nvalue":0,"svalue":"1649"} 
But that is maybe off topic?
In https://www.domoticz.com/wiki/Domoticz_API/JSON_URL%27s I read about the Smart meter:
For USAGE and RETURN supply the data in total Wh with no decimal point.
(So if your meter displays f.i. USAGE1= 523,66 KWh you need to send 523660).
So I have to have to multiply by 1000 in Node-RED I guess to solve that problem?
As you mentioned https://piandmore.wordpress.com/2019/02 ... -domoticz/
Alert

The alert sensor will show you a message and an alert level.

{"command":"udevice", "idx":1234, "svalue":"any text", "nvalue":lvl}

where level can be any of the following:

0 off, grey
1 ok, green
2 yellow, level 1 warning
3 orange, level 2 warning
4 red, level 3 warning
Another goal is to use in Domoticz an Alert sensor instead of a textsensor for IDX 206 an IDX 207 and rearrange the "nvalue" so that I get an alert level.
I see new cases for the near future.

Thanks again!
Pieter
Synology with Domoticz build (V2024.7) in Docker
User avatar
FireWizard
Posts: 1770
Joined: Tuesday 25 December 2018 12:11
Target OS: Raspberry Pi / ODroid
Domoticz version: Beta
Location: Voorthuizen (NL)
Contact:

Re: MQTT to virtual sensors

Post by FireWizard »

Hello Pieter, @PieterS,
Thanks again for your step by step explanation. I succeeded in getting all the data of the MQTT-stream via Node-RED to Domoticz. :D Just create some more virtual sensors and rebuild the function-node in Node-RED.
Looks nice.
Only thing is to figure out why I lose some decimals in the exposed values in Domoticz... While the data is provided by Node-RED..
Not all sensors accept decimal digits. So this is some try and error, I'm afraid.
So I have to have to multiply by 1000 in Node-RED I guess to solve that problem?
Yes, you have to, e.g if the value for EToday is 2.566 kWh, you have to send as svalue: (msg.payload.EToday *1000).toString().
Another goal is to use in Domoticz an Alert sensor instead of a textsensor for IDX 206 an IDX 207 and rearrange the "nvalue" so that I get an alert level.
You can change the "Text" sensor to an "Alert" sensor. This sensor will show a colored triangle with the desired text.

You have to change the command in the "Function" node.

In case of a Text node", you have something like:

Code: Select all

msg5.payload = {"command":"udevice","idx":206,"nvalue":0,"svalue":msg.payload.InvStatus};
Let us assume, that in case the Inver Status is OK, you want to see a green triangle and in case, it is not OK, you want to see a red triangle.

Let us assume that this is msg5. In this case you have to change the previous command to:

Code: Select all

msg1 = {};
msg2 = {};
msg3 = {};
msg4 = {};
msg5 = {};

msg1.payload = {"command":"udevice","idx":181,"nvalue":0,"svalue":msg.payload.IDC1.toString()}; 
msg2.payload = {"command":"udevice","idx":182,"nvalue":0,"svalue":msg.payload.IDC2.toString()};
msg3.payload = {"command":"udevice","idx":183,"nvalue":0,"svalue":msg.payload.UDC1.toString()};
msg4.payload = {"command":"udevice","idx":184,"nvalue":0,"svalue":msg.payload.UDC2.toString()};

if (msg.payload.InvStatus == "Ok") {
    msg5.payload = {"command":"udevice","idx":206,"nvalue":1,"svalue":msg.payload.InvStatus}
} else {
    msg5.payload = {"command":"udevice","idx":206,"nvalue":4,"svalue":msg.payload.InvStatus}
}

return [[msg1,msg2,msg3,msg4,msg5]];
Regards
PieterS
Posts: 196
Joined: Wednesday 31 May 2017 16:06
Target OS: NAS (Synology & others)
Domoticz version: 2024.7
Location: NL
Contact:

Re: MQTT to virtual sensors

Post by PieterS »

H FireWizard,

Thanks again for your information. Learned now about programming and calculating with a value in the function-node.

And how to comment with //

All works like a charm and I am satisfied for now with this flow.

One last question:

How did you use (import) my data of my first post in your example of the flow you made in your first reply? :?:

I mention the screenshots of your exemple of the flow? In the first one I see a node called msg.payload but I cannot add there my data..

My solution to test was edit the timer in the crontab to 1 minute. That works only on daytime, not in the evening as the solar-inverter is asleep.
Or, as I write this text: You send something from the broker. But even then the question: How??

Pieter
Synology with Domoticz build (V2024.7) in Docker
User avatar
FireWizard
Posts: 1770
Joined: Tuesday 25 December 2018 12:11
Target OS: Raspberry Pi / ODroid
Domoticz version: Beta
Location: Voorthuizen (NL)
Contact:

Re: MQTT to virtual sensors

Post by FireWizard »

Hello Pieter, @PieterS

You asked:

One last question:

How did you use (import) my data of my first post in your example of the flow you made in your first reply? :?:

I mention the screenshots of your example of the flow? In the first one I see a node called msg.payload but I cannot add there my data..
There is something that I didn't show, as it was only helpful for myself, to simulate your received MQTT message.

Screenshot_SBFspot3.png
Screenshot_SBFspot3.png (6.29 KiB) Viewed 8175 times
In your first post (04 May 2021, 13:02), you posted the message, you received from your SBFspot.
You said:
SBFspot dumps every 5 minutes its data to the broker and Domoticz/in tells me in the log this information:
The Message between { } is the received JSON data.

As already explained, domoticz/in is not the correct topic, but the Message is very helpful.
I copied and pasted that message in the "Inject" node as a JSON message.

Screenshot_SBFspot4.png
Screenshot_SBFspot4.png (33.36 KiB) Viewed 8175 times
This message, I send to my MQTT server (broker), by the MQTT out node, called SBFspot Out, with the topic I suggested (sbfspot/data) and this will be received by the MQTT In node, with the same topic.

Screenshot_SBFspot5.png
Screenshot_SBFspot5.png (21.55 KiB) Viewed 8175 times
Well, and if you connect a "Debug" node to the MQTT Input node, you will see the result, as I showed you in one of the screenshots.

Of course you can configure to "Inject" node to send the data every x seconds, but that is not very practical.
So I use the manual inject method (the small square at the left of the "Inject" node.
Of course the data is always the same, but to try things, that does not matter.

Very helpful, if you want to try things.

Regards
PieterS
Posts: 196
Joined: Wednesday 31 May 2017 16:06
Target OS: NAS (Synology & others)
Domoticz version: 2024.7
Location: NL
Contact:

Re: MQTT to virtual sensors

Post by PieterS »

H FireWizard,

Thanks again for your fabulous explanation.

I made a new flow with Inject and MQTT-nodes. After a little struggle (What belongs to a message), so in between the { }, you wrote it :oops:

Good help in the application:
Nice to see the triangle at the Inject-node when there is an error in the syntax :shock:

Finaly I got some output in the debug-screen: TypeError: Cannot read property 'toString' of undefined

Error which was related to old JSON what you picked up from my first post... Caused because I changed the MQTT-message in SBFspot.cfg and not useable for the conversionrules in the function-node... :cry:

So I used MQTT Explorer to get a new message for testing my second flow. :D And now I had success in the debug-sidebar!!!
Very happy with your lessons!! I can handle this for now with some trial and error.

I will better follow your input on the forum in the future.

Yours sincerely
Pieter
Synology with Domoticz build (V2024.7) in Docker
Post Reply

Who is online

Users browsing this forum: No registered users and 0 guests