Join several dataloggers and calculate values in Node-Red Topic is solved

Moderator: leecollings

PieterS
Posts: 196
Joined: Wednesday 31 May 2017 16:06
Target OS: NAS (Synology & others)
Domoticz version: 2024.7
Location: NL
Contact:

Join several dataloggers and calculate values in Node-Red

Post by PieterS »

I use a Node-red flow to upload data from a solar-inverter to a Influxdatabase and to Domoticz and PVoutput. Works like a charm.

Image

But I have 3 inverters. For this case I keep it simple: they are from same brand and give every 5 minutes all the same sort of JSON-output.

Code: Select all

{"device": "FPH1B0001", "time": "2022-03-25T12:49:27", "buffered": "no", "values": {"datalogserial": "XGD2AAAAAA", "pvserial": "FPH1B0001", "pvstatus": 1, "pvpowerin": 15401, "pv1voltage": 2286, "pv1current": 69, "pv1watt": 15401, "pv2voltage": 0, "pv2current": 0, "pv2watt": 0, "pvpowerout": 15093, "pvfrequentie": 5000, "pvgridvoltage": 2334, "pvgridcurrent": 64, "pvgridpower": 15120, "pvgridvoltage2": 0, "pvgridcurrent2": 0, "pvgridpower2": 0, "pvgridvoltage3": 0, "pvgridcurrent3": 0, "pvgridpower3": 0, "totworktime": 17713903, "pvenergytoday": 56, "pvenergytotal": 7419, "epvtotal": 7507, "epv1today": 56, "epv1total": 7507, "epv2today": 0, "epv2total": 0, "pvtemperature": 458, "pvipmtemperature": 479}}
Of course the devicenumber, datalogserial and pvserial and the values will be different.

Next goal is to combine some values to totalize the values of these inverters: pvpowerout of each inverter and pvenergytoday of each inverter. I want to send these new values to Domoticz and PVoutput.

This is the flow I got up till now.

Code: Select all

[{"id":"85b1d997.d9083","type":"tab","label":"Meerdere inverters naar Pvoutput.org","disabled":false,"info":""},{"id":"7de2dcb9.8ebacc","type":"mqtt in","z":"85b1d997.d9083","name":"Growatt Energie","topic":"energy/growatt","qos":"2","datatype":"auto","broker":"7fb3e06.2d91f2","nl":false,"rap":false,"x":200,"y":320,"wires":[["23d9d2cd.5f108e"]]},{"id":"23d9d2cd.5f108e","type":"json","z":"85b1d997.d9083","name":"","property":"payload","action":"","pretty":false,"x":410,"y":320,"wires":[["84a7ea50.b4f3b8"]]},{"id":"f09186.d82ed678","type":"http request","z":"85b1d997.d9083","name":"PVOUTPUT.ORG","method":"POST","ret":"txt","paytoqs":"ignore","url":"https://pvoutput.org/service/r2/addstatus.jsp","tls":"","persist":false,"proxy":"","authType":"","x":1030,"y":320,"wires":[["bfd2a50a.c77f9"]]},{"id":"84a7ea50.b4f3b8","type":"function","z":"85b1d997.d9083","name":"Create PV Message for Solar PVOUTPUT ","func":"//Create PVOUTPUT Message ;\nmsg.headers={};\n//Create time\nvar date = new Date();\nday = date.getDate();\nif (day < 10) {day = \"0\" + day} \nmonth = date.getMonth()+1;\nif (month < 10) {month = \"0\" + month}\nyear = date.getFullYear();\n//node.warn(year);\nhour = date.getHours();\nif (hour < 10) {hour = \"0\" + hour}\nminute = date.getMinutes();\nif (minute < 10) {minute = \"0\" + minute}\n//Create Headers\nmsg.headers[\"X-Pvoutput-Apikey\"] = \"your api key\";\nmsg.headers[\"X-Pvoutput-SystemId\"] = 12345          //\"your system id\";\nmsg.headers[\"content-type\"]=\"application/x-www-form-urlencoded\"\n//Create Payload ]\nd = \"\"+year+month+day;\nt = hour+\":\"+minute; \n//Growatt publish pvenergytoday in 0.1 KW, PVOUTPUT.ORG expext watt hours.\nv1 = msg.payload.values.pvenergytoday*100\nv2 = msg.payload.values.pvpowerout/10\nmsg.payload = \"&d=\"+d+\"&t=\"+t+\"&v1=\"+v1+\"&v2=\"+v2;\n//optional you can also sent other value to pvoutput.org \n//This can be done in a second flow also\n//the flows will be combined by PVOUTPUT.ORG\n//you have to retrieve value from (smart) energy  meter and outside temperature indicator\n//v3 = 12345     //energy used today (watts)\n//v4 = 1234.5    //energy used currently (watts)\n//v5 = 12.3      //temperature  \n//v6 = 123.4     //voltage\n//msg.payload = \"&d=\"+d+\"&t=\"+t+\"&v3=\"+v3+\"&v4=\"+v4+\"&v5=\"+v5+\"&v6=\"+v6;\nreturn msg;\n","outputs":1,"noerr":0,"initialize":"","finalize":"","libs":[],"x":700,"y":320,"wires":[["f09186.d82ed678"]]},{"id":"bfd2a50a.c77f9","type":"debug","z":"85b1d997.d9083","name":"","active":false,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","statusVal":"","statusType":"auto","x":1270,"y":320,"wires":[]},{"id":"774cad2f.aa55b4","type":"join","z":"85b1d997.d9083","name":"","mode":"auto","build":"string","property":"payload","propertyType":"msg","key":"topic","joiner":"\\n","joinerType":"str","accumulate":"false","timeout":"","count":"","reduceRight":false,"x":850,"y":620,"wires":[["c2709313.ec5178"]]},{"id":"ca425d6e.70806","type":"mqtt in","z":"85b1d997.d9083","name":"Growatt inverter 1","topic":"energy/growatt","qos":"2","datatype":"auto","broker":"7fb3e06.2d91f2","nl":false,"rap":true,"rh":0,"x":200,"y":560,"wires":[["c6a67dc5.e10ea8"]]},{"id":"c6a67dc5.e10ea8","type":"json","z":"85b1d997.d9083","name":"","property":"payload","action":"","pretty":false,"x":390,"y":560,"wires":[["c2709313.ec5178"]]},{"id":"5879b181.af42b","type":"change","z":"85b1d997.d9083","name":"Extract pvpowerout","rules":[{"t":"set","p":"values","pt":"msg","to":"pvpowerout","tot":"str"},{"t":"set","p":"payload","pt":"msg","to":"pvenergytoday","tot":"str"}],"action":"","property":"","from":"","to":"","reg":false,"x":610,"y":500,"wires":[["c2709313.ec5178"]]},{"id":"c2709313.ec5178","type":"debug","z":"85b1d997.d9083","name":"","active":true,"tosidebar":true,"console":true,"tostatus":true,"complete":"payload","targetType":"msg","statusVal":"payload","statusType":"auto","x":1080,"y":560,"wires":[]},{"id":"97be43b3.4c7c58","type":"mqtt in","z":"85b1d997.d9083","name":"Growatt inverter 2","topic":"energy/growatt2","qos":"2","datatype":"auto","broker":"7fb3e06.2d91f2","nl":false,"rap":true,"rh":0,"x":190,"y":660,"wires":[["354027b2.54462"]]},{"id":"354027b2.54462","type":"json","z":"85b1d997.d9083","name":"","property":"payload","action":"","pretty":false,"x":390,"y":660,"wires":[["203102d6.fa6516"]]},{"id":"46cecca5.928434","type":"debug","z":"85b1d997.d9083","name":"","active":true,"tosidebar":true,"console":true,"tostatus":true,"complete":"payload","targetType":"msg","statusVal":"payload","statusType":"auto","x":1080,"y":660,"wires":[]},{"id":"203102d6.fa6516","type":"change","z":"85b1d997.d9083","name":"Extract pvpowerout","rules":[{"t":"set","p":"values","pt":"msg","to":"pvpowerout","tot":"str"},{"t":"set","p":"payload","pt":"msg","to":"pvenergytoday","tot":"str"}],"action":"","property":"","from":"","to":"","reg":false,"x":570,"y":660,"wires":[[]]},{"id":"7fb3e06.2d91f2","type":"mqtt-broker","name":"","broker":"192.168.1.18","port":"1883","clientid":"","usetls":false,"protocolVersion":"4","keepalive":"60","cleansession":true,"birthTopic":"","birthQos":"0","birthPayload":"","birthMsg":{},"closeTopic":"","closePayload":"","closeMsg":{},"willTopic":"","willQos":"0","willPayload":"","willMsg":{},"sessionExpiry":""}]
Can someone give me a hint or sample how to combine/join the 3 flows and its values to 1 flow so that I can calculate values and send them to PVoutput and Domoticz?

Thanks in advance
Last edited by PieterS on Friday 25 March 2022 15:45, edited 2 times in total.
Synology with Domoticz build (V2024.7) in Docker
User avatar
waltervl
Posts: 5369
Joined: Monday 28 January 2019 18:48
Target OS: Linux
Domoticz version: 2024.7
Location: NL
Contact:

Re: Join several dataloggers and calculate values to Domoticz

Post by waltervl »

There is a special node-red sub forum for if this question does not get response: viewforum.php?f=81
Domoticz running on Udoo X86 (on Ubuntu)
Devices/plugins: ZigbeeforDomoticz (with Xiaomi, Ikea, Tuya devices), Nefit Easy, Midea Airco, Omnik Solar, Goodwe Solar
PieterS
Posts: 196
Joined: Wednesday 31 May 2017 16:06
Target OS: NAS (Synology & others)
Domoticz version: 2024.7
Location: NL
Contact:

Re: Join several dataloggers and calculate values to Domoticz

Post by PieterS »

waltervl wrote: Friday 25 March 2022 14:46 There is a special node-red sub forum for if this question does not get response: viewforum.php?f=81
Sorry! Missed that subforum. If you are able to move it than it is oke for me. Didn't find the tool for it
Synology with Domoticz build (V2024.7) in Docker
User avatar
waltervl
Posts: 5369
Joined: Monday 28 January 2019 18:48
Target OS: Linux
Domoticz version: 2024.7
Location: NL
Contact:

Re: Join several dataloggers and calculate values to Domoticz

Post by waltervl »

I do not have the tools too. Perhaps you can add Node-red to the title to be clear.
Domoticz running on Udoo X86 (on Ubuntu)
Devices/plugins: ZigbeeforDomoticz (with Xiaomi, Ikea, Tuya devices), Nefit Easy, Midea Airco, Omnik Solar, Goodwe Solar
thomasbaetge
Posts: 153
Joined: Wednesday 02 October 2019 11:47
Target OS: Linux
Domoticz version: 2023.1
Location: DE / BY / LT
Contact:

Re: Join several dataloggers and calculate values in Node-Red

Post by thomasbaetge »

Hi,
I think what you are looking for is the 'join' node in manual mode (see the nodes documentation).
you can combine a given number of messages into one message and set each inverter as a separate topic.
So you end up with a single message that contains your data and you can simply sum it up in a subsequent function node and send it to influx.

I'm using the same with my thermo panel, where current power and output come in 2 messages and I feed it into one domoticz counter.

pretty easy.
PieterS
Posts: 196
Joined: Wednesday 31 May 2017 16:06
Target OS: NAS (Synology & others)
Domoticz version: 2024.7
Location: NL
Contact:

Re: Join several dataloggers and calculate values in Node-Red

Post by PieterS »

thomasbaetge wrote: Friday 25 March 2022 15:56 Hi,
I think what you are looking for is the 'join' node in manual mode (see the nodes documentation).
you can combine a given number of messages into one message and set each inverter as a separate topic.
So you end up with a single message that contains your data and you can simply sum it up in a subsequent function node and send it to influx.

I'm using the same with my thermo panel, where current power and output come in 2 messages and I feed it into one domoticz counter.

pretty easy.
Thanks for reply. That is what I had in mind: JOIN-function. But as a noob I studied and looked for selection of elements/values in a message and rename them so that I can do calculations..

This is what I got:
Image
a href picture

Your case is a bit different I think.. How did you select a value out of the JSON-message?
Synology with Domoticz build (V2024.7) in Docker
User avatar
FireWizard
Posts: 1762
Joined: Tuesday 25 December 2018 12:11
Target OS: Raspberry Pi / ODroid
Domoticz version: Beta
Location: Voorthuizen (NL)
Contact:

Re: Join several dataloggers and calculate values in Node-Red

Post by FireWizard »

Hello @PieterS,

Indeed, as @thomasbaetge also said, is to use the " Join" node in manual mode.

Every stream that flows into the input of that node should have a unique property to identify that stream. Often the msg.topic is used for that, but you can use any other property.

I believe your device (FPH1B0001) is unique, but probably datalogserial or pvserial as well.
Pvserial is equal to the device.

So use msg.payload.device or msg.payload.values.datalogserial as the key.

Another thing.

If you join two or more streams the required data from all streams should be available before the next node can do further processing e.g. adding values.
Therefore, as you have 3 streams to be joined, configure "After a number of message parts" to 3. And tick the box "and every subsequent message" as well.

However this works only reliable if there is no chance that one stream already receive a second message, before all 3 streams has received one message.

Let me know.

Regards
thomasbaetge
Posts: 153
Joined: Wednesday 02 October 2019 11:47
Target OS: Linux
Domoticz version: 2023.1
Location: DE / BY / LT
Contact:

Re: Join several dataloggers and calculate values in Node-Red

Post by thomasbaetge »

Firewizzard is right.
I've created a small example how to do it, you can import it and play around. it combines 3 messages with a number as payload and 'inverterX' as topic into one. the function node then sums the values up.

It's important what Firewizzard says, this will only work, if every inverter sends one message and all 3 are sent, before the first fires again.

Code: Select all

[
    {
        "id": "bb82a5502cbb188e",
        "type": "tab",
        "label": "Flow 11",
        "disabled": false,
        "info": "",
        "env": []
    },
    {
        "id": "99306548e30380c9",
        "type": "inject",
        "z": "bb82a5502cbb188e",
        "name": "",
        "props": [
            {
                "p": "payload"
            },
            {
                "p": "topic",
                "vt": "str"
            }
        ],
        "repeat": "",
        "crontab": "",
        "once": false,
        "onceDelay": 0.1,
        "topic": "inverter1",
        "payload": "10",
        "payloadType": "num",
        "x": 150,
        "y": 100,
        "wires": [
            [
                "adb0811afaae79a4"
            ]
        ]
    },
    {
        "id": "e939a5c9661e1246",
        "type": "inject",
        "z": "bb82a5502cbb188e",
        "name": "",
        "props": [
            {
                "p": "payload"
            },
            {
                "p": "topic",
                "vt": "str"
            }
        ],
        "repeat": "",
        "crontab": "",
        "once": false,
        "onceDelay": 0.1,
        "topic": "inverter3",
        "payload": "22",
        "payloadType": "num",
        "x": 150,
        "y": 220,
        "wires": [
            [
                "adb0811afaae79a4"
            ]
        ]
    },
    {
        "id": "61758daea823e8f7",
        "type": "inject",
        "z": "bb82a5502cbb188e",
        "name": "",
        "props": [
            {
                "p": "payload"
            },
            {
                "p": "topic",
                "vt": "str"
            }
        ],
        "repeat": "",
        "crontab": "",
        "once": false,
        "onceDelay": 0.1,
        "topic": "inverter2",
        "payload": "15",
        "payloadType": "num",
        "x": 150,
        "y": 160,
        "wires": [
            [
                "adb0811afaae79a4"
            ]
        ]
    },
    {
        "id": "adb0811afaae79a4",
        "type": "join",
        "z": "bb82a5502cbb188e",
        "name": "",
        "mode": "custom",
        "build": "object",
        "property": "payload",
        "propertyType": "msg",
        "key": "topic",
        "joiner": "\\n",
        "joinerType": "str",
        "accumulate": false,
        "timeout": "",
        "count": "3",
        "reduceRight": false,
        "reduceExp": "",
        "reduceInit": "",
        "reduceInitType": "",
        "reduceFixup": "",
        "x": 370,
        "y": 160,
        "wires": [
            [
                "98713ad111e98053",
                "505df18fa9fc11ab"
            ]
        ]
    },
    {
        "id": "98713ad111e98053",
        "type": "debug",
        "z": "bb82a5502cbb188e",
        "name": "",
        "active": true,
        "tosidebar": true,
        "console": false,
        "tostatus": false,
        "complete": "true",
        "targetType": "full",
        "statusVal": "",
        "statusType": "auto",
        "x": 530,
        "y": 160,
        "wires": []
    },
    {
        "id": "505df18fa9fc11ab",
        "type": "function",
        "z": "bb82a5502cbb188e",
        "name": "",
        "func": "msg.payload = msg.payload.inverter1 + msg.payload.inverter2 + msg.payload.inverter3\nreturn msg;",
        "outputs": 1,
        "noerr": 0,
        "initialize": "",
        "finalize": "",
        "libs": [],
        "x": 540,
        "y": 220,
        "wires": [
            [
                "e9edf15f347b49be"
            ]
        ]
    },
    {
        "id": "e9edf15f347b49be",
        "type": "debug",
        "z": "bb82a5502cbb188e",
        "name": "",
        "active": true,
        "tosidebar": true,
        "console": false,
        "tostatus": false,
        "complete": "false",
        "statusVal": "",
        "statusType": "auto",
        "x": 730,
        "y": 220,
        "wires": []
    }
]
PieterS
Posts: 196
Joined: Wednesday 31 May 2017 16:06
Target OS: NAS (Synology & others)
Domoticz version: 2024.7
Location: NL
Contact:

Re: Join several dataloggers and calculate values in Node-Red

Post by PieterS »

FireWizard wrote: Saturday 26 March 2022 14:35 Hello @PieterS,

Indeed, as @thomasbaetge also said, is to use the " Join" node in manual mode.

Every stream that flows into the input of that node should have a unique property to identify that stream. Often the msg.topic is used for that, but you can use any other property.

I believe your device (FPH1B0001) is unique, but probably datalogserial or pvserial as well.
Pvserial is equal to the device.

So use msg.payload.device or msg.payload.values.datalogserial as the key.

Regards
H FireWizard,

Thanks for your reply.
I have to start with telling that every inverter will have its own Topic in MQTT. So for example energy/growatt1 and energy/growatt2. That is because I have to run 3 seperate Docker containers. The software to read the data and send it to Influxdb is not (yet) multithreading, as far as I know. As discussed in https://github.com/johanmeijer/grott/discussions/112

So in my opinion: I don't have to filter on a devicenumber. Of course I need to fill in the right Topicname.

But after that: I do not understand how to pick up the right values (pvpowerout and pvenergytoday) out of the message and send them to a function node. Maybe you can explain that to me?

To make it probably more complex... The solarpanels of one of the inverters is at the back of the building. So it will start up later then the other two. I do not know yet how much delay there will be... In wintertime maybe half an hour or so.. Under normal conditions those 2 will start together (on the same roof and orientation) and all of them will send there message every 5 minutes when they are alive/running.

Looking forward to your answer.
Pieter
Synology with Domoticz build (V2024.7) in Docker
PieterS
Posts: 196
Joined: Wednesday 31 May 2017 16:06
Target OS: NAS (Synology & others)
Domoticz version: 2024.7
Location: NL
Contact:

Re: Join several dataloggers and calculate values in Node-Red

Post by PieterS »

thomasbaetge wrote: Saturday 26 March 2022 16:28 Firewizzard is right.
I've created a small example how to do it, you can import it and play around. it combines 3 messages with a number as payload and 'inverterX' as topic into one. the function node then sums the values up.

It's important what Firewizzard says, this will only work, if every inverter sends one message and all 3 are sent, before the first fires again.
Thanks thomasbaetge.

Your exampleflow is clear to me. Now I understand how to summarize three values. Need some programming at the end of the flow.

But there must be some work before... Filter the right values out of the message and give them a temporary name.

I will wait for the answer of my questions to Firewizard. Thanks for now.
Synology with Domoticz build (V2024.7) in Docker
User avatar
FireWizard
Posts: 1762
Joined: Tuesday 25 December 2018 12:11
Target OS: Raspberry Pi / ODroid
Domoticz version: Beta
Location: Voorthuizen (NL)
Contact:

Re: Join several dataloggers and calculate values in Node-Red

Post by FireWizard »

Hello @PieterS

You wrote:
I have to start with telling that every inverter will have its own Topic in MQTT. So for example energy/growatt1 and energy/growatt2.
Okay. This means that you can use the msg.topic as a key in the "Join" node.
I assume you will use "energy/growatt1", "energy/growatt2" and "energy/growatt3", as the different topics

Please find below an example, how to use a "Join" node, select a variable and use it in a "Function" node.

Screenshot_Join1.png
Screenshot_Join1.png (36.21 KiB) Viewed 3817 times

I created 3 "Inject" nodes with 3 different topics, as indicated above. Ikey inject a payload as you presented in your first post in this thread.

The 3 debug nodes (under the Inject nodes) will show that. It are the three messages at the top of the picture below.

The "Join" node merges these 3 different messages and the result you will find in the fourth message in the picture below.

If you hoover with your mouse over the item you wish to extract you will see some extra buttons at the right.
See also: https://nodered.org/docs/user-guide/messages
I use "pvpowerout", which is (in all 3 cases) 15093.

Form the "Join" node I go to a "Function" node and there I simply adds the 3 different pvpowerout values (which are in this case the same).
The result is just a number, as you see in the last message.

Screenshot_Join2.png
Screenshot_Join2.png (53.34 KiB) Viewed 3817 times
I suggest you play a little with it, as you also did it with the flow from Thomas.
It is the same flow, but now with your values (both msg.payload and your suggested msg.topics)

But there must be some work before... Filter the right values out of the message and give them a temporary name.
I do not know, what you mean, but in my opinion no work "before" is needed.

BUT.
To make it probably more complex... The solar-panels of one of the inverters is at the back of the building. So it will start up later then the other two. I do not know yet how much delay there will be... In wintertime maybe half an hour or so.. Under normal conditions those 2 will start together (on the same roof and orientation) and all of them will send there message every 5 minutes when they are alive/running.

As you have 3 streams running into a "Join" node, I hope that it is clear to you, that ALL data in ALL streams should be available before we can do a reliable calculation and to add the 3 different values.
If I inject the first "Inject" node 3 times the "Join" node cannot see the difference, compared with injecting 3 nodes 1 time.

I'm pretty sure, that that will happen and not only with the inverter at the back of the building. Also the other two may start at different times, due to component tolerances, shades, aging of the solar-panels, etc.

We have to find a solution for that. Therefore it is important to know what you receive, if the solar panels are not delivering power.
Do they send a value of "0" or do they not send anything at all? Or anything else?

This is something I need to know.
I assume that if the first starts the calculations has to start also, however the result will be the value of the first started inverter.
If the second comes in, we shall add 2 values and if the third jumps in, finally we have to add 3 values. Agree?

Let me know?

Regards
thomasbaetge
Posts: 153
Joined: Wednesday 02 October 2019 11:47
Target OS: Linux
Domoticz version: 2023.1
Location: DE / BY / LT
Contact:

Re: Join several dataloggers and calculate values in Node-Red

Post by thomasbaetge »

reading all this, maybe the join node is not the right solution.
It would only work, if you receive data from all inverters regularily and in in a more or less stable sequence.

Maybe a time based solution would be an option:
Your receive the mesages as they come in (regardless the source) and sum up the data in flow context variables. Then after a fixed time you submit data to influx and reset the variables.
This could be done collective or per single inverter.
I am using something similar for other purposes.

this way the sequence or timing of the messages from your inverter would not matter anymore.
User avatar
FireWizard
Posts: 1762
Joined: Tuesday 25 December 2018 12:11
Target OS: Raspberry Pi / ODroid
Domoticz version: Beta
Location: Voorthuizen (NL)
Contact:

Re: Join several dataloggers and calculate values in Node-Red

Post by FireWizard »

Hello @PieterS and @thomasbaetge,

@thomasbaetge,

You wrote:
reading all this, maybe the join node is not the right solution.
It would only work, if you receive data from all inverters regularily and in in a more or less stable sequence.

Maybe a time based solution would be an option:
Your receive the mesages as they come in (regardless the source) and sum up the data in flow context variables. Then after a fixed time you submit data to influx and reset the variables.
This was exactly my thoughts as well and therefore I asked:
Therefore it is important to know what you receive, if the solar panels are not delivering power.
Do they send a value of "0" or do they not send anything at all? Or anything else?
Then after a fixed time you submit data to influx and reset the variables.
It is important to know whether the inverters are continuing transmitting "0" in the night or stop to transmit at all.

I do not think that we should
reset the variables
If we do that and we update the domoticz sensor, before a new value has been received (so the value is 0) we will have a wrong calculation.
In my opinion it would be better to update the flow.context store by overwriting the old value. That is more accurate.
What do you think?

The only thing we should do, is to write 0 (reset) at midnight, in order to be sure that when production starts the value is 0.
This has not been implemented yet.
Also the energy has not been implemented, but the power works.

See the example flow below.

Screenshot_Growatt1.png
Screenshot_Growatt1.png (44.01 KiB) Viewed 3796 times

The first 3 "Inject" and "MQTT Out" nodes, you may forget, as they were needed to simulate the inverters.

The 3 "MQTT In" nodes receive the data from the Inverters and with a "Change" node the desired data is stored.

The last "Inject" node should fire, if the data should be send to Domoticz, e.g. every 5 or 15 minutes. That is up to Piet to decide.

I suggest you play a little with the flow and see, if this is what you want.
Which virtual Sensor do you use in Domoticz? Electric?

Code: Select all

[{"id":"328f573d7ea7bb89","type":"tab","label":"GroWatt","disabled":false,"info":"","env":[]},{"id":"8f8b00a16cdd4b74","type":"inject","z":"328f573d7ea7bb89","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"energy/growatt1","payload":"{\"device\": \"FPH1B0001\", \"time\": \"2022-03-25T12:49:27\", \"buffered\": \"no\", \"values\": {\"datalogserial\": \"XGD2AAAAAA\", \"pvserial\": \"FPH1B0001\", \"pvstatus\": 1, \"pvpowerin\": 15401, \"pv1voltage\": 2286, \"pv1current\": 69, \"pv1watt\": 15401, \"pv2voltage\": 0, \"pv2current\": 0, \"pv2watt\": 0, \"pvpowerout\": 15093, \"pvfrequentie\": 5000, \"pvgridvoltage\": 2334, \"pvgridcurrent\": 64, \"pvgridpower\": 15120, \"pvgridvoltage2\": 0, \"pvgridcurrent2\": 0, \"pvgridpower2\": 0, \"pvgridvoltage3\": 0, \"pvgridcurrent3\": 0, \"pvgridpower3\": 0, \"totworktime\": 17713903, \"pvenergytoday\": 56, \"pvenergytotal\": 7419, \"epvtotal\": 7507, \"epv1today\": 56, \"epv1total\": 7507, \"epv2today\": 0, \"epv2total\": 0, \"pvtemperature\": 458, \"pvipmtemperature\": 479}}","payloadType":"json","x":250,"y":60,"wires":[["d2cbb1c1fb15c5cf"]]},{"id":"d2cbb1c1fb15c5cf","type":"mqtt out","z":"328f573d7ea7bb89","name":"MQTT Out 1","topic":"energy/growatt1","qos":"0","retain":"false","respTopic":"","contentType":"","userProps":"","correl":"","expiry":"","broker":"f9f13036.e28b58","x":430,"y":60,"wires":[]},{"id":"2bceddb4bb0da304","type":"inject","z":"328f573d7ea7bb89","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"energy/growatt2","payload":"{\"device\": \"FPH1B0002\", \"time\": \"2022-03-25T12:49:27\", \"buffered\": \"no\", \"values\": {\"datalogserial\": \"XGD2AAAAAA\", \"pvserial\": \"FPH1B0002\", \"pvstatus\": 1, \"pvpowerin\": 15401, \"pv1voltage\": 2286, \"pv1current\": 69, \"pv1watt\": 15401, \"pv2voltage\": 0, \"pv2current\": 0, \"pv2watt\": 0, \"pvpowerout\": 15093, \"pvfrequentie\": 5000, \"pvgridvoltage\": 2334, \"pvgridcurrent\": 64, \"pvgridpower\": 15120, \"pvgridvoltage2\": 0, \"pvgridcurrent2\": 0, \"pvgridpower2\": 0, \"pvgridvoltage3\": 0, \"pvgridcurrent3\": 0, \"pvgridpower3\": 0, \"totworktime\": 17713903, \"pvenergytoday\": 56, \"pvenergytotal\": 7419, \"epvtotal\": 7507, \"epv1today\": 56, \"epv1total\": 7507, \"epv2today\": 0, \"epv2total\": 0, \"pvtemperature\": 458, \"pvipmtemperature\": 479}}","payloadType":"json","x":250,"y":120,"wires":[["3569a94ec7a98621"]]},{"id":"3569a94ec7a98621","type":"mqtt out","z":"328f573d7ea7bb89","name":"MQTT Out 2","topic":"energy/growatt2","qos":"0","retain":"false","respTopic":"","contentType":"","userProps":"","correl":"","expiry":"","broker":"f9f13036.e28b58","x":430,"y":120,"wires":[]},{"id":"0a78e441cee47090","type":"inject","z":"328f573d7ea7bb89","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"energy/growatt1","payload":"{\"device\": \"FPH1B0003\", \"time\": \"2022-03-25T12:49:27\", \"buffered\": \"no\", \"values\": {\"datalogserial\": \"XGD2AAAAAA\", \"pvserial\": \"FPH1B0003\", \"pvstatus\": 1, \"pvpowerin\": 15401, \"pv1voltage\": 2286, \"pv1current\": 69, \"pv1watt\": 15401, \"pv2voltage\": 0, \"pv2current\": 0, \"pv2watt\": 0, \"pvpowerout\": 15093, \"pvfrequentie\": 5000, \"pvgridvoltage\": 2334, \"pvgridcurrent\": 64, \"pvgridpower\": 15120, \"pvgridvoltage2\": 0, \"pvgridcurrent2\": 0, \"pvgridpower2\": 0, \"pvgridvoltage3\": 0, \"pvgridcurrent3\": 0, \"pvgridpower3\": 0, \"totworktime\": 17713903, \"pvenergytoday\": 56, \"pvenergytotal\": 7419, \"epvtotal\": 7507, \"epv1today\": 56, \"epv1total\": 7507, \"epv2today\": 0, \"epv2total\": 0, \"pvtemperature\": 458, \"pvipmtemperature\": 479}}","payloadType":"json","x":250,"y":180,"wires":[["c738888f27ae9eb1"]]},{"id":"c738888f27ae9eb1","type":"mqtt out","z":"328f573d7ea7bb89","name":"MQTT Out 3","topic":"energy/growatt3","qos":"0","retain":"false","respTopic":"","contentType":"","userProps":"","correl":"","expiry":"","broker":"f9f13036.e28b58","x":430,"y":180,"wires":[]},{"id":"f459f57f0c5746e9","type":"mqtt in","z":"328f573d7ea7bb89","name":"MQTT In 1","topic":"energy/growatt1","qos":"0","datatype":"json","broker":"f9f13036.e28b58","nl":false,"rap":true,"rh":0,"inputs":0,"x":240,"y":280,"wires":[["93a6265c9704e392"]]},{"id":"78923483c2aa04c6","type":"mqtt in","z":"328f573d7ea7bb89","name":"MQTT In 2","topic":"energy/growatt2","qos":"2","datatype":"json","broker":"f9f13036.e28b58","nl":false,"rap":true,"rh":0,"inputs":0,"x":240,"y":340,"wires":[["fc15b7c977b0244f"]]},{"id":"757f32fc9dcba9fc","type":"mqtt in","z":"328f573d7ea7bb89","name":"MQTT In 3","topic":"energy/growatt3","qos":"2","datatype":"json","broker":"f9f13036.e28b58","nl":false,"rap":true,"rh":0,"inputs":0,"x":240,"y":400,"wires":[["7c2c907eac473fde"]]},{"id":"93a6265c9704e392","type":"change","z":"328f573d7ea7bb89","name":"Store Data 1","rules":[{"t":"set","p":"pvpowerout1","pt":"flow","to":"payload.values.pvpowerout","tot":"msg"},{"t":"set","p":"pvenergytoday1","pt":"flow","to":"payload.values.pvenergytoday","tot":"msg"}],"action":"","property":"","from":"","to":"","reg":false,"x":430,"y":280,"wires":[[]]},{"id":"fc15b7c977b0244f","type":"change","z":"328f573d7ea7bb89","name":"Store Data 2","rules":[{"t":"set","p":"pvpowerout2","pt":"flow","to":"payload.values.pvpowerout","tot":"msg"},{"t":"set","p":"pvenergytoday2","pt":"flow","to":"payload.values.pvenergytoday","tot":"msg"}],"action":"","property":"","from":"","to":"","reg":false,"x":430,"y":340,"wires":[[]]},{"id":"7c2c907eac473fde","type":"change","z":"328f573d7ea7bb89","name":"Store Data 3","rules":[{"t":"set","p":"pvpowerout3","pt":"flow","to":"payload.values.pvpowerout","tot":"msg"},{"t":"set","p":"pvenergytoday3","pt":"flow","to":"payload.values.pvenergytoday","tot":"msg"}],"action":"","property":"","from":"","to":"","reg":false,"x":430,"y":400,"wires":[[]]},{"id":"85bda728286247c9","type":"inject","z":"328f573d7ea7bb89","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"","payloadType":"date","x":240,"y":500,"wires":[["a6399e43a4a46139"]]},{"id":"a6399e43a4a46139","type":"function","z":"328f573d7ea7bb89","name":"Function","func":"msg.payload = flow.get(\"pvpowerout1\") + flow.get(\"pvpowerout2\") + flow.get(\"pvpowerout3\");\nreturn msg;","outputs":1,"noerr":0,"initialize":"","finalize":"","libs":[],"x":420,"y":500,"wires":[["8d0bce4ab027b9f6"]]},{"id":"8d0bce4ab027b9f6","type":"debug","z":"328f573d7ea7bb89","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","statusVal":"","statusType":"auto","x":610,"y":500,"wires":[]},{"id":"f9f13036.e28b58","type":"mqtt-broker","name":"localhost","broker":"127.0.0.1","port":"1883","clientid":"","autoConnect":true,"usetls":false,"protocolVersion":"4","keepalive":"60","cleansession":true,"birthTopic":"","birthQos":"0","birthRetain":"false","birthPayload":"","birthMsg":{},"closeTopic":"","closeQos":"0","closePayload":"","closeMsg":{},"willTopic":"","willQos":"0","willPayload":"","willMsg":{},"sessionExpiry":""}]
Regards
PieterS
Posts: 196
Joined: Wednesday 31 May 2017 16:06
Target OS: NAS (Synology & others)
Domoticz version: 2024.7
Location: NL
Contact:

Re: Join several dataloggers and calculate values in Node-Red

Post by PieterS »

Thanks @Firewizard and @thomasbaetge for your replies. Was not able to study and play with your information up till now. :oops: Will do that tomorrow.
Synology with Domoticz build (V2024.7) in Docker
PieterS
Posts: 196
Joined: Wednesday 31 May 2017 16:06
Target OS: NAS (Synology & others)
Domoticz version: 2024.7
Location: NL
Contact:

Re: Join several dataloggers and calculate values in Node-Red

Post by PieterS »

Thanks gentlemen for your attention and input!

Lots of information and issues to think about.
this way the sequence or timing of the messages from your inverter would not matter anymore.
About the start of the sequence from my inverters at home this morning:
The first inverter starts to send the first message at 7:50 (eastsite), the second at 8:02 (south) and the third (south) at 8:05... That same problem will be in the evening. When there is no production the loggers are asleep and send no data at all.

So I think we have to use a time based solution. I guess that is the second option from @Firewizard? With a inject node (with timestamp) related to sunrise and sunset with an interval of 5 minutes? No idea if these options are possible..

For me, this has to be clear and an agreement of the right strategy first. Otherwise I lose contact with you guys...

Another question was:
Which virtual Sensor do you use in Domoticz? Electric?
As I started this topic I had in mind to build a flow as solution for my brother: he will receive his 3 Growatt inverters at the end of this week. I will implement this flow also (a bit different) for my own situation. I have a SMA, Omnik and Growatt... I suggest to build my flow first. ;)
For the Growatt I own:
Image
and for the SMA I own:
Image
For the value pvenergytoday of the SMA:
Image
For the value pvenergytoday of the Growatt:
Image

I notice that I use different sort of sensors for the same values.. I will change that to make it not to complicated. :roll:

Hopefully enough info to make a step in the right direction? Thanks a lot!

Pieter
Synology with Domoticz build (V2024.7) in Docker
User avatar
FireWizard
Posts: 1762
Joined: Tuesday 25 December 2018 12:11
Target OS: Raspberry Pi / ODroid
Domoticz version: Beta
Location: Voorthuizen (NL)
Contact:

Re: Join several dataloggers and calculate values in Node-Red

Post by FireWizard »

Hello @PieterS,

You asked:
With a inject node (with timestamp) related to sunrise and sunset with an interval of 5 minutes? No idea if these options are possible..
Yes, this is possible and in fact it is quite easy.

I use a simple time switch node, but you have to install this yourself via the palette as it is not part of the NR core nodes, search for
node-red-contrib-time-switch.

Screenshot_NR Simple_Time_Switch.png
Screenshot_NR Simple_Time_Switch.png (13.21 KiB) Viewed 3772 times

As you can see today the switch is "open" on the upper output from 6:50 until 20:36.

Domoticz indicates today sunrise at 7:19 and sunset 20:06 at my co-ordinates.
You have to configure that yourself.

Screenshot_NR_Simple_Time_Switch_Config.png
Screenshot_NR_Simple_Time_Switch_Config.png (23.46 KiB) Viewed 3772 times
When there is no production the loggers are asleep and send no data at all.
Okay, and this means we have to reset the flow.context variables at, let's say, midnight. So at 00:00 we write a 0 (zero) value to the stores.
Or after sunset we have to reset (write 0). But that is up to you.
The latest value is visible until you reset. Then you will see 0.

Find below the modified flow.

Code: Select all

[{"id":"328f573d7ea7bb89","type":"tab","label":"GroWatt","disabled":false,"info":"","env":[]},{"id":"8f8b00a16cdd4b74","type":"inject","z":"328f573d7ea7bb89","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"energy/growatt1","payload":"{\"device\": \"FPH1B0001\", \"time\": \"2022-03-25T12:49:27\", \"buffered\": \"no\", \"values\": {\"datalogserial\": \"XGD2AAAAAA\", \"pvserial\": \"FPH1B0001\", \"pvstatus\": 1, \"pvpowerin\": 15401, \"pv1voltage\": 2286, \"pv1current\": 69, \"pv1watt\": 15401, \"pv2voltage\": 0, \"pv2current\": 0, \"pv2watt\": 0, \"pvpowerout\": 15093, \"pvfrequentie\": 5000, \"pvgridvoltage\": 2334, \"pvgridcurrent\": 64, \"pvgridpower\": 15120, \"pvgridvoltage2\": 0, \"pvgridcurrent2\": 0, \"pvgridpower2\": 0, \"pvgridvoltage3\": 0, \"pvgridcurrent3\": 0, \"pvgridpower3\": 0, \"totworktime\": 17713903, \"pvenergytoday\": 56, \"pvenergytotal\": 7419, \"epvtotal\": 7507, \"epv1today\": 56, \"epv1total\": 7507, \"epv2today\": 0, \"epv2total\": 0, \"pvtemperature\": 458, \"pvipmtemperature\": 479}}","payloadType":"json","x":250,"y":60,"wires":[["d2cbb1c1fb15c5cf"]]},{"id":"d2cbb1c1fb15c5cf","type":"mqtt out","z":"328f573d7ea7bb89","name":"MQTT Out 1","topic":"energy/growatt1","qos":"0","retain":"false","respTopic":"","contentType":"","userProps":"","correl":"","expiry":"","broker":"f9f13036.e28b58","x":430,"y":60,"wires":[]},{"id":"2bceddb4bb0da304","type":"inject","z":"328f573d7ea7bb89","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"energy/growatt2","payload":"{\"device\": \"FPH1B0002\", \"time\": \"2022-03-25T12:49:27\", \"buffered\": \"no\", \"values\": {\"datalogserial\": \"XGD2AAAAAA\", \"pvserial\": \"FPH1B0002\", \"pvstatus\": 1, \"pvpowerin\": 15401, \"pv1voltage\": 2286, \"pv1current\": 69, \"pv1watt\": 15401, \"pv2voltage\": 0, \"pv2current\": 0, \"pv2watt\": 0, \"pvpowerout\": 15093, \"pvfrequentie\": 5000, \"pvgridvoltage\": 2334, \"pvgridcurrent\": 64, \"pvgridpower\": 15120, \"pvgridvoltage2\": 0, \"pvgridcurrent2\": 0, \"pvgridpower2\": 0, \"pvgridvoltage3\": 0, \"pvgridcurrent3\": 0, \"pvgridpower3\": 0, \"totworktime\": 17713903, \"pvenergytoday\": 56, \"pvenergytotal\": 7419, \"epvtotal\": 7507, \"epv1today\": 56, \"epv1total\": 7507, \"epv2today\": 0, \"epv2total\": 0, \"pvtemperature\": 458, \"pvipmtemperature\": 479}}","payloadType":"json","x":250,"y":120,"wires":[["3569a94ec7a98621"]]},{"id":"3569a94ec7a98621","type":"mqtt out","z":"328f573d7ea7bb89","name":"MQTT Out 2","topic":"energy/growatt2","qos":"0","retain":"false","respTopic":"","contentType":"","userProps":"","correl":"","expiry":"","broker":"f9f13036.e28b58","x":430,"y":120,"wires":[]},{"id":"0a78e441cee47090","type":"inject","z":"328f573d7ea7bb89","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"energy/growatt1","payload":"{\"device\": \"FPH1B0003\", \"time\": \"2022-03-25T12:49:27\", \"buffered\": \"no\", \"values\": {\"datalogserial\": \"XGD2AAAAAA\", \"pvserial\": \"FPH1B0003\", \"pvstatus\": 1, \"pvpowerin\": 15401, \"pv1voltage\": 2286, \"pv1current\": 69, \"pv1watt\": 15401, \"pv2voltage\": 0, \"pv2current\": 0, \"pv2watt\": 0, \"pvpowerout\": 15093, \"pvfrequentie\": 5000, \"pvgridvoltage\": 2334, \"pvgridcurrent\": 64, \"pvgridpower\": 15120, \"pvgridvoltage2\": 0, \"pvgridcurrent2\": 0, \"pvgridpower2\": 0, \"pvgridvoltage3\": 0, \"pvgridcurrent3\": 0, \"pvgridpower3\": 0, \"totworktime\": 17713903, \"pvenergytoday\": 56, \"pvenergytotal\": 7419, \"epvtotal\": 7507, \"epv1today\": 56, \"epv1total\": 7507, \"epv2today\": 0, \"epv2total\": 0, \"pvtemperature\": 458, \"pvipmtemperature\": 479}}","payloadType":"json","x":250,"y":180,"wires":[["c738888f27ae9eb1"]]},{"id":"c738888f27ae9eb1","type":"mqtt out","z":"328f573d7ea7bb89","name":"MQTT Out 3","topic":"energy/growatt3","qos":"0","retain":"false","respTopic":"","contentType":"","userProps":"","correl":"","expiry":"","broker":"f9f13036.e28b58","x":430,"y":180,"wires":[]},{"id":"f459f57f0c5746e9","type":"mqtt in","z":"328f573d7ea7bb89","name":"MQTT In 1","topic":"energy/growatt1","qos":"0","datatype":"json","broker":"f9f13036.e28b58","nl":false,"rap":true,"rh":0,"inputs":0,"x":240,"y":280,"wires":[["93a6265c9704e392"]]},{"id":"78923483c2aa04c6","type":"mqtt in","z":"328f573d7ea7bb89","name":"MQTT In 2","topic":"energy/growatt2","qos":"2","datatype":"json","broker":"f9f13036.e28b58","nl":false,"rap":true,"rh":0,"inputs":0,"x":240,"y":420,"wires":[["fc15b7c977b0244f"]]},{"id":"757f32fc9dcba9fc","type":"mqtt in","z":"328f573d7ea7bb89","name":"MQTT In 3","topic":"energy/growatt3","qos":"2","datatype":"json","broker":"f9f13036.e28b58","nl":false,"rap":true,"rh":0,"inputs":0,"x":240,"y":560,"wires":[["7c2c907eac473fde"]]},{"id":"93a6265c9704e392","type":"change","z":"328f573d7ea7bb89","name":"Store Data 1","rules":[{"t":"set","p":"pvpowerout1","pt":"flow","to":"payload.values.pvpowerout","tot":"msg"},{"t":"set","p":"pvenergytoday1","pt":"flow","to":"payload.values.pvenergytoday","tot":"msg"}],"action":"","property":"","from":"","to":"","reg":false,"x":430,"y":320,"wires":[[]]},{"id":"fc15b7c977b0244f","type":"change","z":"328f573d7ea7bb89","name":"Store Data 2","rules":[{"t":"set","p":"pvpowerout2","pt":"flow","to":"payload.values.pvpowerout","tot":"msg"},{"t":"set","p":"pvenergytoday2","pt":"flow","to":"payload.values.pvenergytoday","tot":"msg"}],"action":"","property":"","from":"","to":"","reg":false,"x":430,"y":460,"wires":[[]]},{"id":"7c2c907eac473fde","type":"change","z":"328f573d7ea7bb89","name":"Store Data 3","rules":[{"t":"set","p":"pvpowerout3","pt":"flow","to":"payload.values.pvpowerout","tot":"msg"},{"t":"set","p":"pvenergytoday3","pt":"flow","to":"payload.values.pvenergytoday","tot":"msg"}],"action":"","property":"","from":"","to":"","reg":false,"x":430,"y":600,"wires":[[]]},{"id":"85bda728286247c9","type":"inject","z":"328f573d7ea7bb89","name":"5 Min Tick","props":[{"p":"payload"}],"repeat":"300","crontab":"","once":true,"onceDelay":0.1,"topic":"","payload":"","payloadType":"date","x":250,"y":760,"wires":[["16dc6bc6e7313a27"]]},{"id":"a6399e43a4a46139","type":"function","z":"328f573d7ea7bb89","name":"Function","func":"msg.payload = flow.get(\"pvpowerout1\") + flow.get(\"pvpowerout2\") + flow.get(\"pvpowerout3\");\nreturn msg;","outputs":1,"noerr":0,"initialize":"","finalize":"","libs":[],"x":640,"y":760,"wires":[["8d0bce4ab027b9f6"]]},{"id":"8d0bce4ab027b9f6","type":"debug","z":"328f573d7ea7bb89","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","statusVal":"","statusType":"auto","x":830,"y":760,"wires":[]},{"id":"16dc6bc6e7313a27","type":"time-switch","z":"328f573d7ea7bb89","name":"Day Time switch","lat":"52.179281","lon":"5.606528","startTime":"sunrise","endTime":"sunset","startOffset":"-30","endOffset":"+30","x":440,"y":760,"wires":[["a6399e43a4a46139"],[]]},{"id":"b7a1b8f0930a8cb4","type":"inject","z":"328f573d7ea7bb89","name":"Set to 0","props":[{"p":"payload.values.pvpowerout","v":"0","vt":"num"},{"p":"payload.values.pvenergytoday","v":"0","vt":"num"}],"repeat":"","crontab":"00 00 * * *","once":false,"onceDelay":0.1,"topic":"","x":240,"y":360,"wires":[["93a6265c9704e392"]]},{"id":"5aa2c2b741e69c78","type":"inject","z":"328f573d7ea7bb89","name":"Set to 0","props":[{"p":"payload.values.pvpowerout","v":"0","vt":"num"},{"p":"payload.values.pvenergytoday","v":"0","vt":"num"}],"repeat":"","crontab":"00 00 * * *","once":false,"onceDelay":0.1,"topic":"","x":240,"y":500,"wires":[["fc15b7c977b0244f"]]},{"id":"33a0ff3c796d3e4a","type":"inject","z":"328f573d7ea7bb89","name":"Set to 0","props":[{"p":"payload.values.pvpowerout","v":"0","vt":"num"},{"p":"payload.values.pvenergytoday","v":"0","vt":"num"}],"repeat":"","crontab":"00 00 * * *","once":false,"onceDelay":0.1,"topic":"","x":240,"y":640,"wires":[["7c2c907eac473fde"]]},{"id":"f9f13036.e28b58","type":"mqtt-broker","name":"localhost","broker":"127.0.0.1","port":"1883","clientid":"","autoConnect":true,"usetls":false,"protocolVersion":"4","keepalive":"60","cleansession":true,"birthTopic":"","birthQos":"0","birthRetain":"false","birthPayload":"","birthMsg":{},"closeTopic":"","closeQos":"0","closePayload":"","closeMsg":{},"willTopic":"","willQos":"0","willPayload":"","willMsg":{},"sessionExpiry":""}]

Regards
PieterS
Posts: 196
Joined: Wednesday 31 May 2017 16:06
Target OS: NAS (Synology & others)
Domoticz version: 2024.7
Location: NL
Contact:

Re: Join several dataloggers and calculate values in Node-Red

Post by PieterS »

@Firewizard,
Thanks a lot for your help so far. I succeeded to add the unknow node into my Node-Red container in Docker. Configured that node to my position and settings. The sky is the limit!! :P

After that part I configured two importnodes: The Growatt and the SMA.
Status is connected but each of them give and error... I don't understand why.
The Growatt should have the right valuenames pvpowerout and pvenergytoday and values in its JSON-message of the manual inject node.
But I get this error in the right pane.
Image

I do not understand why there is no output. Maybe you can explain this to me?
In my opinion the datastream on daytime is Node named Growatt MQTT In 1 -> Store Data 1 -> 5 Min Tick -> Day Time switch -> Function.
My questions:
1: is this thought correct?
2: the data is temporarily stored in the memory of Node-Red?

Tomorrow I will configure the second manual inject node for the SMA and add a calculation for pvenergytoday in the function node.

This is my current flow:

Code: Select all

[{"id":"b23a181c.6db8c8","type":"tab","label":"GroWatt","disabled":false,"info":""},{"id":"e16df0af.e84e4","type":"inject","z":"b23a181c.6db8c8","name":"Manual Inject Growatt","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"energy/growatt","payload":"{\"device\": \"FPH1B0001\", \"time\": \"2022-03-25T12:49:27\", \"buffered\": \"no\", \"values\": {\"datalogserial\": \"XGD2AAAAAA\", \"pvserial\": \"FPH1B0001\", \"pvstatus\": 1, \"pvpowerin\": 15401, \"pv1voltage\": 2286, \"pv1current\": 69, \"pv1watt\": 15401, \"pv2voltage\": 0, \"pv2current\": 0, \"pv2watt\": 0, \"pvpowerout\": 15093, \"pvfrequentie\": 5000, \"pvgridvoltage\": 2334, \"pvgridcurrent\": 64, \"pvgridpower\": 15120, \"pvgridvoltage2\": 0, \"pvgridcurrent2\": 0, \"pvgridpower2\": 0, \"pvgridvoltage3\": 0, \"pvgridcurrent3\": 0, \"pvgridpower3\": 0, \"totworktime\": 17713903, \"pvenergytoday\": 56, \"pvenergytotal\": 7419, \"epvtotal\": 7507, \"epv1today\": 56, \"epv1total\": 7507, \"epv2today\": 0, \"epv2total\": 0, \"pvtemperature\": 458, \"pvipmtemperature\": 479}}","payloadType":"json","x":300,"y":60,"wires":[["629bed59.838554"]]},{"id":"629bed59.838554","type":"mqtt out","z":"b23a181c.6db8c8","name":"MQTT Out Growatt","topic":"energy/growatt","qos":"0","retain":"false","respTopic":"","contentType":"","userProps":"","correl":"","expiry":"","broker":"7fb3e06.2d91f2","x":590,"y":60,"wires":[]},{"id":"e75b8555.13ec58","type":"inject","z":"b23a181c.6db8c8","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"energy/growatt2","payload":"{\"device\": \"FPH1B0002\", \"time\": \"2022-03-25T12:49:27\", \"buffered\": \"no\", \"values\": {\"datalogserial\": \"XGD2AAAAAA\", \"pvserial\": \"FPH1B0002\", \"pvstatus\": 1, \"pvpowerin\": 15401, \"pv1voltage\": 2286, \"pv1current\": 69, \"pv1watt\": 15401, \"pv2voltage\": 0, \"pv2current\": 0, \"pv2watt\": 0, \"pvpowerout\": 15093, \"pvfrequentie\": 5000, \"pvgridvoltage\": 2334, \"pvgridcurrent\": 64, \"pvgridpower\": 15120, \"pvgridvoltage2\": 0, \"pvgridcurrent2\": 0, \"pvgridpower2\": 0, \"pvgridvoltage3\": 0, \"pvgridcurrent3\": 0, \"pvgridpower3\": 0, \"totworktime\": 17713903, \"pvenergytoday\": 56, \"pvenergytotal\": 7419, \"epvtotal\": 7507, \"epv1today\": 56, \"epv1total\": 7507, \"epv2today\": 0, \"epv2total\": 0, \"pvtemperature\": 458, \"pvipmtemperature\": 479}}","payloadType":"json","x":250,"y":120,"wires":[["19a3af2a.04c389"]]},{"id":"19a3af2a.04c389","type":"mqtt out","z":"b23a181c.6db8c8","name":"MQTT Out 2","topic":"energy/growatt2","qos":"0","retain":"false","respTopic":"","contentType":"","userProps":"","correl":"","expiry":"","broker":"f9f13036.e28b58","x":430,"y":120,"wires":[]},{"id":"d9a4226a.6ddf1","type":"inject","z":"b23a181c.6db8c8","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"energy/growatt1","payload":"{\"device\": \"FPH1B0003\", \"time\": \"2022-03-25T12:49:27\", \"buffered\": \"no\", \"values\": {\"datalogserial\": \"XGD2AAAAAA\", \"pvserial\": \"FPH1B0003\", \"pvstatus\": 1, \"pvpowerin\": 15401, \"pv1voltage\": 2286, \"pv1current\": 69, \"pv1watt\": 15401, \"pv2voltage\": 0, \"pv2current\": 0, \"pv2watt\": 0, \"pvpowerout\": 15093, \"pvfrequentie\": 5000, \"pvgridvoltage\": 2334, \"pvgridcurrent\": 64, \"pvgridpower\": 15120, \"pvgridvoltage2\": 0, \"pvgridcurrent2\": 0, \"pvgridpower2\": 0, \"pvgridvoltage3\": 0, \"pvgridcurrent3\": 0, \"pvgridpower3\": 0, \"totworktime\": 17713903, \"pvenergytoday\": 56, \"pvenergytotal\": 7419, \"epvtotal\": 7507, \"epv1today\": 56, \"epv1total\": 7507, \"epv2today\": 0, \"epv2total\": 0, \"pvtemperature\": 458, \"pvipmtemperature\": 479}}","payloadType":"json","x":250,"y":180,"wires":[["52e8c134.a85f1"]]},{"id":"52e8c134.a85f1","type":"mqtt out","z":"b23a181c.6db8c8","name":"MQTT Out 3","topic":"energy/growatt3","qos":"0","retain":"false","respTopic":"","contentType":"","userProps":"","correl":"","expiry":"","broker":"f9f13036.e28b58","x":430,"y":180,"wires":[]},{"id":"ac7e1b02.f0dc6","type":"mqtt in","z":"b23a181c.6db8c8","name":"Growatt MQTT In 1","topic":"energy/growatt","qos":"0","datatype":"json","broker":"7fb3e06.2d91f2","nl":false,"rap":true,"rh":0,"x":270,"y":280,"wires":[["ded18253.8caeb"]]},{"id":"304485db.9c561a","type":"mqtt in","z":"b23a181c.6db8c8","name":"SMA MQTT In 2","topic":"sbfspot/data","qos":"2","datatype":"json","broker":"7fb3e06.2d91f2","nl":false,"rap":true,"rh":0,"x":260,"y":420,"wires":[["e3f550c.f4147b"]]},{"id":"51fb39a.911c7c8","type":"mqtt in","z":"b23a181c.6db8c8","name":"MQTT In 3","topic":"energy/growatt3","qos":"2","datatype":"json","broker":"f9f13036.e28b58","nl":false,"rap":true,"rh":0,"x":240,"y":560,"wires":[["6791e283.1d7fac"]]},{"id":"ded18253.8caeb","type":"change","z":"b23a181c.6db8c8","name":"Store Data 1","rules":[{"t":"set","p":"pvpowerout1","pt":"flow","to":"payload.values.pvpowerout","tot":"msg"},{"t":"set","p":"pvenergytoday1","pt":"flow","to":"payload.values.pvenergytoday","tot":"msg"}],"action":"","property":"","from":"","to":"","reg":false,"x":470,"y":320,"wires":[[]]},{"id":"e3f550c.f4147b","type":"change","z":"b23a181c.6db8c8","name":"Store Data 2","rules":[{"t":"set","p":"pvpowerout2","pt":"flow","to":"payload.values.pvpowerout","tot":"msg"},{"t":"set","p":"pvenergytoday2","pt":"flow","to":"payload.values.pvenergytoday","tot":"msg"}],"action":"","property":"","from":"","to":"","reg":false,"x":470,"y":460,"wires":[[]]},{"id":"6791e283.1d7fac","type":"change","z":"b23a181c.6db8c8","name":"Store Data 3","rules":[{"t":"set","p":"pvpowerout3","pt":"flow","to":"payload.values.pvpowerout","tot":"msg"},{"t":"set","p":"pvenergytoday3","pt":"flow","to":"payload.values.pvenergytoday","tot":"msg"}],"action":"","property":"","from":"","to":"","reg":false,"x":430,"y":600,"wires":[[]]},{"id":"c212a3fa.058d78","type":"inject","z":"b23a181c.6db8c8","name":"5 Min Tick","props":[{"p":"payload"}],"repeat":"300","crontab":"","once":true,"onceDelay":0.1,"topic":"","payload":"","payloadType":"date","x":250,"y":760,"wires":[["bbfbcf75.3efa08"]]},{"id":"9a094e69.733838","type":"function","z":"b23a181c.6db8c8","name":"Function","func":"msg.payload = flow.get(\"pvpowerout1\") + flow.get(\"pvpowerout2\") + flow.get(\"pvpowerout3\");\nreturn msg;","outputs":1,"noerr":0,"initialize":"","finalize":"","libs":[],"x":640,"y":760,"wires":[["f4d48b2e.e485e8"]]},{"id":"f4d48b2e.e485e8","type":"debug","z":"b23a181c.6db8c8","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","statusVal":"","statusType":"auto","x":830,"y":760,"wires":[]},{"id":"f1a542b4.138358","type":"inject","z":"b23a181c.6db8c8","name":"Set to 0","props":[{"p":"payload.values.pvpowerout","v":"0","vt":"num"},{"p":"payload.values.pvenergytoday","v":"0","vt":"num"}],"repeat":"","crontab":"00 00 * * *","once":false,"onceDelay":0.1,"topic":"","payloadType":"str","x":240,"y":360,"wires":[["ded18253.8caeb"]]},{"id":"f05d9e3e.f2af28","type":"inject","z":"b23a181c.6db8c8","name":"Set to 0","props":[{"p":"payload.values.pvpowerout","v":"0","vt":"num"},{"p":"payload.values.pvenergytoday","v":"0","vt":"num"}],"repeat":"","crontab":"00 00 * * *","once":false,"onceDelay":0.1,"topic":"","x":240,"y":500,"wires":[["e3f550c.f4147b"]]},{"id":"a5559973.746878","type":"inject","z":"b23a181c.6db8c8","name":"Set to 0","props":[{"p":"payload.values.pvpowerout","v":"0","vt":"num"},{"p":"payload.values.pvenergytoday","v":"0","vt":"num"}],"repeat":"","crontab":"00 00 * * *","once":false,"onceDelay":0.1,"topic":"","x":240,"y":640,"wires":[["6791e283.1d7fac"]]},{"id":"bbfbcf75.3efa08","type":"time-range-switch","z":"b23a181c.6db8c8","name":"Day Time switch","lat":"51.3202409","lon":"5.4581819","startTime":"sunrise","endTime":"sunset","startOffset":"-30","endOffset":"+30","x":440,"y":760,"wires":[["9a094e69.733838"],[]]},{"id":"7fb3e06.2d91f2","type":"mqtt-broker","name":"","broker":"192.168.1.18","port":"1883","clientid":"","usetls":false,"protocolVersion":"4","keepalive":"60","cleansession":true,"birthTopic":"","birthQos":"0","birthPayload":"","birthMsg":{},"closeTopic":"","closePayload":"","closeMsg":{},"willTopic":"","willQos":"0","willPayload":"","willMsg":{},"sessionExpiry":""},{"id":"f9f13036.e28b58","type":"mqtt-broker","name":"localhost","broker":"127.0.0.1","port":"1883","clientid":"","usetls":false,"protocolVersion":"4","keepalive":"60","cleansession":true,"birthTopic":"","birthQos":"0","birthRetain":"false","birthPayload":"","birthMsg":{},"closeTopic":"","closeQos":"0","closePayload":"","closeMsg":{},"willTopic":"","willQos":"0","willPayload":"","willMsg":{},"sessionExpiry":""}]
FYI: I had to change each field of a server to an IP:portnumber as I run InfluxDB, Node-Red and Domoticz in seperate Docker-containers. Sorry for the inconvenience.

Thanks for all your help so far.

Pieter
Synology with Domoticz build (V2024.7) in Docker
PieterS
Posts: 196
Joined: Wednesday 31 May 2017 16:06
Target OS: NAS (Synology & others)
Domoticz version: 2024.7
Location: NL
Contact:

Re: Join several dataloggers and calculate values in Node-Red

Post by PieterS »

@Firewizard,
We made steps forward!
This morning the flow is reading input from the Growatt-inverter and now I got the first results from the Function-node:
Image

So now I will add the calculation for pvenergytoday and will try to change the JSON-output from the SMA. I'll keep you informed
Synology with Domoticz build (V2024.7) in Docker
PieterS
Posts: 196
Joined: Wednesday 31 May 2017 16:06
Target OS: NAS (Synology & others)
Domoticz version: 2024.7
Location: NL
Contact:

Re: Join several dataloggers and calculate values in Node-Red

Post by PieterS »

@Firewizard,
This morning I spent some time to get data through MQTT from my Omnik-inverter.
I succeeded after installing PAHO MQTT client on the Synology. But the format in MQTT Explorer looks different to me, compared to the Growatt and SMA. And its in a (sort of) subtopic..
Image
pic code
This is the application I use for reading the Omnik-inverter. https://github.com/XtheOne/Inverter-Data-Logger
This is output from the logfile:

Code: Select all

2022-03-30 11:12:18,726 DEBUG Importing output plugin DomoticzOutput
2022-03-30 11:12:18,753 DEBUG Importing output plugin MQTTOutput
2022-03-30 11:12:22,215 INFO Loggers found  on the network: ['192.168.1.22', '60544XXXX'].
2022-03-30 11:12:22,216 INFO Connecting to logger with IP: 192.168.1.22 and SN 60544XXXX
2022-03-30 11:12:22,216 INFO connecting to 192.168.1.22 port 8899
2022-03-30 11:12:22,219 DEBUG RAW sent Packet (len=16): 68:02:41:b1:c9:64:16:24:c9:64:16:24:01:00:c3:16  hAd$d$
2022-03-30 11:12:23,293 DEBUG RAW received Packet (len=139): 68:7d:41:b0:c9:64:16:24:c9:64:16:24:81:02:01:4e:4c:44:4e:31:30:32:30:31:33:38:42:31:30:30:39:01:09:04:ea:00:00:ff:ff:00:39:00:b5:ff:ff:00:1c:ff:ff:ff:ff:09:07:ff:ff:ff:ff:13:8e:02:8f:ff:ff:ff:ff:ff:ff:ff:ff:00:56:00:00:cc:c2:00:00:62:55:00:01:00:00:00:00:ff:ff:00:00:00:00:00:00:00:00:00:00:4e:4c:31:2d:56:31:2e:30:2d:30:30:34:33:2d:34:00:00:00:00:00:56:31:2e:36:2d:30:30:31:38:00:00:00:00:00:00:00:00:00:00:00:34:16  h}Ad$d$NLDN1020138B10099VbUNL1-V1.0-0043-4V1.6-00184
2022-03-30 11:12:23,294 DEBUG DATA len=125: 
2022-03-30 11:12:23,294 INFO Inverter ID: NLDN1020138B1XXXX
2022-03-30 11:12:23,295 INFO RUN State: 1
2022-03-30 11:12:23,295 DEBUG Run pluginDomoticzOutput
2022-03-30 11:12:23,296 INFO Uploading data from inverter with ID: NLDN1020138BXXXX to Domoticz
2022-03-30 11:12:23,296 DEBUG url: http://192.168.1.18:8080/json.htm
2022-03-30 11:12:23,297 DEBUG Temperature   : 26.5 degrees celcius
2022-03-30 11:12:23,297 DEBUG PV1 voltage   : 125.8 Volt
2022-03-30 11:12:23,297 DEBUG AC1 voltage   : 231.1 Volt
2022-03-30 11:12:23,298 DEBUG AC total power: 655 Watt
2022-03-30 11:12:23,298 DEBUG e_today       : 0.86 kWh
2022-03-30 11:12:23,298 DEBUG msg.e_total   : 5241.8 kWh
2022-03-30 11:12:23,298 DEBUG e_total       : 5241.86 kWh
2022-03-30 11:12:23,299 DEBUG E_total not defined: 5241.86 kWh
2022-03-30 11:12:23,299 DEBUG H_total not defined: 25173 hours
2022-03-30 11:12:23,300 DEBUG PV2 current out of range, or not defined: 18.1 Ampere
2022-03-30 11:12:23,300 DEBUG PV3 current out of range, or not defined: -0.1 Ampere
2022-03-30 11:12:23,301 DEBUG PV1/2/3 current not defined: 5.7/18.1/-0.1 Ampere
2022-03-30 11:12:23,301 DEBUG PV2 voltage out of range, or not defined: 0.0 Volts
2022-03-30 11:12:23,302 DEBUG PV3 voltage out of range, or not defined: -0.1 Volts
2022-03-30 11:12:23,302 DEBUG PV1/2/3 voltage not defined: 125.8/0.0/-0.1 Volts
2022-03-30 11:12:23,302 DEBUG AC2 current out of range, or not defined: -0.1 Ampere
2022-03-30 11:12:23,303 DEBUG AC3 current out of range, or not defined: -0.1 Ampere
2022-03-30 11:12:23,303 DEBUG AC1/2/3 current not defined: 2.8/-0.1/-0.1 Ampere
2022-03-30 11:12:23,304 DEBUG AC2 voltage out of range, or not defined: -0.1 Volt
2022-03-30 11:12:23,304 DEBUG AC3 voltage out of range, or not defined: -0.1 Volt
2022-03-30 11:12:23,304 DEBUG AC1/2/3 voltage not defined: 231.1/-0.1/-0.1 Volts
2022-03-30 11:12:23,305 DEBUG AC2 power out of range, or not defined: -1 Watt
2022-03-30 11:12:23,305 DEBUG AC3 power out of range, or not defined: -1 Watt
2022-03-30 11:12:23,306 DEBUG AC123 power not defined: 655 Watt
2022-03-30 11:12:23,306 DEBUG AC2 frequency out of range, or not defined: -0.01 Hertz
2022-03-30 11:12:23,306 DEBUG AC3 frequency out of range, or not defined: -0.01 Hertz
2022-03-30 11:12:23,307 DEBUG http://192.168.1.18:8080/json.htm?type=command&svalue=125.8&idx=1278&param=udevice&nvalue=0
2022-03-30 11:12:23,314 DEBUG {
	"status" : "OK",
	"title" : "Update Device"
}

2022-03-30 11:12:23,357 DEBUG http://192.168.1.18:8080/json.htm?type=command&svalue=5241860.0&idx=1288&param=udevice&nvalue=0
2022-03-30 11:12:23,363 DEBUG {
	"status" : "OK",
	"title" : "Update Device"
}

2022-03-30 11:12:23,363 DEBUG Run pluginMWTTOutput
2022-03-30 11:12:23,370 DEBUG mqtt_topic: energy/omnik/NLDN1020138B1XXXX/
2022-03-30 11:12:23,377 DEBUG RAW received Packet (len=99): 68:11:41:f0:c9:64:16:24:c9:64:16:24:44:41:54:41:20:53:45:4e:44:20:49:53:20:4f:4b:0d:0a:01:16:ff:ff:ff:ff:ff:ff:ff:ff:ff:ff:ff:ff:ff:ff:ff:ff:ff:ff:ff:ff:ff:ff:ff:ff:ff:ff:ff:ff:ff:ff:ff:ff:ff:ff:ff:ff:ff:ff:ff:ff:ff:ff:ff:ff:ff:ff:ff:ff:ff:ff:ff:ff:ff:ff:ff:ff:ff:ff:ff:ff:ff:ff:ff:ff:ff:ff:ff:ff  hAd$d$DATA SEND IS OK
2022-03-30 11:12:23,377 DEBUG DATA len=17: 
2022-03-30 11:12:23,378 DEBUG Exit Status: DATA SEND IS OK
This is the section for MQTT in the config.cfg of the application

Code: Select all

[mqtt]
# Host where the mqtt server is, including user/pass authentication
host = 192.168.1.18
port = 1883
user = 
pass = 
topic = energy/omnik
I am not familair with programming in Python but I guess there must be a change in line 13 of the MQTToutput.py-script. But that is for the name of the topic.. Not the format of the output, I think.
I hope you can help me to analyse this data in Node-Red.

Pieter
Synology with Domoticz build (V2024.7) in Docker
thomasbaetge
Posts: 153
Joined: Wednesday 02 October 2019 11:47
Target OS: Linux
Domoticz version: 2023.1
Location: DE / BY / LT
Contact:

Re: Join several dataloggers and calculate values in Node-Red

Post by thomasbaetge »

Hi,

seems that your MQTT message comes as unformatted as raw data. Actually it is key - value pairs, separated by ':'.
This also explains your NaN error from earlier (NaN = not a number. A numeric format was expected, but a string or smth else was delivered). That can be solved by a simple ParseFloat(YOURVALUE) conversion in a function node.

you may want to run your data through a JSON node (standard) and set it to 'always convert to Json Object'.
Try that and see if it gets better ;)

don't give up...we will get you there... ;)
Post Reply

Who is online

Users browsing this forum: No registered users and 0 guests