Victron Venus OS Topic is solved

[tux21]

Hi all, please is there anybody who successfuly managed to cooperate Domoticz with Venus OS from Victron Energy to be able to see the status of solar batteries, current from SmartShunt etc? There is running some MQTT where all these informations should be broadcasted, but I'm not experienced enough to connect things together this way and also I haven't find any solution across the web, or FB groups.

I would like to interact to battery level etc in my off grid solar system.

Thank you for any ideas how to continue.

[waltervl]

There is also node-red implementation https://github.com/victronenergy/node-r ... ib-victron to read the Victron MQTT messages.
On the Domoticz wiki there is basic information how to convert messages through node-red flows to a payload that Domoticz MQTT can understand.
https://www.domoticz.com/wiki/MQTT
https://www.domoticz.com/wiki/Flows

There is also a node-red sub forum if you need more information.

So my guess would be:
1. Install http://mqtt-explorer.com/ so you can see what is going on.
2. Get the MQTT running on Victron.
3. Get MQTT running on Domoticz
4. Make dummy devices in Domoticz so you can interact
5. Update with node-red flows from Victron to Domoticz (and vice versa)

[tux21]

Thank you for your reply. I've verified everything is working as it should. I can see in mqtt explorer broadcasted messages from Domoticz (which I'm already using for CO2 and other sensors hooked to Tasmota) and even Victron, there is everything i would like to see in Domoticz. So if I understand correctly, now I need some man in the middle, what could be Node Red to translate this messages to Domoticz?

There is also another protocol, which Domoticz should know - modbus TCP, but I'm lost even there.

[waltervl]

Yes it seems everything is there so node-red flows should be the way to go. But I cannot help you with that, I have no MQTT nor node-red.
You could try if you get the Victron node-red flows working. User FireWizard is very helpful in creating Domoticz specific flows
viewforum.php?f=81

[FireWizard]

Hello @tux21

First of all, I do not own that product from Victron, but together we can try to make it work with Domotiicz.

You said:

There is running some MQTT where all these information should be broadcasted

1. Can you tell me. if you use Venus OS Large or the standard version?
See: https://www.victronenergy.com/live/venus-os:large

This version includes Node RED, but as I read it, uses also a Node RED Dashboard.
This is not what you want, as you want to send the data to Domoticz.

2. Do you use the MQTT broker (server) on the Victron Device or do you use an external broker, such as Mosquitto?

You wrote also:

There is also another protocol, which Domoticz should know - modbus TCP, but I'm lost even there.

I would not prefer to use ModBus over MQTT, as Domoticz has no standard hardware or interface to ModBus.

ModBus is a protocol where the values are written into Read and Read/Write registers.
Of course you need the information of what is written in which register.
Node-Red has ModBus nodes, but at the end you need either MQTT or HTTP.

As @waltervl already mentioned, Node Red has Victron nodes, which are probably useful.

I see in your screenshot that you receive some data (a lot).
Can you indicate what data from your Victron device, you like to see to send to Domoticz?

Regards

[tux21]

Guys, as first, thank you both for your effort and helpfulness.

Hello @FireWizard,

Here are information based on your questions.

1. I'm using standard version of Venus OS as I don't see any mentioned options in menu for Node Red settings. I don't know which one is better for our purposes. I have root access and SSH running, so maybe I can try to install Node Red manually. Or also there shouldn't be any problem to switch to Large version if would be better.

2. MQTT broker is already embedded inside Victron Venus OS image, I haven't made any changes to be working, just allowed this service in menu and it is broadcasting what I've send in the picture before.

Ok, modbus is not the way, thank you for explanation.

And what I would like to see in Domoticz - I have off-grid solar system with several batteries. I would like to see there state of charge of currently charged battery, current battery drain and also power coming from solar charger. These stats are also available within Victron ecosystem, but I would like to interact on these statuses. Like for example switch this panel to another battery because the main is full and also there is no current running out of the main battery and so on, which are easy to set in Domoticz.

[kiddigital]

Look like Venus OS has it own MQTT broker where different topics are available depending on the connected hardware modules.

(Check https://github.com/victronenergy/dbus-mqtt)

It handles the translation from Victron’s D-bus messages into MQTT messages on specific topics.

More info can be found on Github pages from Victron:

For example

https://github.com/victronenergy/venus- ... /TOPICS.md

And there is an (example) UI that reads these topics.

I think there are at least two options:
- a Domoticz ‘hardware’ module that connects to the MQTT broker on the Venus OS (similar to some other modules that read from external MQTT brokers and translates specific topics/messages into Domoticz devices)
- Find a way to let the MQTT broker ‘forward’ the messages to another (Domotic) broker

Interesting topic.

Also I have been looking (but not started or succeeded) into ways to read the data from certain Victron products that do not use Venus OS but have bluetooth interfaces. Did not find any docs on it (yet).

[FireWizard]

Hi, @tux21

Okay, In my opinion it is not a problem to use Victrons built-in broker. We can communicate from Node Red with it.

Currently there seems to be an issue with the Victrons nodes for Node RED and so we will not use these.

See: https://discourse.nodered.org/t/node-re ... ance/48671

To start:

Can you install Node Red?
Install version 1.3.5 and not the version released today (See:https://nodered.org/blog/2021/07/20/ver ... 0-released)
You may try it, but I still use 1.3.5
See: https://nodered.org/docs/getting-started/raspberrypi
Use the install script

Code: Select all

bash <(curl -sL https://raw.githubusercontent.com/node-red/linux-installers/master/deb/update-nodejs-and-nodered)

If the installation has been finished open a webbrowser and try to connect to http://<IP Address>:1880

Let me know.

Regards

[tux21]

Hi @FireWizard,

I tried to install Node Red on the Raspberry Pi running Venus OS, but unfortunately without success. The script does not working and also known commands as apt-get or so are not working. So I tried to run mentioned large version and it works just fine. I'm able to see the gui of Node Red on IP:1880/ and it is version 1.0.5. Could this cause any problems and should I still try to get install Node Red manually instead? Thank you

[FireWizard]

Good evening @tux21,

The script does not working

Strange, should not be the case, but I never installed it on a device, that had Venus OS installed.

known commands as apt-get or so are not working.

I understand, as that is not the correct way of installing.

So I tried to run mentioned large version and it works just fine. I'm able to see the gui of Node Red on IP:1880/ and it is version 1.0.5. Could this cause any problems and should I still try to get install Node Red manually instead?

No this should not cause a problem, but it might happen that a particular flow doesn't work, as I use a later version.
However I will this keep in mind.

I will come back to you later, as I have to sort out a few things first.

Regards

[tux21]

Hello @FireWizard,

I'm sorry, I'm not well experienced in unix based OSes, but I will read about it, because I was surprised there are not working lot of commands I know from Raspbian.

Thank you very much again

[FireWizard]

Hello @tux21,

You wrote:

I'm sorry, I'm not well experienced in unix based OSes, but I will read about it, because I was surprised there are not working lot of commands I know from Raspbian.

That does not matter, as we will not go that deep.

We will use the Node Red editor, which use drag/drop nodes from the library to the canvas. You will learn that this is quite easy, as soon as you start working with it. Perhaps we need a little bit of JavaScript, but this will not be the problem.

Let us start.

1. In Domoticz, install the hardware "MQTT Client Gateway with LAN interface". Use as IP Address the Address of your VIctron MQTT server.
Check with MQTT Explorer, that you receive messages on the topic domoticz/out.

2. Drag and drop one of the "Victron" nodes to the canvas and also the same with a "Debug" node.
Connect these two together.
(I have not installed these nodes, for the reason described in the previous link)
Read the result of the "Debug" messages in the Debug pane.

Example: https://energytalk.co.za/t/starting-wit ... -guide/456

Let me know.

Regards

[tux21]

Hello @FireWizard,

It seems it works as expected so far:

1. Domoticz settings


2. MQTT explorer see my Domoticz server


3. And finally NodeRed dashboard


Thank you much for your time

[FireWizard]

Hello @tux21,

This looks good, so we can continue

However, I have a few questions, if I look to your screenshots.

1. In the 1st screenshot, I see that you have 2 MQTT Client Gateways with LAN interface installed.
The first one, with IDX 9 and called "victron" is connected with the MQTT broker on IP Address 192.168.31.68.
I assume that this is the Victron broker. But Domoticz is not directly communicating with this broker.
You can use it, but you have also a second interface, called "mqtt", with IP Address 192.168.31.148.
Is there a MQTT broker running on this address? If Yes, which one (Mosquitto?)

2. In your 2nd screenshot, I see the published topics on the Victron broker.
What does domoticz show? Click on the arrow, so that the tree opens.

3. In the 3rd screenshot I see a Victron node presenting the Battery Voltage (as a number).
It presents the Voltage with 15 (!) digits after the decimal point. I suggest that we reduce this to 2 digits.
Otherwise it will not look nice in the Domoticz widget.

Below you will find an example, how to handle this battery voltage.

You can do the same with the other Victron nodes, but keep in mind that some might not work.
See the links in the previous posts.

Okay, let 's start.

1. Create a new "Hardware" device, called e.g. Victron of the type "Dummy (Does nothing, use for virtual switches only)".

2. Create a new device of the type "Voltage" and give it a name, you like, e.g. "Victron Battery".

3. If you go to: https://piandmore.wordpress.com/2019/02 ... -domoticz/ or
https://www.domoticz.com/wiki/Domoticz_API/JSON_URL%27s, you will find that the required syntax for a "Voltage" sensor is:
{"command":"udevice", "idx":1234, "svalue":"3.45"}

The voltage sensor will show voltage of something. You can add digits after the decimal if needed.

As you will see the value is presented after "svalue". This is a string.
So we have to convert the Voltage (number) to a string.

2. Drag and drop a "Function" node to the canvas and connect the input of the "Function" to the output of the "Victron" node.

The contents of the "Function"node is as follows:

Code: Select all

msg.payload = {"command":"udevice","idx":12,"nvalue":0,"svalue":msg.payload.toFixed(2).toString()};
return msg;

The elements in this object are:

- "command":"udevice" ; This is a default value and may be left out.
- "idx":12; This is the IDX of the device, you created previously. Change it to your device number.
- "nvalue":0; Just for completeness. May be removed.
- "svalue":msg.payload.toFixed(2).toString(); The important part.

Function toFixed(2) is responsible for 2 digits after the decimal point. You can change the number, if you want more or less digits.
Function toString() converts the numerical value to the string value.

Let 's look to the result. (I simulate the Victron node with the "Inject" node)

Screenshot_victron1.png (33.42 KiB) Viewed 5469 times

This will result in Domoticz to:

Screenshot_victron2.png (39.25 KiB) Viewed 5469 times

You have to configure your MQTT out node to your settings (It depends on which one, you use)
The message should be published to domoticz/in

Screenshot_victron3.png (20.58 KiB) Viewed 5469 times

Please find the complete flow (excluding the Inject node) below:

Code: Select all

[{"id":"db4f012.1fb0f","type":"debug","z":"4137f1f8.cf269","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","statusVal":"","statusType":"auto","x":1010,"y":120,"wires":[]},{"id":"16dc3c48.4c0c1c","type":"function","z":"4137f1f8.cf269","name":"","func":"msg.payload = {\"command\":\"udevice\",\"idx\":12,\"nvalue\":0,\"svalue\":msg.payload.toFixed(2).toString()};\nreturn msg;","outputs":1,"noerr":0,"initialize":"","finalize":"","libs":[],"x":1000,"y":180,"wires":[["332aeeb.adb9092","f803b251.5c6ff8"]]},{"id":"332aeeb.adb9092","type":"debug","z":"4137f1f8.cf269","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","statusVal":"","statusType":"auto","x":1230,"y":120,"wires":[]},{"id":"f803b251.5c6ff8","type":"mqtt out","z":"4137f1f8.cf269","name":"To Domoticz","topic":"domoticz/in","qos":"","retain":"","respTopic":"","contentType":"","userProps":"","correl":"","expiry":"","broker":"f9f13036.e28b58","x":1230,"y":180,"wires":[]},{"id":"f9f13036.e28b58","type":"mqtt-broker","name":"localhost","broker":"127.0.0.1","port":"1883","clientid":"","usetls":false,"compatmode":true,"keepalive":"60","cleansession":true,"birthTopic":"","birthQos":"0","birthRetain":"false","birthPayload":"","closeTopic":"","closeQos":"0","closePayload":"","willTopic":"","willQos":"0","willPayload":""}]

If everything is functioning correct remove the (green) debug nodes.

Regards and let me know.

[tux21]

Hello @FireWizard,

You are genius, it works exactly as you said, thank you so much for your help!

To finish the thread:

1. Yes, you are right. It is Mosquitto running on Raspberry Pi where is my Domoticz instance and where are hooked some ESP01 with Tasmota for CO2, humidity sensors etc. I was afraid there could be any conflict, but no, it works just great all together.

2. There are running all data from all sensors periodically in one row, see the screenshot


3. Yes exactly, I was also suprised how many digits are there and you are right again - two digits are far enough.

So here is how is it looking what I needed in Domoticz:


And here is the flow in NodeRed:


So once again, thank you (and you all guys) for your help. I was unable to get this working without you.

[tux21]

Hello all,

So thanks to FireWizard I was able to set up my offgrid solar system from Victron Energy and interact to the statuses in Domoticz.
My solar system is created from MPPT solar charger Bluesolar 100/20, Smart Shunt 500A, Phoenix 24/800 inverter a Raspberry Pi running official large Venus OS image.

Here is the link to install large Venus OS:
https://www.victronenergy.com/live/venus-os:large

Here is Node Red flow I used for my setup:
https://1drv.ms/u/s!AshuXSMP155fgaZQji1 ... w?e=dgf3Jk

And picture:


In Domoticz I have just created another MQTT gateway pointing to Raspberry with Venus OS and couple of dummy devices which IDX have to be set in each "function" module in Node Red.


And at last, this is how is it looking on my dashboard:


With this setup I could easily interact on state of charge of my battery and solar charger output. For example I could switch some of my solar panels to water heater in case there is no battery drain and connect it back on demand, charge secondary battery for my electric scooter etc. It works really great.

Thank you once again for help to make this all happen.