Connecting Plugwise-2-py via MQTT to Domoticz

[dorenberg]

SevenW, good that there is still good progress in this topic. I think the best option is to try and add the source code into domoticz so it can function independently, but maybe another option is still better......

[bloemkool]

I have been looking into setting the switch-state of a circle to domoticz, and switching it with domoticz using the on/off actions. As a starting point I took the Nov. 5th node-red scripts of bloemkool.

Note... there is a config error in that/my nodered config.
In the power readout function there is a powerJSON.mac.substr(10,6), so 6 chars after the tenth char. In the switch/schedule function is a msg.payload.mac used together with a part of the mac adress ...What you fill in the script://-line in domoticz is a 7 digit part of the mac-address. So actually the power readout function should contain powerJSON.mac.substr(9,7).
This is because of a replaced circle and that replacement uses a "larger" mac address. So working with an complete MAC address is always favorable.

An excerpt from the pw-conf.json makes that very clear :

Code: Select all

{"mac": "000D6F000278F3A1", "category": "computer", "name": " ...
{"mac": "000D6F000038BE8A", "category": "EV", "name": "Volvo" ...
{"mac": "000D6F000038D3A4", "category": "witgoed", "name": "d ...
{"mac": "000D6F000038C3AF", "category": "witgoed", "name": "k ...

[bloemkool]

I do not have yet the killer idea to solve this. There is always a chance of the device getting an external update (due to external switching, plugwise schedule switching, a state-update heartbeat), and there is always the chance that at the same time the switch is operated form the domoticz UI(s). In the end, domoticz and plugwise need to agree on the actual switch state.
SevenW

It's a complicated situation.
The problem is not the switching itself, that is working.
The problem is the status update of the switch status in Domoticz. And we want to update the status of the switch in domoticz every second or 30 seconds. Not only once or when it is switched. No, every x times per second.

Can we alter the http:// into switch?mac=038BE8A&cmd=[switch|update]&val=on
And do some hocus pocus with "cmd" with the function that is attached to the [get] in NodeRed?
Domoticz runs the command always with cmd=switch, maybe the crontab/timer in NodeRed can do a cmd=update to differentiate.
There is also a possibility to do stuff with user variables in Domoticz via the API. Is that usable?

Thinking of it... This might include a third virtual switch. ( three units is better I think )
Why three?
1 virtual switch for the power reading
2 virtual switch for the actual switching
3 virtual switch for the status of the switch

[SevenW]

I am making some interesting progress. Per circle I have two virtual switches. The first is the switch, and the second switches the schedule on or off. With the schedule switched on, the switch should follow the state of the plugwise circle. The following graph shows that the switch gets updated according to the schedule in the circle:

domoticz plugwise switch by schedule.png (127.41 KiB) Viewed 7280 times

I have also added cumulative energy (kWh's) to the virtual sensors. I use the General,kWh instead of the Usage, electric sensors for this. After running this for a few days and validating the behavior I shall post my node-red stuff.

SevenW

[SevenW]

... and I found a way to break the update loop. It is also a simpler setup. I do not use the On / Off actions anymore. I stopped listening to http calls to node-red from domoticz, but I subscribe to domticz/out MQTT messages. I filter them so that I get only MQTT switch messages. With only a minor trick / workaround I know when a switch is invoked by the plugwsise-2-py webapp, or when it is generated form domoticz (or any other http/json/api client).

This makes the Plugwise-2-py - Domoticz integration very workable.

As promised will publish node-red scripts soon.

SevenW

[bloemkool]

Ahh ... Can't wait!
Is it also possible to completely eliminate NodeRed and let plugwise2py work directly with the /domoticz/in/# /domoticz/out/# MQTT ? Or is that the next step?
Or do you still need NodeRed for the circle power readout?

I;d wish I had more time to contribute on this...

[dorenberg]

Can't wait either for this. WOW! What a progress. Thanks in advance!

[SevenW]

Is it also possible to completely eliminate NodeRed and let plugwise2py work directly with the /domoticz/in/# /domoticz/out/# MQTT ? Or is that the next step?
Or do you still need NodeRed for the circle power readout?

I tested it a bit, and looked into the MQTT source code of domoticz, but currently it looks like the MQTT interface supports fewer commands, and I doubt I can automatically create devices, and I can not address them by did (plugwise ID), but only by idx. And then, I do not see a MQTT command to query all devices to map idx to did. A pure MQTT interface seems unfeasible, or at least impractical. Too bad!

Very interesting thought anyway.

SevenW

[SevenW]

I have added the NodeRed scripts to my github, as well as some very rudimentary instructions, to be found in the domoticz folder:
https://github.com/SevenW/Plugwise-2-py ... r/domoticz

It is required that you update your PlugWise-2-py from github master if you have it already installed before today. In Plugwise-2-py I have added provision for cumulative energy and to break the update loop.

Key features:

• Energy (savelog=yes) is using General-kWh sensor (cumulative kWh)
• Power (monitor=yes) is using the Usage-Electric sensor (summing the 10 seconds values is not robust in a PW network).
• When a schedule is used, the schedule name is displayed in a text-sensor.
• When a circle is offline, it is also displayed in that text-sensor.
• Separate Domoticz-switches to switch the circle on/off, and to switch the circle-schedule on/off.
• Note: when using the circle-switch, the schedule will be turned off.
• The infamous update/feedback loop of smart switches is fully prevented.
• The Device IDs are last seven digits of MAC, prefixed with: A=status text, B=breaker/switch, C=clock/schedule, E=Energy(kWh), F=Power(W)

[dorenberg]

Everything updated and so far so good. 6 months of circle data imported into domoticz as used today. But the hourly update works good! Switches and schedules and kwh meters and watt meters are being added to the unused section. I didn't have to make a dummy to get everything working.

[SevenW]

so what hardware id did you assign then? Can it be arbitrary? Or did you use the MQTT hardware id?

SevenW

[dorenberg]

In hardware number 10 is for dummy sensors (in my system). So both values are set at ten. That is the category. The idx will be auto generated in domoticz in the unused list. the circle ID with additonal letters in front as explained is the ID of the switch.

[proohu01]

So how is the MQTT hardware item configured? Should I keep the username and password empty (Domoticz Pi image is used for installation). How about the "Publish Topic"? I don't understand the choices. (flat/hierarchical/combined/none)

[dorenberg]

So how is the MQTT hardware item configured? Should I keep the username and password empty (Domoticz Pi image is used for installation). How about the "Publish Topic"? I don't understand the choices. (flat/hierarchical/combined/none)

The setup as described here uses MQTT translation via Node-red. If you have domoticz installed on port 8080 and excluded the localhost from the list to ask for a password, no adaptations of the script posted above are needed. I don't recognize the choices you present here.

[bloemkool]

Just had the time to git-pull plugwise2py and started to change the domoticz content. Was a bit of work but looks good right now!
I also saved the power data I already collected from previous config.
All we need is a change to the API: add descriptions to a device.

Great job SevenW!!

And I think I found a bug in domoticz, can anyone confirm?
-- When I add a group, name it christmas lights.
- Add a KAKU switch
- Add a Plugwise switch
-- Switch the group off/on
-- Domoticz says the Circle and the KAKU is off or on...
-- Only the KAKU switch actually switches off and on, the Plugwise switch does not respond.

When I directly switch the Circle it responds immediately.
I had this same thing with the previous http:// call I did. And hoped it would be solved with this config.

[SevenW]

So how is the MQTT hardware item configured? Should I keep the username and password empty (Domoticz Pi image is used for installation). How about the "Publish Topic"? I don't understand the choices. (flat/hierarchical/combined/none)

I assume that you followed the instructions on the MQTT wiki. https://www.domoticz.com/wiki/MQTT

Did you install Mosquitto (may be standard in Raspberry pi image, I am not sure). Mosquittto MQTT server default install is using no username/password, and default port 1883 (at localhost or 127.0.0.1).

So leave user/password fields empty. Publish topic can be set to "Flat". This works. I did not test other settings, but they may work as well, expect for "none", which is just disabling MQTT

Let me know if it works.

Regards,
SevenW

[proohu01]

@SevenW. Yes I did follow the WiKi. There was no mention of a username and password in the wiki so I already left that empty.
Setting the publish topic to "flat" seems to have fixed it for me. I now see the new devices in Domoticz. Thank you.

The power I get from all of my Plugs are negative values. Why is that? Is that normal?

[SevenW]

The power I get from all of my Plugs are negative values. Why is that? Is that normal?

That could be me causing this. When in plugwise-2-py, for a given circle production = yes, I negate that values in the nodered script, as plugwsie-2-py reports production as negative numbers, while Domoticz seems to dislike negative values (it does not draw a graph below zero). When you have a circle for a mixed production/consumption device, then my strategy is not applicable. I use it for PV-solare energy, and that circle sees only production or zero Watts.

If the negative numbers are just absolute values below 1W, than it is just noise in the circle measurement.

SevenW

[proohu01]

When you have a circle for a mixed production/consumption device....

My bad! I thought that the value "production=True/False" was to include or exclude a circle plug. As in production/test.

All of my Plugwise cicles are consumption devices so I changed it to production=false.

[bacsa]

I cannot switch de plugwise circle. In the devices i have a B/C/F/E prefix. All values are updated and seems good only switching is not possible.

What have i done wrong ?