Domoticz

Posted: **Tuesday 07 November 2023 17:42**

This topic is dedicated to the ShellyTeacher for Domoticz application which will add Shelly devices to Domoticz MQTT Autodiscovery hardware

https://github.com/enesbcs/shellyteacher4domo

Why i need this?
Because Shelly devices speaks MQTT, but has no autodiscovery config objects included, so somebody has to tell to Domoticz how can it handle them. This will be done by the Teacher app, which sends predefined config objects to the MQTT broker (retain method is highly recommended) based on the device ID's, which will be discovered by MQTT AD hardware.
The predefined config object templates are simple text files named "mqtt_templates.txt" and "mqtt_templates_gen2.txt" can be altered by anyone if something is not working as it should.

The teacher is only needed to run until adding devices, as soon as the config objects are send and saved to the MQTT broker and Domoticz detects them,the teacher can (and should) be stoppped.

Currently almost every Gen1 and Gen2 devices are supported, except RGB devices. If anybody has idea about how to add them, please share.

Here is a little tutorial:

Posted: **Friday 17 November 2023 12:05**

I am trying to control 2 Shelly TRV's from Domoticz. I want to control the valve opening, not the temperature setpoint, but valve % is added as a sensor, not as a dimmer switch like some other z-wave valves I'm using. Is this something I can fix by editing mqtt_templates?

Posted: **Sunday 19 November 2023 14:30**

@toxwa, better create a issue here: https://github.com/enesbcs/shellyteacher4domo

Posted: **Monday 20 November 2023 13:09**

toxwa wrote: ↑Friday 17 November 2023 12:05 I am trying to control 2 Shelly TRV's from Domoticz. I want to control the valve opening, not the temperature setpoint, but valve % is added as a sensor, not as a dimmer switch like some other z-wave valves I'm using. Is this something I can fix by editing mqtt_templates?

Yes,you can add manually the necessary templates to mqtt_templates.txt at [SHTRV-01] section with a proper text editor, which does not mess up line endings. Although there is a big warning in the Shelly docs, that manually setting the valve will immediately disable thermostat functions.

Shelly TRV accepts number from 0 to 100 to set manually the valve position, in % at the following MQTT topic:

Code: Select all

shellies/shellytrv-<id>/thermostat/0/command/valve_pos

And actual position returned by a nice JSON, which is in thermostats[0].pos structure at the following MQTT topic:

Code: Select all

shellies/shellytrv-<id>/info

Its a very nice idea to put this into a Light/Dimmer device, although i am unable to do it. Maybe

gizmocuz wrote: ↑Sunday 19 November 2023 14:30

can tell how to describe properly the bri_val_tpl to get value from a position_topic JSON structure and send to the set_position_topic a simple number...
This is the config object for a Dimmer, which is not working:

homeassistant/light/shellytrv-ID-pos-2/config

Code: Select all

{"name": "shellytrv-ID Valve Set Position 2", "enabled_by_default": false, "cmd_t": "~thermostat/0/command/valve_pos", "stat_t": "~info", "payload_on": "100","payload_off": "0", "val_tpl": "{{ value_json.calibrated }}","stat_val_tpl": "{{ value_json.calibrated }}","state_on":"true","state_off":"false","brightness_command_topic":"~thermostat/0/command/valve_pos","brightness_state_topic":"~info","brightness": true,"color_mode": true, "supported_color_modes": ["brightness"], "brightness_scale":100, "bri_val_tpl":"{{value_json.thermostats[0].pos}}", "avty_t": "~online", "pl_avail": "true", "pl_not_avail": "false", "uniq_id": "shellytrv-ID-pos-2", "~": "shellies/shellytrv-ID/"}

But in the meantime i figured out, that a Cover device is able to do the same, so i created a second config object, which seems to work:

homeassistant/cover/shellytrv-ID-pos-1/config

Code: Select all

{"name": "shellytrv-ID Valve Set Position 1", "enabled_by_default": false, "cmd_t": "~thermostat/0/command/valve_pos", "stat_t": "~info", "pos_t": "~info", "set_pos_t": "~thermostat/0/command/valve_pos", "pos_tpl": "{{ value_json.thermostats[0].pos }}", "pos_open": 100, "pos_clsd": 0, "avty_t": "~online", "pl_avail": "true", "pl_not_avail": "false", "uniq_id": "shellytrv-ID-pos-1", "qos": 1, "~": "shellies/shellytrv-ID/"}

(You have to replace shellytrv-ID with you device ID of course in the above example)

This two line has to be added to the mqtt_templates.txt to the [SHTRV-01] section:

Code: Select all

topic=%discovery_prefix%/cover/%shelly_id%-pos-1/config
payload={"name": "%shelly_id% Valve Set Position 1", "enabled_by_default": false, "cmd_t": "~thermostat/0/command/valve_pos", "stat_t": "~info", "pos_t": "~info", "set_pos_t": "~thermostat/0/command/valve_pos", "pos_tpl": "{{ value_json.thermostats[0].pos }}", "pos_open": 100, "pos_clsd": 0, "avty_t": "~online", "pl_avail": "true", "pl_not_avail": "false", "uniq_id": "%shelly_id%-pos-1", "qos": 1, "~": "shellies/%shelly_id%/"}

Posted: **Tuesday 21 November 2023 16:53**

Thank you very much! I added the 2 lines and can controll the valve position from Domoticz now so I can work with that.
But the only thing that doesn't work is if I change the opening with the buttons on the Shelly TRV, Domoticz doens't update the valve opening percentage. It's the same if I put them in thermostat mode. Domoticz doesn't update any values when I change them with the buttons or with the webinterface of the TRV.
With my other older z-wave valves I can pick a temperature with the buttons and Domoticz uses an external temperature sensor to control the valves. After doing some scripting I don't need my phone to control them but any user in the room can ask for heat using the buttons. Ideally I can get this to work with the TRV's as well.

Posted: **Tuesday 21 November 2023 18:05**

toxwa wrote: ↑Tuesday 21 November 2023 16:53 Thank you very much! I added the 2 lines and can controll the valve position from Domoticz now so I can work with that.
But the only thing that doesn't work is if I change the opening with the buttons on the Shelly TRV, Domoticz doens't update the valve opening percentage. It's the same if I put them in thermostat mode. Domoticz doesn't update any values when I change them with the buttons or with the webinterface of the TRV.
With my other older z-wave valves I can pick a temperature with the buttons and Domoticz uses an external temperature sensor to control the valves. After doing some scripting I don't need my phone to control them but any user in the room can ask for heat using the buttons. Ideally I can get this to work with the TRV's as well.

You should check the incoming MQTT data when you change the valve percentage with buttons. I do not have a TRV, so i can only read the docs, which says its under ~/info in a JSON as thermostats[0].pos. If its somewhere else, than the config object has to changed according to it.

Posted: **Tuesday 21 November 2023 21:10**

I'm new to MQTT and JSON, but this is an example of "info" from a TRV after I changed the valve opening to 30% using the buttons.

Code: Select all

info = {
    "wifi_sta": {
        "connected": true,
        "ssid": "***********",
        "ip": "************",
        "rssi": -56
    },
    "cloud": {
        "enabled": false,
        "connected": false
    },
    "mqtt": {
        "connected": true
    },
    "time": "19:59",
    "unixtime": 1700593194,
    "serial": 0,
    "has_update": false,
    "mac": "*************",
    "cfg_changed_cnt": 0,
    "actions_stats": {
        "skipped": 0
    },
    "thermostats": [
        {
            "pos": 30. 0,
            "target_t": {
                "enabled": false,
                "value": 25. 0,
                "value_op": 8. 0,
                "units": "C"
            },
            "tmp": {
                "value": 19. 1,
                "units": "C",
                "is_valid": true
            },
            "schedule": false,
            "schedule_profile": 1,
            "boost_minutes": 0,
            "window_open": false
        }
    ],
    "calibrated": true,
    "bat": {
        "value": 100,
        "voltage": 4. 123
    },
    "charger": false,
    "update": {
        "status": "unknown",
        "has_update": false,
        "new_version": "20231009-064451/v2. 2. 1@1e8f2795",
        "old_version": "20231009-064451/v2. 2. 1@1e8f2795",
        "beta_version": null
    },
    "ram_total": 97280,
    "ram_free": 22488,
    "fs_size": 65536,
    "fs_free": 59472,
    "uptime": 12278,
    "fw_info": {
        "device": "shellytrv-CC86ECB3E3E5",
        "fw": "20231009-064451/v2. 2. 1@1e8f2795"
    },
    "ps_mode": 0,
    "dbg_flags": 0
}

The payload I copied from your post was:

Code: Select all

payload={
    "name": "%shelly_id% Valve Set Position 1",
    "enabled_by_default": false,
    "cmd_t": "~thermostat/0/command/valve_pos",
    "stat_t": "~info",
    "pos_t": "~info",
    "set_pos_t": "~thermostat/0/command/valve_pos",
    "pos_tpl": "{{ value_ json .thermostats[0].pos }}",
    "pos_open": 100, "pos_clsd": 0,
    "avty_t": "~online",
    "pl_avail": "true",
    "pl_not_avail": "false",
    "uniq_id": "%shelly_id%-pos-1",
    "qos": 1,
    "~": "shellies/%shelly_id%/"
    }

I'm assuming I need to change something to the "pos_t" : "~info" part, but I'm not familiar with the syntax.
note: I had to add some spaces after dots and underscores in the code to avoid the "You can't post image, email or url links that are external to this domain"-error

Posted: **Thursday 23 November 2023 9:57**

So far I have only used Shelly devices from 1st generation. They were automatically recognized without any problems using “Shelly MQTT”.

Now I have the first “Plus” device of the 2nd generation and would use “Shelly Teacher” for it.

Question: Will all old 1st generation devices be recognized again by the teacher and then appear twice in the device list?

Posted: **Thursday 23 November 2023 16:44**

I have added a shelly plus 1pm switch and I got 4switxhes in domoticz, but unfortunately I can only run the switch on, not off, any idea what could be wrong?

I also have a shelly flood sensor paired with the shelly app in my phone, but this device won't be detected in the teacher app, do I need to reset and reconfigure the flood sensor for the teacher app to see it?

Posted: **Thursday 23 November 2023 17:14**

tjabas wrote: ↑Thursday 23 November 2023 16:44 I have added a shelly plus 1pm switch and I got 4switxhes in domoticz, but unfortunately I can only run the switch on, not off, any idea what could be wrong?

Please describe exactly how did you add that. Did you follow the video tutorial and checked every checkbox at the MQTT panel on the device side?
If you did not enabled "Generic status update over MQTT" than it wont work, for sure.

tjabas wrote: ↑Thursday 23 November 2023 16:44 I also have a shelly flood sensor paired with the shelly app in my phone, but this device won't be detected in the teacher app, do I need to reset and reconfigure the flood sensor for the teacher app to see it?

If MQTT is not enabled, than it wont be found by the Teacher, as it is MQTT only. As you can see in the video, it is important to enable MQTT on the device side.

Posted: **Thursday 23 November 2023 17:17**

imautohuttraeger wrote: ↑Thursday 23 November 2023 9:57 So far I have only used Shelly devices from 1st generation. They were automatically recognized without any problems using “Shelly MQTT”.

Now I have the first “Plus” device of the 2nd generation and would use “Shelly Teacher” for it.

Question: Will all old 1st generation devices be recognized again by the teacher and then appear twice in the device list?

If you uncheck Gen1 checkbox, before pressing "Start detection loop" button, than only Gen2 devices will be added by the Teacher.
If you leave Gen1 checked, then it will be duplicated.
This setting is also available by editing settings.py and changing to "gen1 = False"

Posted: **Thursday 23 November 2023 17:41**

toxwa wrote: ↑Tuesday 21 November 2023 21:10 I'm new to MQTT and JSON, but this is an example of "info" from a TRV after I changed the valve opening to 30% using the buttons.
Code: Select all
info = {
    "thermostats": [
        {
            "pos": 30. 0,
            "target_t": {
                "enabled": false,
                "value": 25. 0,
                "value_op": 8. 0,
                "units": "C"
            },
}
I'm assuming I need to change something to the "pos_t" : "~info" part, but I'm not familiar with the syntax.

Code: Select all

"pos_t": "~info"

just says that the incoming position data structure located under the device /info topic, which seems fine, if it is under shellies/shellytrv-xxxx/info

Code: Select all

"pos_tpl": "{{ value_json.thermostats[0].pos }}"

says that the data is in JSON format, and in an array named thermostats at index 0 and the name of the property is "pos".
The structure seems fine. In the matter of fact, i have created a virtual sensor on my Domoticz 2023.2 (build 15601), and send it to the info payload you showed and it works, the Cover goes to 30%.
What Domoticz version do you use exactly? TRV only supported after 2023 beta 15515. Did you enabled the new Cover device in Domoticz after creation?

toxwa wrote: ↑Tuesday 21 November 2023 21:10 note: I had to add some spaces after dots and underscores in the code to avoid the "You can't post image, email or url links that are external to this domain"-error

Okay, just make sure you do not add spaces between . _ and other parts in the template file.

Posted: **Thursday 23 November 2023 17:44**

I followed the guide last night, I guess that didn't tic all options, I'll have to chech it tonight after work.
But how about the flood sensor, does that one also have web interface?

Posted: **Thursday 23 November 2023 17:49**

tjabas wrote: ↑Thursday 23 November 2023 17:44 I followed the guide last night, I guess that didn't tic all options, I'll have to chech it tonight after work.
But how about the flood sensor, does that one also have web interface?

I dont have flood sensor, but every Shelly sensor has a web interface. Although battery operated devices can only be accessed when a button on them is pressed, as they are in deepsleep otherwise. (unless there is an alarm, for example: flood)
See Flood User Guide:
https://kb.shelly.cloud/__attachments/6 ... 9064a7cc66

(web GUI is available at 192.168.33.1 before initializing and adding the device to the app, but the GUI is also available on standard port 80 after initialization, although the IP address has to be found first - on your router which serves DHCP addresses, or there is a shelly device scanner: https://www.usna.it/shellyscanner/index_en.html)

Posted: **Thursday 23 November 2023 20:52**

lwolf wrote: ↑Thursday 23 November 2023 17:41 The structure seems fine. In the matter of fact, i have created a virtual sensor on my Domoticz 2023.2 (build 15601), and send it to the info payload you showed and it works, the Cover goes to 30%.
What Domoticz version do you use exactly? TRV only supported after 2023 beta 15515. Did you enabled the new Cover device in Domoticz after creation?

This is the problem. I'm on the latest stable, because I haven't migrated my openzwave-devices tot Zwave-JS-UI yet. I can't upgrade before making sure my Z-wave setup keeps working, but that's another project. Thanks for your help.

Posted: **Thursday 23 November 2023 22:45**

lwolf wrote: ↑Thursday 23 November 2023 17:17
imautohuttraeger wrote: ↑Thursday 23 November 2023 9:57 So far I have only used Shelly devices from 1st generation. They were automatically recognized without any problems using “Shelly MQTT”.

Now I have the first “Plus” device of the 2nd generation and would use “Shelly Teacher” for it.

Question: Will all old 1st generation devices be recognized again by the teacher and then appear twice in the device list?
If you uncheck Gen1 checkbox, before pressing "Start detection loop" button, than only Gen2 devices will be added by the Teacher.
If you leave Gen1 checked, then it will be duplicated.
This setting is also available by editing settings.py and changing to "gen1 = False"

Thx a lot!

Posted: **Tuesday 28 November 2023 16:41**

I've used Domoticz and Shelly devices since 2017 and it worked greated. After Domoticz was not supported for the Synology (without docker) anymore, I created a new installation on a Ubuntu platform. The Domoticz stable version 2023.2 (15457) is running without issues. The database from the older version is re-used but beacuse the Shelly devices are on a new broker, I had to add them via Shellyteacher.

All devices are added, including 4 pieces of Shellyswitches 2.5 to open and close shutters. 3 of them work fine but 1 isn't.
Domoticz reads al information from Shellyteacher and I can see that data is shared (for instance the temperature is updated). But the rollers won't open and close. Status in domoticz gives -1% and seems abnormal When I connect directly to the device (shelly app or url) it works proparly with no issues.

In what part are thinks going wrong (domoticz, Mosquitto, Teacher, database)? I re-added the devive with shelly teacher (got a new IDX). Domoticz logging does't give a clue. All items are the latest versions.

Hope to hear

Posted: **Tuesday 28 November 2023 19:10**

Jonkheer wrote: ↑Tuesday 28 November 2023 16:41 All devices are added, including 4 pieces of Shellyswitches 2.5 to open and close shutters. 3 of them work fine but 1 isn't.
Domoticz reads al information from Shellyteacher and I can see that data is shared (for instance the temperature is updated). But the rollers won't open and close. Status in domoticz gives -1% and seems abnormal When I connect directly to the device (shelly app or url) it works proparly with no issues.

If i understand correctly 3 from 4 device works correctly and 1 is not.
Without knowing what is the exact setup and difference of these units, i dont think i can do anything for you.
ShellyTeacher sends the same config for same device types, so if one is not working, you can try these steps on the non-working device:
1. Update device firmware to latest
2. Do a full factory reset
3. Delete device from Domoticz
4. Setup mqtt again on the device
5. Run the teacher again

The Teacher sends this config payload for Shelly2/2.5 devices:

Code: Select all

{"name": "%shelly_id% Roller 0", "cmd_t": "~roller/0/command", "stat_t": "~roller/0", "pos_t": "~roller/0/pos", "set_pos_t": "~roller/0/command/pos", "avty_t": "~online", "pl_avail": "true", "pl_not_avail": "false", "json_attr_t": "~info", "pl_open": "open", "pl_cls": "close", "pl_stop": "stop", "stat_opening": "open", "stat_closing": "close", "pos_open": 100, "pos_clsd": 0, "uniq_id": "%shelly_id%-roller-0", "qos": 1, "~": "shellies/%shelly_id%/"}

Which means it awaits current state into shellies/shelly2-xxx/roller/0 MQTT topic and current position into shellies/shelly2-xxx/roller/0/pos
-1% means it have difficulties to get the position. Or some data arrived which seems invalid for Domoticz..
You can check its values with MQTT Explorer for debugging.

What you can check if nothing helps:
- make sure you are not using special characters in device name/unique topic on the shelly
- try to disable inversion, if you enabled it

Posted: **Tuesday 28 November 2023 19:34**

Thanks Iwolf for your quick response!

lwolf wrote: ↑Tuesday 28 November 2023 19:10 If i understand correctly 3 from 4 device works correctly and 1 is not.

Thats correct.

Ik will try a factory reset on the Shelly device.
There are no strange special characters and I disabled inversion (it was enabled!)

When you set the device to a roller opening of (for eaxampe) 35% it holds that level for aproximatly 3 seconds. Status will be updated bij domoticz and set back to -1%.

I'll post the results.

Posted: **Wednesday 29 November 2023 17:42**

Jonkheer wrote: ↑Tuesday 28 November 2023 19:34 When you set the device to a roller opening of (for eaxampe) 35% it holds that level for aproximatly 3 seconds. Status will be updated bij domoticz and set back to -1%.

I'll post the results.

This 3 seconds sounds like Domoticz receiving back from "~roller/0/pos" an invalid position with a strange value. Or maybe "~roller/0" status report something other than open/close/stop.. maybe overpower?

All right, waiting for the MQTT debugging data!

Domoticz

ShellyTeacher for Domoticz

ShellyTeacher for Domoticz

Re: ShellyTeacher for Domoticz

Re: ShellyTeacher for Domoticz

Re: ShellyTeacher for Domoticz

Re: ShellyTeacher for Domoticz

Re: ShellyTeacher for Domoticz

Re: ShellyTeacher for Domoticz

Re: ShellyTeacher for Domoticz

Re: ShellyTeacher for Domoticz

Re: ShellyTeacher for Domoticz

Re: ShellyTeacher for Domoticz

Re: ShellyTeacher for Domoticz

Re: ShellyTeacher for Domoticz

Re: ShellyTeacher for Domoticz

Re: ShellyTeacher for Domoticz

Re: ShellyTeacher for Domoticz

Re: ShellyTeacher for Domoticz

Re: ShellyTeacher for Domoticz

Re: ShellyTeacher for Domoticz

Re: ShellyTeacher for Domoticz