Python Plugin: Shelly MQTT

[lwolf]

If anyone interested, I've made a Python plugin for Shelly MQTT integration into Domoticz. Currently I have only got a Shelly1 device to test, but more will come soon.
I would like to thank the zigbee2mqtt project for it's nice MQTT library that i reused.
https://github.com/enesbcs/Shelly_MQTT

[minime]

Great, thats something I'm looking. My shelly devices will arrive soon (hope so). So I will test this soon.
Because I want to use Shelly2 for shutter control, do you think theres a way to get them support soon, too?

Thanks alot in advance
Georg

[lwolf]

Great, thats something I'm looking. My shelly devices will arrive soon (hope so). So I will test this soon.
Because I want to use Shelly2 for shutter control, do you think theres a way to get them support soon, too?

My Shelly2 arrived yesterday, so i think roller shutter will be supported within a few days.

update: roller shutter is supported

[minime]

Great job. Hopefully my devices will arrive this week. Can't wait for testing them.

Thanks alot for your work!

[gianfrdp]

Hello, I tried you plugin for a Shelly1.
I have Domoticz v3.8805 on linux amd_x64 with mosquitto on board and python version

Code: Select all

python --version
Python 2.7.9

I cloned repository into

Code: Select all

/opt/domoticz/plugins/Shelly_MQTT

Restarted domoticz.

I crated hw with mqtt server 127.0.0.1:1883.

Looking on domticz log I have

Code: Select all

2018-12-03 12:16:28.464  (Shelly) Calling message handler 'onHeartbeat'.
2018-12-03 12:16:28.464  (Shelly) Heartbeating...
2018-12-03 12:16:28.464  (Shelly) Reconnecting
2018-12-03 12:16:28.464  (Shelly) MqttClient::Open
2018-12-03 12:16:28.464  (Shelly) MqttClient::Close
2018-12-03 12:16:28.464  (Shelly) Protocol set to: 'MQTT'.
2018-12-03 12:16:28.464  (Shelly) Transport set to: 'TCP/IP', 127.0.0.1:1883.
2018-12-03 12:16:28.464  (Shelly) Connect directive received, action initiated successfully.
2018-12-03 12:16:28.514  (Shelly) Calling message handler 'onConnect'.
2018-12-03 12:16:28.514  (Shelly) MQTT connected successfully.
2018-12-03 12:16:28.514  (Shelly) MqttClient::Connect
2018-12-03 12:16:28.514  (Shelly) MQTT CONNECT ID: 'Domoticz_1543835788'
2018-12-03 12:16:28.515  Error: (ProcessOutbound) Send request Python object parameter was not of type Unicode or Byte, ignored.
2018-12-03 12:16:28.515  (Shelly) Sending 0 bytes of data:
2018-12-03 12:16:28.965  (Shelly) Disconnect directive received for '127.0.0.1:1883'.
2018-12-03 12:16:28.965  (Shelly): CPluginTransportTCP::handleDisconnect
2018-12-03 12:16:29.015  (Shelly) Calling message handler 'onDisconnect'.
2018-12-03 12:16:29.016  (Shelly) MqttClient::onDisonnect Disconnected from: 127.0.0.1:1883
2018-12-03 12:16:29.016  (Shelly) MqttClient::Close
2018-12-03 12:16:29.016  (Shelly) onMQTTDisconnected
2018-12-03 12:16:29.466  (Shelly) Disconnect directive received for '127.0.0.1:1883'.
2018-12-03 12:16:29.466  (Shelly): CPluginTransportTCP::handleDisconnect
2018-12-03 12:16:29.517  (Shelly) Calling message handler 'onDisconnect'.
2018-12-03 12:16:29.517  (Shelly) MqttClient::onDisonnect Disconnected from: 127.0.0.1:1883
2018-12-03 12:16:29.517  (Shelly) MqttClient::Close
2018-12-03 12:16:29.517  (Shelly) onMQTTDisconnected
2018-12-03 12:16:38.479  (Shelly) Calling message handler 'onHeartbeat'.
2018-12-03 12:16:38.479  (Shelly) Heartbeating...
2018-12-03 12:16:38.479  (Shelly) Reconnecting
2018-12-03 12:16:38.479  (Shelly) MqttClient::Open
2018-12-03 12:16:38.479  (Shelly) Protocol set to: 'MQTT'.
2018-12-03 12:16:38.479  (Shelly) Transport set to: 'TCP/IP', 127.0.0.1:1883.
2018-12-03 12:16:38.479  PluginSystem: Starting I/O service thread.
2018-12-03 12:16:38.479  (Shelly) Connect directive received, action initiated successfully.
2018-12-03 12:16:38.529  (Shelly) Calling message handler 'onConnect'.
2018-12-03 12:16:38.529  (Shelly) MQTT connected successfully.
2018-12-03 12:16:38.529  (Shelly) MqttClient::Connect
2018-12-03 12:16:38.529  (Shelly) MQTT CONNECT ID: 'Domoticz_1543835798'
2018-12-03 12:16:38.530  Error: (ProcessOutbound) Send request Python object parameter was not of type Unicode or Byte, ignored.
2018-12-03 12:16:38.530  (Shelly) Sending 0 bytes of data:

On mosquitto

Code: Select all

1543832868: New connection from 192.168.2.40 on port 1883.
1543832868: New client connected from 192.168.2.40 as shelly1-4A5936 (c1, k60).

How can I solve it?

Thanks

[lwolf]

Hello, I tried you plugin for a Shelly1.
I have Domoticz v3.8805 on linux amd_x64 with mosquitto on board and python version

Code: Select all

python --version
Python 2.7.9

How can I solve it?

Thanks

Please install Python 3.4 or newer following the Domoticz guide:
https://www.domoticz.com/wiki/Using_Python_plugins

[gianfrdp]

Sorry, I checked on domoticz log: I have

Code: Select all

2018-12-11 15:43:32.794 PluginSystem: Started, Python version '3.4.2'.

Code: Select all

python3 --version
Python 3.4.2

[lwolf]

Sorry, I checked on domoticz log: I have

Code: Select all

2018-12-11 15:43:32.794 PluginSystem: Started, Python version '3.4.2'.

Code: Select all

python3 --version
Python 3.4.2

I've downgraded my Python 3.6.6 to the oldest possible 3.4.9 version, but i am unable to reproduce this error.
"Error: (ProcessOutbound) Send request Python object parameter was not of type Unicode or Byte, ignored."

I assume you purged the previously installed Python 2.x to not interfere with v3.4?

My test environment is producing the logs below with Verbose debug output and works fine:

Code: Select all

2018-12-11 20:25:55.567 Status: Domoticz V4.9701 (c)2012-2018 GizMoCuz
2018-12-11 20:25:55.868 Status: PluginSystem: Started, Python version '3.4.9'.
2018-12-11 20:25:58.384 (shelly) Debug logging mask set to: PYTHON PLUGIN DEVICE CONNECTION MESSAGE
2018-12-11 20:25:58.384 (shelly) MqttClient::__init__
2018-12-11 20:25:58.384 (shelly) MqttClient::Open
2018-12-11 20:25:58.384 (shelly) Protocol set to: 'MQTT'.
2018-12-11 20:25:58.384 (shelly) Transport set to: 'TCP/IP', 192.168.2.100:1883.
2018-12-11 20:25:58.384 PluginSystem: Starting I/O service thread.
2018-12-11 20:25:58.384 (shelly) Connect directive received, action initiated successfully.
2018-12-11 20:25:58.434 (shelly) MQTT connected successfully.
2018-12-11 20:25:58.435 (shelly) MqttClient::Connect
2018-12-11 20:25:58.435 (shelly) MQTT CONNECT ID: 'Domoticz_1544556358'
2018-12-11 20:25:58.435 (shelly) Sending 33 bytes of data
2018-12-11 20:25:58.435 (shelly) 10 1f 00 04 4d 51 54 54 04 02 00 3c 00 13 44 6f 6d 6f 74 69 ....MQTT...<..Domoti
2018-12-11 20:25:58.435 (shelly) 63 7a 5f 31 35 34 34 35 35 36 33 35 38 .. .. .. .. .. .. .. cz_1544556358
2018-12-11 20:25:58.485 (shelly) Received 4 bytes of data
2018-12-11 20:25:58.485 (shelly) 20 02 00 00 .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ....
2018-12-11 20:25:58.485 (shelly) MqttClient::Subscribe
2018-12-11 20:25:58.485 (shelly) Sending 17 bytes of data
2018-12-11 20:25:58.485 (shelly) 82 0f 00 01 00 0a 73 68 65 6c 6c 69 65 73 2f 23 00 .. .. .. �.....shellies/#.
2018-12-11 20:25:58.535 (shelly) Received 5 bytes of data
2018-12-11 20:25:58.536 (shelly) 90 03 00 01 00 .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. �....
2018-12-11 20:25:58.536 (shelly) onMQTTSubscribed
2018-12-11 20:25:58.342 Status: PluginSystem: Entering work loop.
2018-12-11 20:25:58.382 Status: (shelly) Initialized version 0.2.0
2018-12-11 20:25:58.384 Status: (shelly) Entering work loop.
2018-12-11 20:26:07.934 (shelly) Heartbeating...
2018-12-11 20:26:07.934 (shelly) MqttClient::Ping
2018-12-11 20:26:07.934 (shelly) Sending 2 bytes of data
2018-12-11 20:26:07.934 (shelly) c0 00 .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. �.
2018-12-11 20:26:07.985 (shelly) Received 2 bytes of data
2018-12-11 20:26:07.985 (shelly) d0 00 .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. �. 

My best guess is that somewhere between Domoticz v3.8805 (2017?) and current stable Domoticz v4.9701 (2018-06-23) the Domoticz API changed somewhat, that causes errors.
Domoticz.Connection() object has a Send() method and that is raising the error you sent.
However the mqtt.py that I've reused is in use by several other plugins, and i do not see any controversy that's code and the Domoticz Python documentation - which is not too long, neither talkative.
https://www.domoticz.com/wiki/Developin ... hon_plugin
Can you use any other MQTT or HTTP based Python plugin on your installation?

Do you have any specific reason for sticking old softwares, such as 4 year old python and Domoticz 3.8? I do not have this version, and do not know, how to downgrade to it, so the further debugging seems impossible to me.

You can try to change this code in mqtt.py from this:

Code: Select all

     else:
            ID = 'Domoticz_'+str(int(time.time()))
            Domoticz.Debug("MQTT CONNECT ID: '" + ID + "'")
            self.mqttConn.Send({'Verb': 'CONNECT', 'ID': ID})

To this:

Code: Select all

     else:
         try:
            ID = 'Domoticz_'+str(int(time.time()))
            Domoticz.Debug("MQTT CONNECT ID: '" + ID + "'")
            self.mqttConn.Send({'Verb': 'CONNECT', 'ID': ID})
         except Exception as e:
            Domoticz.Debug("Unknown connection error: "+str(e))

Then restart Domoticz and you will see more in the logs..

[gianfrdp]

Hello,
I have a Debian Jessie on a OpenMediaVault server, which provides

Code: Select all

python                                2.7.9-1
python3                               3.4.2-2

I installed a separate python3.5 on /usr/local, I have to understand how to force Domoticz to use it.

I will try also your suggested changes

I downgraded Domoticz to v3.8805 because last versions crash with signal 6.
I have no other python plugin.

Thanks

EDIT:

I tried another python plugin https://github.com/emontnemery/domoticz_mqtt_discovery with same error.
Searching into domoticz source files I found in https://github.com/domoticz/domoticz/bl ... tocols.cpp point where error message is printed

Code: Select all

        std::vector<byte> CPluginProtocol::ProcessOutbound(const WriteDirective* WriteMessage)
        {
                std::vector<byte>       retVal;

                // Handle Bytes objects
                if ((((PyObject*)WriteMessage->m_Object)->ob_type->tp_flags & (Py_TPFLAGS_BYTES_SUBCLASS)) != 0)
                {
                        const char*     pData = PyBytes_AsString(WriteMessage->m_Object);
                        int                     iSize = PyBytes_Size(WriteMessage->m_Object);
                        retVal.reserve((size_t)iSize);
                        retVal.assign(pData, pData + iSize);
                }
                // Handle ByteArray objects
                else if ((((PyObject*)WriteMessage->m_Object)->ob_type->tp_name == std::string("bytearray")))
                {
                        size_t  len = PyByteArray_Size(WriteMessage->m_Object);
                        char*   data = PyByteArray_AsString(WriteMessage->m_Object);
                        retVal.reserve(len);
                        retVal.assign((const byte*)data, (const byte*)data + len);
                }
                // Handle String objects
                else if ((((PyObject*)WriteMessage->m_Object)->ob_type->tp_flags & (Py_TPFLAGS_UNICODE_SUBCLASS)) != 0)
                {
                        std::string     sData = PyUnicode_AsUTF8(WriteMessage->m_Object);
                        retVal.reserve((size_t)sData.length());
                        retVal.assign((const byte*)sData.c_str(), (const byte*)sData.c_str() + sData.length());
                }
                else
                        _log.Log(LOG_ERROR, "(%s) Send request Python object parameter was not of type Unicode or Byte, ignored.", __func__);

                return retVal;
        }

so I think it should be a general problem with domoticz version.

I will try to upgade domoticz and see what appens.

Thanks

[midomot]

Hello
I want to thank you so very very much for this plugin!

I received my shelly2 yesterday, today I made a mock-up installation to test
I found your plugin, and it now all works, all the way to Domoticz's !

Good to know:

This line below was incorrect, mqtt server 1883 can be used, no problem

Spoiler: show

- if you already have a mqtt server running on port 1883, you need to set another port in both Shelly's interface and Domoticz's for this Shelly_MQTT
If you don't do that.. you get a bunch of errors !

- after above is done, don't worry if you see still errors going by in the log (I got worried...) - just follow the instructions from lwolf : you need to get the devices automatically created ! so authorize in Domoticz's settings, manipulate the switches, and voila: you'll have your switches created, and errors will be gone !
Now you can set it all up in Domoticz as you'll like

Finally,
- if like me you're using the Shelly2 behind a double switch (one switch per lamp), you will want to go into the individual button settings in Shelly's interface and change button type to "Edge Switch"

you may also want to update the "power on mode" - I need to check that one out myself.

[lwolf]

- if you already have a mqtt server running on port 1883, you need to set another port in both Shelly's interface and Domoticz's for this Shelly_MQTT
If you don't do that.. you get a bunch of errors !

You are welcome. Could you send the error messages, that you got? I am using the Shelly's on the same MQTT broker and port 1883 as my other Tasmota devices, and i did not saw any interesting yet.
Errors maybe pop up when it tries to create a new device, that just found, and automatic device creation is disabled?

[midomot]

I think I owe you an apology !
Not sure what happened, but it seems like I had set Port back to 1883 for this plugin - and it worked just fine for last couple days on my test-setup !

I now changed the port to 1884 (on shelly and in domoticz) and got

(Shelly_MQTT) Failed to connect to: 192.168.178.3:1884, Description: Connection refused

So better leave it with standard MQTT port 1883
I probably confused myself and saw a problem where there was none

One thing I did notice: in hardware page, the port number is not visible on the Shelly_MQTT line
When that line is selected, the port is however properly visible in the lower part where settings can be adjusted

Domoticz V4.10276
Pi with Raspbian Jessie Lite
Python V3.4.2 installed

[nabru99]

hello,
great plugin- i tested with shelly 2 in roller mode.

it works perfect.

is it possible to use slider and percentage -blinds percentage-as in web config-shelly?

it is usefull for roller shutter to set position.


U

[nabru99]

hello,
great plugin- i tested with shelly 2 in roller mode.

it works perfect.

is it possible to use slider and percentage -blinds percentage-as in web config-shelly?

it is usefull for roller shutter to set position.


U

New in v1.4.0: roller mode position control. For this to work, the device must first be successfully calibrated, so that the time it takes for closing and opening are known.

shellies/shellyswitch-<deviceid>/roller/0/pos reports the current position in percent
shellies/shellyswitch-<deviceid>/roller/0/command/pos accepts a number between 0 and 100, which is target position in percent.

[lwolf]

New in v1.4.0: roller mode position control. For this to work, the device must first be successfully calibrated, so that the time it takes for closing and opening are known.

shellies/shellyswitch-<deviceid>/roller/0/pos reports the current position in percent
shellies/shellyswitch-<deviceid>/roller/0/command/pos accepts a number between 0 and 100, which is target position in percent.

Someday, when i have a little spare time, it can be upgraded but it will be a little tricky to me to calibrate, as i do not have any blind device or motor to calibrate it.

[lwolf]

is it possible to use slider and percentage -blinds percentage-as in web config-shelly?

it is usefull for roller shutter to set position.

I am updated the plugin, but i am not sure, that it is working correctly, as i am unable to calibrate Shelly2 without a real device. Waiting for feedbacks.

[nabru99]

is it possible to use slider and percentage -blinds percentage-as in web config-shelly?

it is usefull for roller shutter to set position.

I am updated the plugin, but i am not sure, that it is working correctly, as i am unable to calibrate Shelly2 without a real device. Waiting for feedbacks.

EDITED:

Domoticz finds new device- BLINDS PERCENTAGE this is OK


When percentage is used it shoud be:
nValue 0 and sValue 0 ==>> Closed
nValue 2 and sValue 2-99 ==>> partially opened
nValue 1 and sValue 100 ==>> Opened

Status with ShellyMQTT:
0% status is OPEN
100% status is OPEN

Slider is fully working - position is correct.

So only status is wrong.

U

[lwolf]

When percentage is used it shoud be:
nValue 0 and sValue 0 ==>> Closed
nValue 2 and sValue 2-99 ==>> partially opened
nValue 1 and sValue 100 ==>> Opened

Status with ShellyMQTT:
0% status is OPEN
100% status is OPEN

Slider is fully working - position is correct.

So only status is wrong.

U

Thank you, I see. I do not know, that nvalue is used at blind percentage.

I think i fixed, please test it.

[nabru99]

When percentage is used it shoud be:
nValue 0 and sValue 0 ==>> Closed
nValue 2 and sValue 2-99 ==>> partially opened
nValue 1 and sValue 100 ==>> Opened

Status with ShellyMQTT:
0% status is OPEN
100% status is OPEN

Slider is fully working - position is correct.

So only status is wrong.

U

Thank you, I see. I do not know, that nvalue is used at blind percentage.

I think i fixed, please test it.

closed is 100% Working
position 1-99% Working
open 0% Not working relay not triggered

i also teste reverse :
100 open
0 closed

also not working...

[lwolf]

closed is 100% Working
position 1-99% Working
open 0% Not working relay not triggered

i also teste reverse :
100 open
0 closed

also not working...

Please be a little more specific, what is not working.

When you issue "stop" command at the "VENETIAN Blind" device the plugin forwards "stop", if "off" then "open", if "on" then "close" message to the original topic:
shellies/shellyswitch-<deviceid>/roller/0/command

If you issue SET LEVEL command to the "Blind Percentage" device than the plugin forwards the percentage to the new topic:
shellies/shellyswitch-<deviceid>/roller/0/command/pos

There are two icon at the percentage bar at left, this is what not working? I guess you thinking about it.

And the plugin has to execute "stop" after reaching 0%? What needs to be triggered at 0%? I think 0 means closed, and Shelly doing the other things at the background.