Breaking change for plugins that use switches with options

[oohlaf]

Hi all,

The latest beta as of today contains a breaking change for plugin authors that use selectors with customer level names.

The previous build used base64 encoded string as options variable, the new code is a Python dictionary.
Example:

Code: Select all

       Options = {"LevelActions": "||||",
                  "LevelNames": "Off|Video|Music|TV Shows|Live TV",
                  "LevelOffHidden": "false",
                  "SelectorStyle": "1"}
       Domoticz.Device(Name="Source", Unit=2, \
                       TypeName="Selector Switch", Options=Options).Create()

You can remove the base64 encode/decode helper.

The options dictionary is now also exposed as read-only variable.

In the update method you can supply a new dictionary of options in case you need to modify it. The parameter is optional and defaults to an empty dictionary, as it does not happen often to adapt it after device creation.

[jorgh]

@Oohlaf,

I absolutely like the new method for creating and reading the options.
However I noticed that the old method now seems broken, I don't know if that was intentional.
I'm not sure it was this specific change that 'broke' my plugin, as I switched from the plugins to the master branch, there might be other changes involved. I'll look into this later today.

Regards,

Jorg

[oohlaf]

Correct, as the Python plugin system is not part of a stable Domoticz release yet, I replaced the previous functionality with this new one.

[jorgh]

Ok, clear answer, I won't have to look into that one.

[Dnpwwo]

@Oohlaf,

I like your change but can you please update the wiki as well. No point having great functionality if people don't know how to use it

[zaraki673]

@Oohlaf,

I like your change but can you please update the wiki as well. No point having great functionality if people don't know how to use it

that's have been done by Oohlaf in the example script

Thk's Oolhaf, work great, make me cry sunday after upgrading to lastest beta, while i was working on a new plugin since a week

[Dnpwwo]

Sorry, I should have been more clear.

I meant the documentation here: http://www.domoticz.com/wiki/Developing ... in#Devices

[oohlaf]

Already did so last weekend
https://www.domoticz.com/wiki/index.php ... oldid=8211

[jorgh]

@oohlaf,

I've come across an issue, that if I update a selector with a new level and immediately select the level I added, Domoticz tents to lockup.

I use the following code to update the selector:

Code: Select all

def addListeningMode(strCode):
  nValue = Devices[MAINLISTENINGMODE].nValue
  sValue = Devices[MAINLISTENINGMODE].sValue
  dictOptions = Devices[MAINLISTENINGMODE].Options
  dictOptions["LevelNames"] = dictOptions["LevelNames"]+'|['+strCode+']'+' New'
  dictOptions["LevelActions"] = dictOptions["LevelActions"]+'|'

  Devices[MAINLISTENINGMODE].Update(nValue = nValue, sValue = sValue, Options = dictOptions)

I select the value using the following call:

Code: Select all

Devices[intId].Update(1,str(intLevel))

In the webinterface, I can see the control is updated in the header of the device, but as the list is not yet updated, the proper value is not selected in the list.
If I restart Domoticz the device control is properly updated, and selecting the level created by the code above, works from that moment on.

Is this a bug, or do I need to add some additional code to refresh the web interface?

Kind regards,

Jorg

[oohlaf]

Hi, am reading this on my phone so a bit difficult to see the code properly formatted. Your levelname formatting looks a bit weird. Strcode in brackets with new appended?

Other than that is looks ok. Domoticz should not hang by executing the python code.

That the web interface does not update is known to me. I don't think you can trigger the browser to reload.

I don't have much time to look into it though.

Verstuurd vanaf mijn XT1562 met Tapatalk

[jorgh]

@oohlaf,

Your levelname formatting looks a bit weird. Strcode in brackets with new appended?

strCode, is a string containing a hexadecimal value, so the formatting of the levelname is '[0C] New'. So the square brackets have no other function that aesthetics.

From what I've experienced, Domoticz often locks-up when a python plugin throws an exception, as the log is in the webinterface, the throwing of the exception is not always visible. So my best guess is, that the selection of the newly added level causes an issue. I'm trying to get the python debugging framework working, but that also seems to cause lockup's at this time if I interrupt the excecution of the plugin. Maybe caused by the heartbeat event that is called by Domoticz when the execution of the plugin is actually paused. I'll look into that later.

Kind regards,

Jorg

[jorgh]

@oohlaf,

I've investigated a bit further. Domoticz actually crashes (segfaults) when trying to select the level just added. It seems to segfault when trying to access the dict containing the levelnames. I've put the code and logging below:

Code: Select all

def setSelectorByCode(intId, strCode):
  Domoticz.Log("Onkyo: setSelectorByCode code: "+strCode)
  dictOptions = Devices[intId].Options
  Domoticz.Log("Onkyo: Fetched Options")
  Domoticz.Log("Onkyo: options: "+dictOptions['LevelNames'])

The following is logged:

Code: Select all

2017-05-12 11:21:55.270  (Onkyo) Onkyo: setSelectorByCode code: 80
2017-05-12 11:21:55.272  (Onkyo) Onkyo: Fetched Options
2017-05-12 11:21:55.273  Error: Domoticz received fatal signal 11 !...
2017-05-12 11:21:55.282  Error:   /home/pi/dev-domoticz/domoticz() [0x1c25c0]
2017-05-12 11:21:55.284  Error:   /home/pi/dev-domoticz/domoticz(_Z14signal_handleri+0x58) [0x1c2678]
2017-05-12 11:21:55.285  Error:   /lib/arm-linux-gnueabihf/libc.so.6(__default_sa_restorer_v2+0) [0xb6b2a180]

This only happens after the level has just been added. After restarting domoticz, the level can be selected and domoticz never crashes.
It seems like the memory referenced by the dict is not valid.

Regards,

Jorg

[Dnpwwo]

@jorgh,

Pushed a fix and merge request (https://github.com/domoticz/domoticz/pull/1634) should be in an upcoming beta

[jorgh]

@Dnpwwo,

Pushed a fix and merge request (https://github.com/domoticz/domoticz/pull/1634) should be in an upcoming beta

Great news, had some more issues with the update method (also when selecting a level) when I changed to the new connection object methods. Once the update is pushed I will do some testing. If it all works, I can release my updated plugin.

Regards,

Jorg

[jorgh]

@Dnpwwo,

I saw a new build was released after merging the pull request. I'm now running V3.7995.
I experienced a new crash when using the same update method for selecting a level. This was already the case before merging your fix, although it worked fine on the version release after the initial 'breaking change' of the selector switch.
As it was a crash when using the same method, I was hoping that it would be resolved too.
It might be, that I'm testing to soon, as I'm not sure how to determine when a pull request is actually available in a new Beta.

I have the following function to update a selector (basically I lookup the name of the level to match it to a level and than updating the switch).

Code: Select all

def setSelectorByName(intId, strName):
  dictOptions = Devices[intId].Options
  listLevelNames = dictOptions['LevelNames'].split('|')
  intLevel = 0
  for strLevelName in listLevelNames:
    if strLevelName == strName:
      Devices[intId].Update(1,str(intLevel))
    intLevel += 10

It results in the following Log:

Code: Select all

2017-06-20 10:59:53.144  (Onkyo - TX-NR646 Main Source) Updating device from 1:'
40' to have values 1:'40'.
2017-06-20 10:59:53.173  Error: Domoticz received fatal signal 11 !...
2017-06-20 10:59:53.182  Error:   /home/pi/dev-domoticz/domoticz() [0x1ccd84]
2017-06-20 10:59:53.183  Error:   /home/pi/dev-domoticz/domoticz(_Z14signal_handleri+0x58) [0x1cce3c]
2017-06-20 10:59:53.184  Error:   /lib/arm-linux-gnueabihf/libc.so.6(__default_sa_restorer_v2+0) [0xb6ad6180]

Am I testing too soon, or is there still a bug somewhere in the code?

Regards,

Jorg

[Dnpwwo]

@jorgh,

I don't think you are testing too soon if the error has changed. I tested with:

Code: Select all

    def onStart(self):
        Options = {"LevelActions": "||||",
                  "LevelNames": "Off|Video|Music|TV Shows|Live TV",
                  "LevelOffHidden": "false",
                  "SelectorStyle": "1"}
        Domoticz.Device(Name="Source", Unit=2, \
                       TypeName="Selector Switch", Options=Options).Create()

    def onHeartbeat(self):
        dictOptions = Devices[2].Options
        dictOptions["LevelNames"] = dictOptions["LevelNames"]+'|[Test]'+' New'
        dictOptions["LevelActions"] = dictOptions["LevelActions"]+'|'
        Devices[2].Update(nValue = 1, sValue = "Test", Options = dictOptions)

and saw additional an level being added successfully every 10 seconds.
What triggers 'setSelectorByName', is it an onCommand call?

[jorgh]

@Dnpwwo,

What triggers 'setSelectorByName', is it an onCommand call?

No, its from onMessage. I receive a frame containing the selected input from the receiver. I match the code for the input with the corresponding configured name. This name is within the receivers configuration XML. The name is also used as the level name, so I use that to lookup the corresponding level and then call to update the selector.

onMessage code:

Code: Select all

def onMessage(self, Connection, Data, Status, Extra):
    self.bInputBuffer = b''.join([self.bInputBuffer, Data])       # We add the received data to the inputbuffer.
    while (self.checkInputBuffer() == True):                      # Check if we have one or more complete frames in the input buffer
      if (self.blDebug ==  True):
        Domoticz.Log('We have a eISCP frame to process')
      self.processeISCPFrame()

Relevant processISCPFrame code:

Code: Select all

if (streISCPData=='!1SLI'):
        if (self.blDebug ==  True):
          Domoticz.Log('Source: '+streISCPMessage)
        for selector in self.XMLRoot.find('device').find('selectorlist'):
          if (selector.get('id').upper() == streISCPMessage.upper()):
            Domoticz.Log('Current Source: '+selector.get('name'))
            setSelectorByName(MAINSOURCE, selector.get('name'))

I posted the setSelectorByName code in the previous post.
I've verified with additional debug logging that it really is the device update function that crashes, and not any surrounding code.

Kind regards,

Jorg

[Dnpwwo]

@jorgh,

I did manage to get Python to abort in Device.Update (I've pushed a fix that will be in a beta soon) but I don't think it was related to your problem.

Can you try running this plugin to see if it works for you? It adds a number of Options after the initial creation then selects one by name. You can also update the device in the web ui and it will update it back. This seems to work as expected for me.

Code: Select all

# Basic Python Plugin Example
#
# Author: GizMoCuz
#
"""
<plugin key="OptPlug" name="Options Plugin" author="gizmocuz" version="1.0.0" wikilink="http://www.domoticz.com/wiki/plugins/plugin.html" externallink="https://www.google.com/">
    <params>
    </params>
</plugin>
"""
import Domoticz

class BasePlugin:
    enabled = False
    def __init__(self):
        return

    def onStart(self):
        if 2 in Devices:
            Devices[2].Delete()
        Options = {"LevelActions": "||||",
                  "LevelNames": "Off|Video|Music|TV Shows|Live TV",
                  "LevelOffHidden": "false",
                  "SelectorStyle": "1"}
        Domoticz.Device(Name="Source", Unit=2, TypeName="Selector Switch", Options=Options, Used=1).Create()
        Domoticz.Log("onStart called")
        Domoticz.Heartbeat(2)

    def onHeartbeat(self):
        Domoticz.Log("onHeartbeat called - "+str(len(Devices[2].Options["LevelNames"])))
        if (len(Devices[2].Options["LevelNames"]) < 80):
            dictOptions = Devices[2].Options
            dictOptions["LevelNames"] = dictOptions["LevelNames"]+'|[Test] '+str(len(Devices[2].Options["LevelNames"]))
            dictOptions["LevelActions"] = dictOptions["LevelActions"]+'|'
            Devices[2].Update(nValue=0, sValue="Test", Options=dictOptions)
        else:
            Domoticz.Heartbeat(10)
            setSelectorByName(2, '[Test] 62')

    def onCommand(self, Unit, Command, Level, Hue):
        Domoticz.Log("onCommand called for Unit " + str(Unit) + ": Parameter '" + str(Command) + "', Level: " + str(Level))
        Devices[2].Update(1 if Level > 0 else 0, str(Level))

global _plugin
_plugin = BasePlugin()

def onStart():
    global _plugin
    _plugin.onStart()

def onHeartbeat():
    global _plugin
    _plugin.onHeartbeat()

def onCommand(Unit, Command, Level, Hue):
    global _plugin
    _plugin.onCommand(Unit, Command, Level, Hue)

def setSelectorByName(intId, strName):
    dictOptions = Devices[intId].Options
    listLevelNames = dictOptions['LevelNames'].split('|')
    intLevel = 0
    for strLevelName in listLevelNames:
        if strLevelName == strName:
            Devices[intId].Update(1, str(intLevel))
        intLevel += 10

[jorgh]

@Dnpwwo,

I did manage to get Python to abort in Device.Update (I've pushed a fix that will be in a beta soon) but I don't think it was related to your problem.

Can you try running this plugin to see if it works for you? It adds a number of Options after the initial creation then selects one by name. You can also update the device in the web ui and it will update it back. This seems to work as expected for me.

The example plugin works, I've done some more testing on Domoticz V3.8007 and have some additional information:

Domoticz does not crash if I remove the switch, the device is than updated correctly. As soon as I add the switch (so the device is not 'unused') Domoticz crashes.
One of the selector switches does not cause a crash, its created with different options.

Switches that cause a crash are created using:

Code: Select all

          Domoticz.Log("Receiver input selector device does not exist, creating device")
          strSelectorNames = 'Off'
          strSelectorActions = ''
          for selector in self.XMLRoot.find('device').find('selectorlist'):
            strSelectorNames += '|' + selector.get('name')
            strSelectorActions += '|'
          dictOptions = {"LevelActions": strSelectorActions, \
                     "LevelNames": strSelectorNames, \
                     "LevelOffHidden": "true", \
                     "SelectorStyle": "1"}
          Domoticz.Device(Name=(self.XMLRoot.find('device')).find('model').text + \
            ' ' + zone.get('name') + " Source", Unit=MAINSOURCE, \
            TypeName="Selector Switch", Switchtype=18, Image=5, \
            Options = dictOptions).Create()

The switch that works correctly:

Code: Select all

Domoticz.Log("Receiver listening mode selector device does not exist, creating device")
          strSelectorNames = 'Off'
          strSelectorActions = ''
          for control in self.XMLRoot.find('device').find('controllist'):
            if (control.get('id')[0:3] == 'LMD'):
              strSelectorNames += '|' + control.get('id')[4:]
              strSelectorActions += '|'
          dictOptions = {"LevelActions": strSelectorActions, \
                     "LevelNames": strSelectorNames, \
                     "LevelOffHidden": "true", \
                     "SelectorStyle": "0"}
          Domoticz.Device(Name=(self.XMLRoot.find('device')).find('model').text + \
            ' ' + zone.get('name') + " Mode", Unit=MAINLISTENINGMODE, \
            TypeName="Selector Switch", Switchtype=18, Image=5, \
            Options = dictOptions).Create()

Kind regards,

Jorg