PluginSystem: Restarting I/O service thread

[TiXav]

I am creating my 1st plugin

Is that normal to have "PluginSystem: Restarting I/O service thread" messages ? Because that pollutes the log so much

[Dnpwwo]

@TiXav,

You will normally see that if there are no open Connections and you create one. If you post your plugin source I could comment more.

[TiXav]

@Dnpwwo,

First, I would like to say thank you for all work that you do for domoticz

So now, the plugin, perhaps I do not use the correct way to do that I want

Code: Select all

# 
# PirCam Checker
# The goal of this script : catch the state of a PIR integrated in Dlink Camera
# when using the following API : http://IpCamAdress:port//config/io.cgi 
# the API return in1=Off /r/n out1=on"
# this means the setup of the duration the output of the camera need to be twice longer compared to the heartbeat to be sure to capture all the changes of the ouput
# in1 can be equal to on or off
# out1 can be equal to on or off
# the API is under basic Auth
#
# At the end this script would be generic to retrieve some string  from an URL to set a switch on or off
#
# tis script is based on :
#             Goolgle Home page example
#
#             Author: Dnpwwo, 2017
#
#             Demonstrates HTTP connectivity.
#             After connection it performs a GET on  www.google.com and receives a 302 (Page Moved) response
#             It then does a subsequent GET on the Location specified in the 302 response and receives a 200 response.
#
#             Author: Dnpwwo, 2017
#

"""
<plugin key="httpt" name="httpt" author="Dnpwwo" version="1.1.0" externallink="https://www.google.com">
    <params>
        <param field="Address" label="IP Address" width="200px" required="true" default="192.168.1.14"/>
        <param field="Port" label="Port" width="30px" required="true" default="1028"/>
        <param field="Username" label="Username" width="200px" required="true" default="admin"/>
        <param field="Password" label="Password" width="200px" required="true" default=""/>
        <param field="Mode1" label="Update the device even the state is the same" width="400px">
            <options>
                <option label="Yes" value="Yes"/>
                <option label="No" value="False"  default="No" />
            </options>
        </param>
        <param field="Mode2" label="String to set the device On" width="75px"/>
        <param field="Mode3" label="String to set the device Off" width="75px"/>
        <param field="Mode6" label="Debug" width="75px">
            <options>
                <option label="True" value="Debug"/>
                <option label="False" value="Normal"  default="true" />
                <option label="Logging" value="File"/>
            </options>
        </param>
    </params>
</plugin>
"""
import Domoticz
import base64

class BasePlugin:
    httpConn = None
    runAgain = 6
   
    def __init__(self):
        return

    def onStart(self):
        if Parameters["Mode6"] == "Debug":
            Domoticz.Debugging(1)
        DumpConfigToLog()
        
        # create the mandatory child device if it does not yet exist
        if 1 not in Devices:
            Domoticz.Device(Name="PIR_CAM", Unit=1, TypeName="Switch").Create()
            Domoticz.Log("Device created.")
        
        self.httpConn = Domoticz.Connection(Name="HTTP Test", Transport="TCP/IP", Protocol="HTTP", Address=Parameters["Address"], Port=Parameters["Port"],)
        self.httpConn.Connect()
        Domoticz.Heartbeat(4)

    def onStop(self):
        Domoticz.Log("onStop - Plugin is stopping.")

    def onConnect(self, Connection, Status, Description):
        Domoticz.Log("onConnect called")
        if (Status == 0):
            
            #base64 string for Basic Auth
            credentials = ('%s:%s' % (Parameters["Username"], Parameters["Password"]))
            encoded_credentials = base64.b64encode(credentials.encode('ascii'))
            bas=encoded_credentials.decode("ascii")
            
            Domoticz.Debug("Device connected successfully."+bas)
            sendData = { 'Verb' : 'GET',
                         'URL'  : '/config/io.cgi',
                         'Headers' : { 'Content-Type': 'text/xml; charset=utf-8', \
                                       'Connection': 'keep-alive', \
                                       'Accept': 'Content-Type: text/html; charset=UTF-8', \
                                       'Host': Parameters["Address"]+":"+Parameters["Port"], \
                                       'Authorization':'Basic '+bas, \
                                       'User-Agent':'Domoticz/1.0' }
                       }
            Connection.Send(sendData)
        else:
            Domoticz.Log("Failed to connect ("+str(Status)+") to: "+Parameters["Address"]+":"+Parameters["Port"]+" with error: "+Description)

    def onMessage(self, Connection, Data):
        #DumpHTTPResponseToLog(Data)
        
        credentials = ('%s:%s' % (Parameters["Username"], Parameters["Password"]))
        encoded_credentials = base64.b64encode(credentials.encode('ascii'))
        bas=encoded_credentials.decode("ascii")        
        
        strData = Data["Data"].decode("utf-8", "ignore")
        Status = int(Data["Status"])
        
        #Domoticz.Log(strData)
        
        
        PosOn=strData.find(Parameters["Mode2"])
        PosOff=strData.find(Parameters["Mode3"])
        
        if ( ( PosOn>-1 and Devices[1].sValue=="Off" and Parameters["Mode1"]=="No") or ( (PosOn>-1 and Parameters["Mode1"]=="Yes") ) ) :
            UpdateDevice(1,1,"On")
            Domoticz.Log("device1="+str(Devices[1].sValue))
        if ( (PosOff>-1 and Devices[1].sValue=="On" and Parameters["Mode1"]=="No") or ( (PosOff>-1 and Parameters["Mode1"]=="Yes") ) ) :
            UpdateDevice(1,0,"Off")
            Domoticz.Log("device1="+str(Devices[1].sValue))

        if (Status == 200):
            Domoticz.Debug("Good Response received.")
            # no more needed
            #self.httpConn.Disconnect()
        # below to delete
        elif (Status == 302):
            Domoticz.Log("Device returned a Page Moved Error.")
            sendData = { 'Verb' : 'GET',
                         'URL'  : Data["Headers"]["Location"],
                         'Headers' : { 'Content-Type': 'text/xml; charset=utf-8', \
                                       'Connection': 'keep-alive', \
                                       'Accept': 'Content-Type: text/html; charset=UTF-8', \
                                       'Host': Parameters["Address"]+":"+Parameters["Port"], \
                                       
                                       'Authorization':'Basic '+bas, \
                                       'User-Agent':'Domoticz/1.0' },
                        }
            Connection.Send(sendData)
        elif (Status == 400):
            Domoticz.Error("Device returned a Bad Request Error.")
        elif (Status == 500):
            Domoticz.Error("Device returned a Server Error.")
        else:
            Domoticz.Error("Device returned a status: "+str(Status))

    def onCommand(self, Unit, Command, Level, Hue):
        Domoticz.Debug("onCommand called for Unit " + str(Unit) + ": Parameter '" + str(Command) + "', Level: " + str(Level))
        if Command=="On":
            Devices[1].Update(1, "1")
        if Command=="Off":
            Devices[1].Update(0,"0")
        #check for debug
        for x in Devices:
                Domoticz.Debug("Device:           " + str(x) + " - " + str(Devices[x]))
    # no need either this appears in the log  : Error: CConnection_disconnect, disconnection request from 'salon' ignored. Transport does not exist.
    #def onDisconnect(self, Connection):
        #Domoticz.Log("onDisconnect called for connection to: "+Connection.Address+":"+Connection.Port)

    def onHeartbeat(self):
        Domoticz.Log("onHeartbeat called")
        self.httpConn.Connect()
        

global _plugin
_plugin = BasePlugin()

def onStart():
    global _plugin
    _plugin.onStart()

def onStop():
    global _plugin
    _plugin.onStop()

def onConnect(Connection, Status, Description):
    global _plugin
    _plugin.onConnect(Connection, Status, Description)

def onMessage(Connection, Data):
    global _plugin
    _plugin.onMessage(Connection, Data)

def onCommand(Unit, Command, Level, Hue):
    global _plugin
    _plugin.onCommand(Unit, Command, Level, Hue)

def onNotification(Name, Subject, Text, Status, Priority, Sound, ImageFile):
    global _plugin
    _plugin.onNotification(Name, Subject, Text, Status, Priority, Sound, ImageFile)

# def onDisconnect(Connection):
    # global _plugin
    # _plugin.onDisconnect(Connection)

def onHeartbeat():
    global _plugin
    _plugin.onHeartbeat()

# Generic helper functions
def LogMessage(Message):
    if Parameters["Mode6"] == "File":
        f = open(Parameters["HomeFolder"]+"http.html","w")
        f.write(Message)
        f.close()
        Domoticz.Log("File written")

def DumpConfigToLog():
    for x in Parameters:
        if Parameters[x] != "":
            Domoticz.Debug( "'" + x + "':'" + str(Parameters[x]) + "'")
    Domoticz.Debug("Device count: " + str(len(Devices)))
    for x in Devices:
        Domoticz.Debug("Device:           " + str(x) + " - " + str(Devices[x]))
        Domoticz.Debug("Device ID:       '" + str(Devices[x].ID) + "'")
        Domoticz.Debug("Device Name:     '" + Devices[x].Name + "'")
        Domoticz.Debug("Device nValue:    " + str(Devices[x].nValue))
        Domoticz.Debug("Device sValue:   '" + Devices[x].sValue + "'")
        Domoticz.Debug("Device LastLevel: " + str(Devices[x].LastLevel))
    return

def DumpHTTPResponseToLog(httpDict):
    if isinstance(httpDict, dict):
        Domoticz.Log("HTTP Details ("+str(len(httpDict))+"):")
        for x in httpDict:
            if isinstance(httpDict[x], dict):
                Domoticz.Log("--->'"+x+" ("+str(len(httpDict[x]))+"):")
                for y in httpDict[x]:
                    Domoticz.Log("------->'" + y + "':'" + str(httpDict[x][y]) + "'")
            else:
                Domoticz.Log("--->'" + x + "':'" + str(httpDict[x]) + "'")

def UpdateDevice(Unit, nValue, sValue):
    # Make sure that the Domoticz device still exists (they can be deleted) before updating it 
    if (Unit in Devices):
        #if (Devices[Unit].nValue != nValue) or (Devices[Unit].sValue != sValue):
        Devices[Unit].Update(nValue, str(sValue))
        Domoticz.Log("Update "+"' ("+Devices[Unit].Name+") "+":'"+str(sValue))
    return

[Dnpwwo]

@TiXav,

The plugin looks good.

You shouldn't need to create the Basic Auth header though, if Username and Password parameters are present then a Basic Auth header should be created for you by the plugin framework when you Send data.

You are probably seeing the messages because of the 'Connect()' in onHeartbeat. That will disconnect the session then reconnect (causing the message). You only need to reconnect if the remote device disconnects (which you will see in onDisconnect().

[TiXav]

I remove the basic auth header, an d yes it is working well, thank you

but if I replace self.httpConn.Connect() in the heartbeat by self.httpConn.Send(sendData)
I have a lot of error in the log:

Code: Select all

2017-11-07 10:49:14.376 Error: (xxxxxx) Transport is not connected, write directive to 'HTTP Test' ignored.
2017-11-07 10:49:18.388 Error: (xxxxxx) Transport is not connected, write directive to 'HTTP Test' ignored.
2017-11-07 10:49:22.350 Error: (xxxxxx) Transport is not connected, write directive to 'HTTP Test' ignored.
2017-11-07 10:49:26.362 Error: (xxxxxx) Transport is not connected, write directive to 'HTTP Test' ignored.
2017-11-07 10:49:30.374 Error: (xxxxxx) Transport is not connected, write directive to 'HTTP Test' ignored.
2017-11-07 10:49:34.385 Error: (xxxxxx) Transport is not connected, write directive to 'HTTP Test' ignored.
2017-11-07 10:49:38.347 Error: (xxxxxx) Transport is not connected, write directive to 'HTTP Test' ignored.
2017-11-07 10:49:42.359 Error: (xxxxxx) Transport is not connected, write directive to 'HTTP Test' ignored.

So if the Transport is not connected that means I need to do a self.httpConn.Connect()

then I placed a self.httpConn.Connect() in the def onDisconnect(self, Connection):
but now arrrrhhhh, instead of connect each 4 seconds via heartbeat, it connect at around 100ms (around 673 times in 2 minutes) until the log show :
Failed to connect (24) to: 192.168.1.14:1028 with error: Too many open files

Code: Select all

2017-11-07 10:57:08.546 (xxxxxx) onDisconnect called for connection to: 192.168.1.14:1028
2017-11-07 10:57:08.649 PluginSystem: Restarting I/O service thread.
2017-11-07 10:57:08.650 (xxxxxx) in1=off 
out1=off 
2017-11-07 10:57:08.651 (xxxxxx) onDisconnect called for connection to: 192.168.1.14:1028
2017-11-07 10:57:08.753 PluginSystem: Restarting I/O service thread.
2017-11-07 10:57:08.755 (xxxxxx) in1=off 
out1=off 
2017-11-07 10:57:08.755 (xxxxxx) onDisconnect called for connection to: 192.168.1.14:1028
2017-11-07 10:57:08.807 (xxxxxx) Failed to connect (24) to: 192.168.1.14:1028 with error: Too many open files
2017-11-07 10:57:10.562 (xxxxxx) onHeartbeat called
2017-11-07 10:57:14.574 (xxxxxx) onHeartbeat called
2017-11-07 10:57:18.585 (xxxxxx) onHeartbeat called
2017-11-07 10:57:22.546 (xxxxxx) onHeartbeat called
2017-11-07 10:57:26.557 (xxxxxx) onHeartbeat called

The problem in fact is soon as the message is received there is an onDisconnect

Now I don't know how I have to manage this, help please

[Dnpwwo]

@TiXav,

Some small devices do weird things with sessions, D-Link in particular seem to be reluctant to keep HTTP session open even when you request that it keeps it alive and some of their devices will close a session after 2 minutes no matter what you do. There is a DLink example that ships with Domoticz that might be useful, also look at this thread http://www.domoticz.com/forum/viewtopic ... 65&t=15753.

I've found the best way is to code defensively in onHeartbeat. Check if the connection is "self.httpConn.Connected()" and send data if it is otherwise "self.httpConn.Connect()" or throttle the re-connections with something like:

Code: Select all

    def onHeartbeat(self):
        if (self.httpConn.Connecting() or self.httpConn.Connected()):
            Domoticz.Debug("onHeartbeat called, Connection is alive.")
        else:
            self.runAgain = self.runAgain - 1
            if self.runAgain <= 0:
                self.httpConn.Connect()
                self.runAgain = 6
            else:
                Domoticz.Debug("onHeartbeat called, run again in "+str(self.runAgain)+" heartbeats.")

[TiXav]

Thank you, I will try this.