Trunkated data on python plugin

[Thorgal789]

Hello, I m trying to make a python webserver, based on this one https://github.com/domoticz/domoticz/bl ... istener.py
And I have problem, sometime it work, sometime data are missing, here a sample log with data missing

Code: Select all

2024-12-29 17:41:36.230 Climatisation: Calling message handler 'onConnect' on 'module' type object.
2024-12-29 17:41:36.230 Climatisation: Connected successfully to: 192.168.1.105:54962
2024-12-29 17:41:36.231 Climatisation: Name: '192.168.1.105:54962', Transport: 'TCP/IP', Protocol: 'HTTP', Address: '192.168.1.105', Port: '54962', Baud: -1, Timeout: 0, Bytes: 198, Connected: True, Last Seen: 2024-12-29 17:41:36, Parent: 'Server Connection'
2024-12-29 17:41:36.231 Climatisation: Acquiring GIL for 'onConnectCallback'
2024-12-29 17:41:36.231 Climatisation: Processing 'ReadEvent' message
2024-12-29 17:41:36.231 Climatisation: Acquiring GIL for 'ReadEvent'
2024-12-29 17:41:36.231 Climatisation: Received 198 bytes of data
2024-12-29 17:41:36.231 Climatisation: 50 4f 53 54 20 2f 20 48 54 54 50 2f 31 2e 31 0d 0a 48 6f 73 POST./.HTTP/1.1..Hos
2024-12-29 17:41:36.231 Climatisation: 74 3a 20 31 39 32 2e 31 36 38 2e 31 2e 31 3a 38 31 0d 0a 55 t:.192.168.1.1:81..U
2024-12-29 17:41:36.231 Climatisation: 73 65 72 2d 41 67 65 6e 74 3a 20 45 53 50 33 32 48 54 54 50 ser-Agent:.ESP32HTTP
2024-12-29 17:41:36.231 Climatisation: 43 6c 69 65 6e 74 0d 0a 43 6f 6e 6e 65 63 74 69 6f 6e 3a 20 Client..Connection:.
2024-12-29 17:41:36.231 Climatisation: 6b 65 65 70 2d 61 6c 69 76 65 0d 0a 41 63 63 65 70 74 2d 45 keep-alive..Accept-E
2024-12-29 17:41:36.231 Climatisation: 6e 63 6f 64 69 6e 67 3a 20 69 64 65 6e 74 69 74 79 3b 71 3d ncoding:.identity;q=
2024-12-29 17:41:36.231 Climatisation: 31 2c 63 68 75 6e 6b 65 64 3b 71 3d 30 2e 31 2c 2a 3b 71 3d 1,chunked;q=0.1,*;q=
2024-12-29 17:41:36.231 Climatisation: 30 0d 0a 43 6f 6e 74 65 6e 74 2d 54 79 70 65 3a 20 61 70 70 0..Content-Type:.app
2024-12-29 17:41:36.232 Climatisation: 6c 69 63 61 74 69 6f 6e 2f 6a 73 6f 6e 0d 0a 43 6f 6e 74 65 lication/json..Conte
2024-12-29 17:41:36.232 Climatisation: 6e 74 2d 4c 65 6e 67 74 68 3a 20 31 34 34 0d 0a 0d 0a .. .. nt-Length:.144....
2024-12-29 17:41:36.232 Climatisation: Acquiring GIL for 'ReadEvent'
2024-12-29 17:41:36.232 Climatisation: Connection '192.168.1.105:54962' released by Python, reference count is 1.
2024-12-29 17:41:36.232 Climatisation: Pushing 'DisconnectDirective' on to queue
2024-12-29 17:41:36.232 Climatisation: Acquiring GIL for 'CPluginTransportTCP::handleRead'
2024-12-29 17:41:36.232 Climatisation: Pushing 'ReadEvent' on to queue
2024-12-29 17:41:36.282 Climatisation: Processing 'DisconnectDirective' message
2024-12-29 17:41:36.282 Climatisation: Acquiring GIL for 'DisconnectDirective'
2024-12-29 17:41:36.282 Climatisation: Pushing 'onMessageCallback' on to queue
2024-12-29 17:41:36.282 Climatisation: Disconnect directive received for '192.168.1.105:54962'.
2024-12-29 17:41:36.282 Climatisation: Handling TCP disconnect, socket (192.168.1.105:54962) is connected
2024-12-29 17:41:36.283 Climatisation: Acquiring GIL for 'CPluginTransportTCP::handleRead'
2024-12-29 17:41:36.283 Climatisation: Acquiring GIL for 'DisconnectDirective'
2024-12-29 17:41:36.283 Queued asynchronous read aborted (192.168.1.105:54962), [125] Operation canceled.
2024-12-29 17:41:36.283 Climatisation: Pushing 'DisconnectedEvent' on to queue
2024-12-29 17:41:36.283 Climatisation: Processing 'ReadEvent' message
2024-12-29 17:41:36.283 Climatisation: Acquiring GIL for 'ReadEvent'
2024-12-29 17:41:36.283 Climatisation: Received 144 bytes of data
2024-12-29 17:41:36.283 Climatisation: 7b 22 74 65 6d 70 65 72 61 74 75 72 65 22 3a 31 37 2c 22 66 {"temperature":17,"f
2024-12-29 17:41:36.283 Climatisation: 61 6e 22 3a 22 51 55 49 45 54 22 2c 22 76 61 6e 65 22 3a 22 an":"QUIET","vane":"
2024-12-29 17:41:36.283 Climatisation: 34 22 2c 22 77 69 64 65 76 61 6e 65 22 3a 22 7c 22 2c 22 6d 4","widevane":"|","m
2024-12-29 17:41:36.283 Climatisation: 6f 64 65 22 3a 22 48 45 41 54 22 2c 22 70 6f 77 65 72 22 3a ode":"HEAT","power":
2024-12-29 17:41:36.283 Climatisation: 22 4f 4e 22 2c 22 72 6f 6f 6d 54 65 6d 70 65 72 61 74 75 72 "ON","roomTemperatur
2024-12-29 17:41:36.283 Climatisation: 65 22 3a 31 38 2c 22 63 6f 6d 70 72 65 73 73 6f 72 46 72 65 e":18,"compressorFre
2024-12-29 17:41:36.283 Climatisation: 71 75 65 6e 63 79 22 3a 30 2c 22 61 63 74 69 6f 6e 22 3a 74 quency":0,"action":t
2024-12-29 17:41:36.283 Climatisation: 72 75 65 7d .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. rue}
2024-12-29 17:41:36.283 Climatisation: Acquiring GIL for 'ReadEvent'
2024-12-29 17:41:36.284 Climatisation: Processing 'onMessageCallback' message
2024-12-29 17:41:36.284 Climatisation: Acquiring GIL for 'onMessageCallback'
2024-12-29 17:41:36.284 Climatisation: Calling message handler 'onMessage' on 'module' type object.
2024-12-29 17:41:36.284 Climatisation: onMessage called for connection: 192.168.1.105:54962
2024-12-29 17:41:36.284 Status: Climatisation: New mitubishi device detected: 192.168.1.105
2024-12-29 17:41:36.284 Status: Climatisation: POST request received.{'Verb': 'POST', 'URL': '/', 'Headers': {'Host': '192.168.1.1:81', 'User-Agent': 'ESP32HTTPClient', 'Connection': 'keep-alive', 'Accept-Encoding': 'identity;q=1,chunked;q=0.1,*;q=0', 'Content-Type': 'application/json', 'Content-Length': '144'}}
2024-12-29 17:41:36.284 Climatisation: Acquiring GIL for 'onMessageCallback'
2024-12-29 17:41:36.284 Climatisation: Processing 'DisconnectedEvent' message
2024-12-29 17:41:36.284 Climatisation: Acquiring GIL for 'DisconnectedEvent'
2024-12-29 17:41:36.284 Climatisation: Disconnect event received for '192.168.1.105:54962'.
2024-12-29 17:41:36.284 Climatisation: Pushing 'onDisconnectCallback' on to queue
2024-12-29 17:41:36.284 Climatisation: Acquiring GIL for 'DisconnectedEvent'
2024-12-29 17:41:36.284 Climatisation: Processing 'onDisconnectCallback' message
2024-12-29 17:41:36.284 Climatisation: Acquiring GIL for 'onDisconnectCallback'
2024-12-29 17:41:36.285 Climatisation: Calling message handler 'onDisconnect' on 'module' type object.
2024-12-29 17:41:36.285 Climatisation: onDisconnect called for connection '192.168.1.105:54962'. 

There is something that abord the procedure, but I don't see what I m doing bad

Code: Select all

   def onMessage(self, Connection, Data):
        Domoticz.Debug("onMessage called for connection: "+Connection.Address+":"+Connection.Port)

        #All the message is done ?
        Domoticz.Log("Byte needed : " + str(Connection.BytesTransferred()) +  "ATM : " + str(len(Data)))

        # Incoming Requests
        if "Verb" in Data:
            strVerb = Data["Verb"]
            #strData = Data["Data"].decode("utf-8", "ignore")
            Domoticz.Status(strVerb + " request received." + str(Data))

            #Answer to client
            if Connection.Connected()
                Connection.Send({"Status":"200 OK", "Headers": {"Connection": "keep-alive", "Accept": "Content-Type: text/html; charset=UTF-8"}})

And BTW

Code: Select all

2024-12-29 17:41:36.231 Climatisation: Name: '192.168.1.105:54962', Transport: 'TCP/IP', Protocol: 'HTTP', Address: '192.168.1.105', Port: '54962',

What is this port "54962", the webserver is listening on port 80 ......

[Thorgal789]

Something different on a working request

Not working

Code: Select all

2024-12-29 17:41:36.231 Climatisation: Name: '192.168.1.105:54962', Transport: 'TCP/IP', Protocol: 'HTTP', Address: '192.168.1.105', Port: '54962', Baud: -1, Timeout: 0, Bytes: 198, Connected: True, Last Seen: 2024-12-29 17:41:36, Parent: 'Server Connection'

The "Bytes" is only the header, but in the working request, there is the total request, so there is problem at start already .

Code: Select all

2024-12-29 17:44:00.327 Climatisation: Name: '192.168.1.105:54963', Transport: 'TCP/IP', Protocol: 'HTTP', Address: '192.168.1.105', Port: '54963', Baud: -1, Timeout: 0, Bytes: 342, Connected: True, Last Seen: 2024-12-29 17:44:00, Parent: 'Server Connection'

And about

Code: Select all

2024-12-29 17:41:36.232 Climatisation: Connection '192.168.1.105:54962' released by Python, reference count is 1.

??

[Thorgal789]

BTW an admin can move this topic in "bugs and problems" ?

For me this issue have nothing to see with python, but directly with the domoticz framework.

[Thorgal789]

Hello, I think there is enought information on logs to see a problem in the python framework ? It's a pity it's not able to work with a simple request as a webserver.

If someone have a possible workaround to test ? I have access to both server and client.

[Dnpwwo]

@Thorgal789,

You would need to publish your actual code for me to comment but from the looks of it you are writing a webserver and handling client connections.

When a client connects the plugin's onConnect is called followed by the onMessage, if the plugin does not add a reference to the client connection in one of these then it will be immediately disconnected right afterwards (which seems to be happening in your case). In the example you copied this is done using:

Code: Select all

def onConnect(self, Connection, Status, Description):
    self.httpServerConns[Connection.Name] = Connection

in the onConnect, have you done that?

[Thorgal789]

Hello.
But on your sample you memorise the "connexion", but what I need to do with it later ?

On my side I don't store it

Code: Select all

    def onConnect(self, Connection, Status, Description):
        if (Status == 0):
            Domoticz.Log("Connected successfully to: "+Connection.Address+":"+Connection.Port)
        else:
            Domoticz.Log("Failed to connect ("+str(Status)+") to: "+Connection.Address+":"+Connection.Port+" with error: "+Description)
        Domoticz.Log(str(Connection))

I can store it, but what I need to do with it after ? And the issue is random, most of the time, all is working, and I never force a disconnection on my side.

Here you have the complete code https://pastebin.com/FBGqWDZj

And have started with the http listener domoticz sample > https://github.com/domoticz/domoticz/bl ... istener.py
The code store them, but don't use them, appart on the disconnection part

Code: Select all

    def onDisconnect(self, Connection):
        Domoticz.Log("onDisconnect called for connection '"+Connection.Name+"'.")
        Domoticz.Log("Server Connections:")
        for x in self.httpServerConns:
            Domoticz.Log("--> "+str(x)+"'.")
        if Connection.Name in self.httpServerConns:
            del self.httpServerConns[Connection.Name]