Domoticz

pipiche wrote: ↑Wednesday 22 May 2019 12:33 @Dnpwwo, I saw in the PluginTransports.h a m_Buffer of 4096 bytes, would that mean that there is a limitation to send not more than 4K per Send request ?
If saw, that is ok, but we should get an erreor back if we are trying to seen a bigger size, no ?

DomoticzVersion: 4.10678
DomoticzHash: 52de7fec
boost version 1.68

Dnpwwo wrote:@pipiche,

With a lot of debugging on it worked for me which also points to a flooding issue.

I've got a few ideas to make it better in Python in the short term, I'll play with it tomorrow

        <param field="Mode6" label="Debug" width="150px">
            <options>
                <option label="None" value="0"  default="true" />
                <option label="Python Only" value="2"/>
                <option label="Basic Debugging" value="62"/>
                <option label="Basic+Messages" value="126"/>
                <option label="Connections Only" value="16"/>
                <option label="Connections+Queue" value="144"/>
                <option label="All" value="-1"/>
            </options>
        </param>

2019-05-25 13:06:25.026 Error: (Test Website) No transport, write directive to '127.0.0.1:64483' ignored.

The write_some operation may not transmit all of the data to the peer.

The function call will block until one or more bytes of the data has been written successfully, or until an error occurs.

Dnpwwo wrote: ↑Saturday 25 May 2019 6:35 @pipiche,

My Python ideas made things better but did not solve it completely.

I've looked at the connections overall by updating your plugin's debugging to show 'Connections Only'

Code: Select all

        <param field="Mode6" label="Debug" width="150px">
            <options>
                <option label="None" value="0"  default="true" />
                <option label="Python Only" value="2"/>
                <option label="Basic Debugging" value="62"/>
                <option label="Basic+Messages" value="126"/>
                <option label="Connections Only" value="16"/>
                <option label="Connections+Queue" value="144"/>
                <option label="All" value="-1"/>
            </options>
        </param>

and can see that the Connection gets an End-of-File notification from Boost. The queued read interprets that as the other end of the connection disconnecting (which is normally the case) which triggers the server side cleanup (disconnection). Subsequent write operations therefore have nowhere to write to and you get

2019-05-25 13:06:25.026 Error: (Test Website) No transport, write directive to '127.0.0.1:64483' ignored.

as a defensive measure by the Plugin Framework. So far, so good. I understand the symptoms but what causes the spurious 'End-of-File' error in the first place?

The smoking gun potentially comes from the Boost documentation for the 'write_some' function the Plugin Framework uses:

The write_some operation may not transmit all of the data to the peer.

and more directly from the description

The function call will block until one or more bytes of the data has been written successfully, or until an error occurs.

which could make it open to flooding although nothing states that an 'End-of-File' will be thrown explicitly if it is flooded.

I don't want to revert to fully synchronous writing because it will be slower in most circumstances but I can try using a per connection mutex to stop a second write operation commencing prior to the previous one completing.

Dnpwwo wrote: ↑Monday 27 May 2019 16:05 Ideally there shouldn't be a compromise between synchronous and asynchronous.

I've added a thread pool to boost::asio to help it scale and now track completion of writes to stop flooding and control memory usage. Your website now serves in around 2 seconds but I'm seeing differences between chrome and firefox.

Firefox seems to terminate connections after a chunked response while chrome does not. Both seem to render the site correctly which is good.

I've changed your test plugn to be multi-threaded and it now trys to ram 'send' requests into the framework as fast a possible with mostly successful results.

Still a bit more debugging to do though.