Domoticz crashes when executing "sudo service domoticz stop"

[meal]

Context:
System RPI3B+, OS bullseye
Domoticz Version: 2023.2
Build Hash: 19efd03
Compile Date: 2023-07-21 17:23:44
dzVents Version: 3.1.8
Python Version: 3.9.2 (default, Mar 12 2021, 04:06:34) [GCC 10.2.1 20210110]

Issue:
Sometimes Domoticz crashes when stopped with "sudo service domoticz stop"
The segmentation fault occurs when domoticz stops the python plugins during the deletion of the access python context
:AccessPython::~AccessPython() ()

After some investigation I found out a root cause of the issue.

When stopping a python plugin the plugin thread (do_work) process the onStopCallBack message
Plugins.cpp Line 1077 : Message->Process(this);

The python plugin execute the following piece of code in PluginMessages.h
virtual void Process(CPlugin* pPlugin)
{
AccessPython Guard(pPlugin, m_Name.c_str());
ProcessLocked(pPlugin);
};

The sequence is as follows:
1- allocate a AccessPython Guard to get the GIL and then :
2 - perform the onStop Python method in plugin.py (Callback(pPlugin, nullptr))
3 - Release the python memory m_Target = Py_None; via the assignament of a pNewRef object
4- perform the Cplugin::Stop() C++ piece of code
5 - desallocate the AccessPython Guard when the Guard gets out of scope.

The issue comes from the fact that Cplugin::Stop() at the end of the sequence :

- gives a end to the python interpreter : Plugins.cpp line 2308 Py_EndInterpreter(m_PyInterpreter):
- releases the GIL : PyEval_ReleaseLock();
At that time the GIL is not owned.
When performing the step 5 - desallocate the AccessPython Guard the thread executes PyErr_Occurred() which is not allowed if the GIL is not owned which leads to the issue.

To sort out the issue i made the following changes in Plugins.cpp

AccessPython::~AccessPython()
{
if (m_pPlugin)
{
//not allowed
//if (PyErr_Occurred())
//{
// m_pPlugin->Log(LOG_NORM, "Python error was set during unlock for '%s'", m_Text.c_str());
// m_pPlugin->LogPythonException();
// PyErr_Clear();
//}
m_pPlugin->ReleaseThread();
}
}

void CPlugin::ReleaseThread()
{
if (m_PyInterpreter)
{
//check if errors
if (PyErr_Occurred())
{
Log(LOG_NORM, "Python error was set during unlock for '%s'", m_PluginKey.c_str());
LogPythonException();
PyErr_Clear();
}
if (!PyEval_SaveThread())
{
Log(LOG_ERROR, "Attempt to release GIL returned NULL value");
}
}
}
It is currently under testing and working so far.

Even if it is not related to the current issue I found a small improvement in the CPlugin::Do_Work() in Pluhins.cpp

Line 1097 insert the 2 lines:
if (m_bIsStopped)
continue;
to prevent sending new onHeartbeatCallback messages and checking the connections when the plugin is stopped.

I thanks in advance the readers of this comment for their valuable feebacks.
BR

[meal]

I created the issue https://github.com/domoticz/domoticz/issues/5831 to support the point
BR

[meal]

Issue fixed