Domoticz

Posted: **Thursday 20 April 2023 21:13**

Hi all,

I'm just gonna throw this out there and see what happens. I'm trying to fix a long term ongoing issue with the RS485 Modbus RTU plugin that uses minimalmodbus viewtopic.php?t=21297. In Domoticz 2023.1 the plugsin are loaded very fast one after another and plugsin that talk to serial bus get stuck straight away if the plugsin executions aren't space apart in time due to the fact that the port is already open.

I'm rubbish at Python but I'll have a go as no one else has managed to come up with a better solution yet.

How can I check if the serial port is already open in the following example taken from https://github.com/remcovanvugt/SDM120M ... icz-plugin

My idea is to check if the port is open, if it isn't then use the onHeartBeat to try the plugin again after say 10 intervals. This would be a much better fix than my editing Plugins.cpp to insert a 1 second delay when initially loading the plugsin in, in the first place.

I've contacted the plugin author and no joy, its not being maintained.

Code: Select all

#!/usr/bin/env python
"""
Eastron SDM120-Modbus Smart Meter Single Phase Electrical System. The Python plugin for Domoticz
Original author: MFxMF and bbossink
Modified by: remcovanvugt
Requirements: 
    1.python module minimalmodbus -> http://minimalmodbus.readthedocs.io/en/master/
        (pi@raspberrypi:~$ sudo pip3 install minimalmodbus)
    2.Communication module Modbus USB to RS485 converter module
"""
"""
<plugin key="SDM120M" name="SDM120M Modbus" version="1.0.0" author="remcovanvugt">
    <params>
        <param field="SerialPort" label="Modbus Port" width="200px" required="true" default="/dev/ttyUSB0" />
        <param field="Mode1" label="Baud rate" width="40px" required="true" default="9600"  />
        <param field="Mode2" label="Device ID" width="40px" required="true" default="1" />
        <param field="Mode3" label="Reading Interval min." width="40px" required="true" default="1" />
        <param field="Mode6" label="Debug" width="75px">
            <options>
                <option label="True" value="Debug"/>
                <option label="False" value="Normal"  default="true" />
            </options>
        </param>
    </params>
</plugin>

"""

import minimalmodbus
import serial
import Domoticz

minimalmodbus.CLOSE_PORT_AFTER_EACH_CALL=True

class BasePlugin:
    def __init__(self):
        self.runInterval = 1
        self.rs485 = "" 
        return

    def onStart(self):
        self.rs485 = minimalmodbus.Instrument(Parameters["SerialPort"], int(Parameters["Mode2"]))
        self.rs485.serial.baudrate = Parameters["Mode1"]
        self.rs485.serial.bytesize = 8
        self.rs485.serial.parity = minimalmodbus.serial.PARITY_NONE
        self.rs485.serial.stopbits = 1
        self.rs485.serial.timeout = 1
        self.rs485.debug = False
                          

        self.rs485.mode = minimalmodbus.MODE_RTU
        devicecreated = []
        Domoticz.Log("SDM120M Modbus plugin start")
        self.runInterval = int(Parameters["Mode3"]) * 1 
       
        if 1 not in Devices:
            Domoticz.Device(Name="Total System Power", Unit=1,TypeName="Usage",Used=0).Create()
        Options = { "Custom" : "1;VA"} 
        if 2 not in Devices:
            Domoticz.Device(Name="Import Wh", Unit=2,Type=243,Subtype=29,Used=0).Create()
        Options = { "Custom" : "1;kVArh"}
        if 3 not in Devices:
            Domoticz.Device(Name="Export Wh", Unit=3,Type=243,Subtype=29,Used=0).Create()
        Options = { "Custom" : "1;kVArh"} 
        if 4 not in Devices:
            Domoticz.Device(Name="Total kWh", Unit=4,Type=243,Subtype=29,Used=0).Create()
        Options = { "Custom" : "1;kVArh"}
        if 5 not in Devices:
            Domoticz.Device(Name="Voltage", Unit=5,Type=243,Subtype=8,Used=0).Create()
        Options = { "Custom" : "1;V"}
        if 6 not in Devices:
            Domoticz.Device(Name="Import power", Unit=6,TypeName="Usage",Used=0).Create()
        Options = { "Custom" : "1;VA"} 
        if 7 not in Devices:
            Domoticz.Device(Name="Export power", Unit=7,TypeName="Usage",Used=0).Create()
        Options = { "Custom" : "1;VA"} 
               
    def onStop(self):
        Domoticz.Log("SDM120M Modbus plugin stop")

    def onHeartbeat(self):
        self.runInterval -=1;
        if self.runInterval <= 0:
            # Get data from SDM120
            Total_System_Power = self.rs485.read_float(12, functioncode=4, numberOfRegisters=2)
            Import_Wh = self.rs485.read_float(72, functioncode=4, numberOfRegisters=2)
            Export_Wh = self.rs485.read_float(74, functioncode=4, numberOfRegisters=2)
            Total_kwh = self.rs485.read_float(342, functioncode=4, numberOfRegisters=2)
            Voltage = self.rs485.read_float(0, functioncode=4, numberOfRegisters=2)
			
            Import_power = self.rs485.read_float(88, functioncode=4, numberOfRegisters=2)
            Export_power = self.rs485.read_float(92, functioncode=4, numberOfRegisters=2)
            
			#Devices[4].Update(0,str(Current_L1)+";"+str(Current_L2)+";"+str(Current_L3))
            #Update devices
            Devices[1].Update(0,str(Total_System_Power))
            Devices[2].Update(0,str(Total_System_Power)+";"+str(Import_Wh*1000))
            Devices[3].Update(0,str(Export_Wh))
            Devices[4].Update(0,str(Total_kwh))
            Devices[5].Update(0,str(Voltage))
            Devices[6].Update(0,str(Import_power))
            Devices[7].Update(0,str(Export_power))
            
            
            if Parameters["Mode6"] == 'Debug':
                Domoticz.Log("SDM120M Modbus Data")
                Domoticz.Log('Total system power: {0:.3f} W'.format(Total_System_Power))
                Domoticz.Log('Import Wh: {0:.3f} kWh'.format(Import_Wh))
                Domoticz.Log('Export Wh: {0:.3f} kWh'.format(Export_Wh))
                Domoticz.Log('Total kwh: {0:.3f} kWh'.format(Total_kwh))
                Domoticz.Log('Voltage: {0:.3f} V'.format(Voltage))
                Domoticz.Log('Import power: {0:.3f} W'.format(Import_power))
                Domoticz.Log('Export power: {0:.3f} W'.format(Export_power))
               
            self.runInterval = int(Parameters["Mode3"]) * 6
        


global _plugin
_plugin = BasePlugin()


def onStart():
    global _plugin
    _plugin.onStart()


def onStop():
    global _plugin
    _plugin.onStop()


def onHeartbeat():
    global _plugin
    _plugin.onHeartbeat()

# Generic helper functions
def DumpConfigToLog():
    for x in Parameters:
        if Parameters[x] != "":
            Domoticz.Debug("'" + x + "':'" + str(Parameters[x]) + "'")
    Domoticz.Debug("Device count: " + str(len(Devices)))
    for x in Devices:
        Domoticz.Debug("Device:           " + str(x) + " - " + str(Devices[x]))
        Domoticz.Debug("Device ID:       '" + str(Devices[x].ID) + "'")
        Domoticz.Debug("Device Name:     '" + Devices[x].Name + "'")
        Domoticz.Debug("Device nValue:    " + str(Devices[x].nValue))
        Domoticz.Debug("Device sValue:   '" + Devices[x].sValue + "'")
        Domoticz.Debug("Device LastLevel: " + str(Devices[x].LastLevel))
    return

Posted: **Thursday 20 April 2023 23:00**

Also no Python export here but perhaps this discussion will help: https://stackoverflow.com/questions/244 ... re-open-it

Posted: **Thursday 20 April 2023 23:18**

waltervl wrote: ↑Thursday 20 April 2023 23:00 Also no Python export here but perhaps this discussion will help: https://stackoverflow.com/questions/244 ... re-open-it

Additional you can define a mode4 python plugin definition field which could contain the order so you can set the heartbeat mode at startup differently for each plugin copy.
The serial.open() check should be in the onHeartbeat() function.

Posted: **Friday 21 April 2023 9:17**

simat wrote: ↑Thursday 20 April 2023 21:13 How can I check if the serial port is already open in the following example...

Hi,

Before open, in your serial settings you may add:

Code: Select all

self.rs485.serial.exclusive = True

With this setting, the call to open should fail if the serial port is already opened... So you may implement retries, active wait (retry open every few seconds until it succeeds) depending on your use-case/needs.

On my side I also added some lock file exclusive setting (Maybe not needed for python 3, but on 2.7 exclusive setting proved a bit buggy), here is a code snippet for opening a serial-usb analog modem I now use for phone spam filtering (using it's caller ID feature):

Code: Select all

# Set COM Port settings
def set_COM_port_settings(com_port, baud_rate):
    analog_modem.port = com_port
    analog_modem.baudrate = baud_rate
    analog_modem.bytesize = serial.EIGHTBITS    # number of bits per bytes
    analog_modem.parity = serial.PARITY_NONE    # set parity check: no parity
    analog_modem.stopbits = serial.STOPBITS_ONE # number of stop bits
    analog_modem.timeout = 1        # non-block read
    analog_modem.xonxoff = False    # disable software flow control
    analog_modem.rtscts = False     # disable hardware (RTS/CTS) flow control
    analog_modem.dsrdtr = False     # disable hardware (DSR/DTR) flow control
    analog_modem.writeTimeout = 1   # timeout for write
    analog_modem.exclusive = True   # modem tty can't be shared

# Open Modem COM Port
def open_modem(com_port, baud_rate, logger):
    TIOCEXCL = 0x540C # From Linux include/uapi/asm-generic/ioctls.h

    #Try to open the COM Port and execute AT Command
    try:
        # Set the COM Port Settings
        set_COM_port_settings(com_port, baud_rate)
        analog_modem.open()
    except: # pylint: disable=bare-except
        logger.log(logging.INFO, "Unable to open COM Port: " + com_port)
    else:
        # Better enforce exclusive locking (over write lslocks based analog_modem.exclusive hereupper)
        try:
            ioctl(analog_modem.fileno(), TIOCEXCL)
        except: # pylint: disable=bare-except
            logger.log(logging.INFO, "Unable to lock COM Port: " + com_port)
            #sys.exit(1)

        # Flush any existing in/out data
        analog_modem.flushInput()
        analog_modem.flushOutput()
        if not exec_AT_cmd("AT", "OK", logger):
            logger.log(logging.INFO, "Modem check KO!")
            if analog_modem.isOpen():
                analog_modem.close()
            sys.exit(1)
        else:
            # Modem found on COM Port
            logger.log(logging.INFO, "Modem OK on " + com_port)

Posted: **Sunday 23 April 2023 10:10**

Many thanks for the replies,

Code: Select all

self.rs485.serial.exclusive = True

This helped alot, the problem I now have is the self.runInterval, if we get an exception we set self.runInterval = 1 so the plugin gets called again in the next heartbeat (10 seconds) to try again, this works for about 5 plugsin instances getting shuffled into their own timeslot 10 seconds apart so that it can exclusivly access the port.

Is there a way I can get it to call onHeartbeat in the next second or so ? I'm trying to fit 20 instances in, if called every second I could fit 59.

Is it polite to sleep in a plugin for a second ? would it shift next onHeartbeat ?

My other work around is to add sleep(1000) between each plugin being loaded with in StartHardware() in plugsin.cpp, but the code devs aren't a big fan of this for some reason, it only gets called upon start up so doesn't block the program later on.

Code: Select all

	bool CPlugin::StartHardware()
	{
		if (m_bIsStarted)
			StopHardware();

		RequestStart();

		// Flush the message queue (should already be empty)
		{
			std::lock_guard<std::mutex> l(m_QueueMutex);
			while (!m_MessageQueue.empty())
			{
				m_MessageQueue.pop_front();
			}
		}

		// Start worker thread
		try
		{
			std::lock_guard<std::mutex> l(m_QueueMutex);
			sleep_milliseconds(1000);	//****** Wait for a second between load plugins ******  
			m_thread = std::make_shared<std::thread>(&CPlugin::Do_Work, this);
			if (!m_thread)
			{
				Log(LOG_ERROR, "Failed start interface worker thread.");
			}
			else
			{
				SetThreadName(m_thread->native_handle(), m_Name.c_str());
				Log(LOG_NORM, "Worker thread started.");
			}
		}
		catch (...)
		{
			Log(LOG_ERROR, "Exception caught in '%s'.", __func__);
		}

		//	Add start command to message queue
		m_bIsStarting = true;
		MessagePlugin(new InitializeMessage());

		Log(LOG_STATUS, "Started.");

		return true;
	}

Thanks.

Posted: **Sunday 23 April 2023 10:48**

Domoticz.Heartbeat(1) - do next Heartbeat in 1 second

after sucessfull read then do Domoticz.Heartbeat(10) to put the next Heartbeat back to every 10 seconds.

Posted: **Monday 24 April 2023 21:03**

Updated and tested plugin now available on

https://github.com/simat-git/SDM120-Modbus

Works with Domoticz 2023.1, now just takes a few minutes to initially allocate each instance to its own timeslot, will also recover from bus disconnection and re-time itself. The previous plugin would error until manual intervention / reset.

Domoticz

Serial port open detection

Serial port open detection

Re: Serial port open detection

Re: Serial port open detection

Re: Serial port open detection

Re: Serial port open detection

Re: Serial port open detection

Re: Serial port open detection