General Modbus support

[bobdvb]

Modbus is a standard protocol which covers inputs and outputs, it is used for relays, PLCs, environment sensors, security systems, energy monitors and importantly HVAC systems. There are some specific modbus libraries which support HVAC appliances but it would be nice to have a generic Modbus library in Domoticz.

The scenario I see being useful is that you can have "coils" (on/off), "inputs" (on/off), and registers for sensors (read only) and configuration values (read/write). It would be good to be able to add a Modbus port (serial or IP) to the hardware section of Dz and then you can add a device such as a coil or an input as an on/off switch, light or sensor. Then as a second phase add sensor devices for temperature, humidity and power.

For me this is non-commercial but I have put some PLCs in my house for home automation purposes and it would be nice to be able to use them. I've written some Python to test but I am a terrible coder so I don't feel confident in my work. Even if this is non-commercial this would be useful to companies who are doing BMS systems and I am also willing to offer a small bounty to anyone doing this work so that I can finish my integration.

[alveman]

+1


Skickat från min iPad med Tapatalk

[papoo]

Good idea
+1

[gizmocuz]

What hardware can be used to receive modbus ?

[pj-r]

Modbus rtu and ascii are serial protocol over rs485 hw layer. So something simple/cheap as this can be used: http://www.ebay.co.uk/itm/322068807457 if you want to connect real modbus ascii/rtu hw. Modus tcp/ip wont require any additional hw.

For testing modbus tcp/ip you can setup slave software running on you computer and make tests against that. For example just grap some slave/server example from pyModbus library and run that.

For testing ascii/rtu you can setup something as simple as nano connected with usb to your computer and this sketch burned on that:
https://github.com/smarmengol/Modbus-Ma ... _slave.ino
Just setup this line to use HW serial of your nano:

Code: Select all

/**
 *  Modbus object declaration
 *  u8id : node id = 0 for master, = 1..247 for slave
 *  u8serno : serial port (use 0 for Serial)
 *  u8txenpin : 0 for RS-232 and USB-FTDI 
 *               or any pin number > 1 for RS-485
 */
Modbus slave(1,0,0); // this is slave @1 and RS-232

It might be possible also to make virtual serial port and run pymodbus or some other (slave)simulation software there but its over my knowledge.

I think most of the modbus hw are rtu on rs458.

In modbus is single master protocol. Network contains one master, multiple slaves. Master polls(read/write) values from/to slave. Slave only responses to master queries. Domoticz needs the master end implementation.

libmodbus could be a good base for domoticz modbus?

There is quite nice FAQ about modbus: http://www.simplymodbus.ca/FAQ.htm
http://gridconnect.com/blog/tag/modbus-explained/

[Skiven78]

+1

Skickat från min SM-N9005 via Tapatalk

[Toulon7559]

Devices like kWh-meters (sometimes) have an RS485-interface applying Modbus-protocol.
Example of implementation through a Linksprite-shield on the Raspberry:
see http://www.domoticz.com/forum/viewtopic ... 808#p64229

Having a general Modbus-support in Domoticz would be a starter:
my experience is that each type of device has the need for a specific protocol on top of the general support

[bobdvb]

What hardware can be used to receive modbus ?

As I said in my posting.

Many HVAC systems: http://www.syxthsense.com/modbus_controllers
and various other IO: http://www.audon.co.uk/rs485_modbus.html

I have some of these: http://www.xlogic-plc.com/
[edit] They have a good Modbus document: http://www.xlogic-plc.com/Modbus%20RTU%20PROTOCOL.pdf

Most PLCs support Modbus.

[Toulon7559]

In the forum you can find reference to kWh-meters DIN-format with RS485/Modbus-interface of type DDS238 and SDM120C.
Other, current examples of such kWh-meters:
http://www.chinaenergymeter.com/1-1-5-s ... meter.html
https://www.alibaba.com/product-detail/ ... 97985.html
https://www.kwhmeter.nl/Files/2/26000/2 ... uacS91.pdf
Clear demonstrations that "RS485 & Modbus" is a starter, requiring an extra specific protocol on top for each type of meter.
'General' Modbus support can already be obtained by various Python-modules, but comparable basic support incorporated in Domoticz obviously is nicer.

[Andrex]

+1

[gizmocuz]

Thanks for the links, do you have an actual AliExpress link ? (or ebay... but not preferred)

[Toulon7559]

Some info in response to the previous message.

One of the present links to AliExpress to buy a kWh-meter type SDM120 (with S0-interface & RS485-Interface):
https://nl.aliexpress.com/item/SDM120C- ... 4.2.CAkK9b

A configuration with 1 (one) kWh-meter type DDS238-1ZN can be handled by my Python-script described in messages such as http://www.domoticz.com/forum/viewtopic ... 808#p68942
[Be aware that also a version DDS238-1 exists, which does not have an RS485-interface!]

If you want more than 1 kWh-meter interfacing on the same RS485-bus, bear in mind that usually at delivery the device's bus-address is set at '1'.
Therefore, simply connecting multiple kWh-meters in a row on the RS485-bus is not working, because of address-conflict.
For a configuration with multiple kWh-meters you have to install 1 kWh-meter at a time to the bus, and immediately change the device's address under control of the installation-script.
The installation-script should shift the addresses for all connected meters to a 'safe, different' setting other than '1'.

[pj-r]

my experience is that each type of device has the need for a specific protocol on top of the general support

Can you give some examples?

[Toulon7559]

@p-jr

Example 'public' protocol description for SDM120 can be downloaded here:
http://www.flanesi.it/blog/download/sdm ... 1.pdf?dl=0

For DDS238-1ZN I got the protocol-description by other ways (not public), from which the below sheet is an extraction.

DDS238-1 ZN DDS238-2ZNS modbus address.pdf

DDS238_Registeraddresses

(30.96 KiB) Downloaded 356 times

Modbus dictates the use of Slave adresses (001 till 247, 000 for 'Broadcast') and certain function codes (03H, 04H and 10H) for primary access, but many differences at the next level:
- communication settings like bitrate and parity sometimes differ, or can be set
- other addresses to be called for write-commands and for read-requests
- other contents and layout of the commands and of read-requests
- other functions to command and to call [e.g. for SDM120 the first S0-interface is with fixed pulse-rate and for the second S0-interface you can remotely set the pulse-rate, while DDS238-1ZN only has one S0-interface with fixed pulse-rate]. DDS238-1ZN has a basic set, while SDM120 has more functions.

Under this link you find an example of another protocol description.
The introduction to RS485&Modbus is a nice expalanation of RS485 and Modbus, and the protocol/message-list just emphasize my opinion that at message-level for each device type 'everything is specific'.

[pj-r]

Summary of differences:
- other address-registers
- other contents and layout of the registers
- other functions to command [e.g. for SDM120 one S0-interface with fixed pulse-rate and you can set the pulse-rate for the second S0-interface, while DDS238 only has one S0-interface with fixed pulse-rate] and to call.

I think the support has to be so general that these differences wont matter. There are four main things needed for one poll event: slave address, function code, register start address, register size

I'd see it this way:
We need combined device and 3 types of modbus register devices: input, output and two way.
Also need a way to specify combined devices.
For example:
electric meter
- instant wattage (input)
- consumed Wh/kWh (input)
- for this we need read two different registers -> 2 modbus register devices combined
thermostat device
- setpoint (output or two way? possible to change this value from actual device)
- current temp reading(input)
relay device
- relay state (output or two way)

Should combined devices be skipped in the beginning to make start of the implementation more simple?

Setup could go this way:
1. User specify HW device what is a serial device or modbus over tcp
2. Adds combined device and specify its type
3. For combined device user adds and setups modbus register device with following info
3.1 direction: in, out, twoway(do we need possiblity to specify functions and registers for both: in and out separately in case of two way?)
3.2 name: name of device
3.3 function: modbus function code
3.4 register: source/target register number
3.5 registercount: size of register
3.6 datatype: datatype and possible calculations/conversions. data is usually saved so that it need some calculations to get human readable value.
3.6.1 Example from my heat pump: Indoor outlet water temp: (temperature+60℃)*2 -> (modbus value/2)-60 -> actual value
3.6 poll interval(for input and two way devices)
3. save and start using device from devices tab

There is example of my Amitime heat pump registers:

Spoiler: show

Code: Select all

--- Read/Write registers
3000 Run Mode 0: Standby, 1: Cooling Mode by Room, 2: Cooling Mode by Water, 
              3: Heating Mode by Room, 4: Heating Mode by Water, 5: Water Heater Mode
              6: Level I of freeze, 7: Level II of freeze
3001 Run mode 1 setpoint: 10℃~31℃
3002 Run mode 2 setpoint: 7℃~25℃
3003 Run mode 3 setpoint: 10℃~31℃
3004 Run mode 4 setpoint: 7℃~52℃
3005 Run mode 5 setpoint: 20℃~52℃
--- Read registers
3100 Compressor frequency: 0-120
3101 Wired controller temp: 0-80
3102 Outdoor outlet water temp: (temperature+60℃)*2 NOT USED IN AIR TO WATER
3103 Indoor outlet water temp: (temperature+60℃)*2
3104 Indoor inlet water temp: (temperature+60℃)*2
3105 Heat sink temp: NOT USED
3106 Indoor coil temp: (temperature+60℃)*2 (Outdoor inlet water temp in water out systems)
3107 Outdoor air temp: (temperature+60℃)*2
3108 Outdoor coil temp: (temperature+60℃)*2
3109 Outdoor exhausting temp: (temperature+60℃)*2
---
3120 Input current: 0-225=0.0-22.5 Arms
3121 Input AC voltage: 0-365=0-365V
3122 Expansion Valve Openness: 0-500
3123 DC Bus Voltage: NOT IN USE CURRENTLY
---
3124 Communication error
3125 Indoor temperature sensor fault
3126 Input voltage or current sensor fault
3127 Drive failure protection or drive fault
3128 Indoor unit EEPROM fault
3129 Overload protection
3130 Input under voltage protection
3131 High pressure sensor fault
3132 Outdoor unit EEPROM fault
3133 Outdoor water flow protection
3134 Outdoor temperature sensor fault
3135 High-pressure protection
3136 Outdoor temperature protection
3137 Indoor coil temperature protection
3138 Indoor water flow protection

I think this way the general modbus support could be added in domoticz.

What do you @Toulon7559 think about this?

[Toulon7559]

As indicated in my previous (meanwhile updated) message I see 'commonality' only at the level of 'primary access', which is
- making a call to a defined slave address (00 till 247) and to a defined register (03hex, 04hex or 10hex/21dec).
Unfortunately, the next level of access is different for each type of device ......

In the context of Domoticz we could go in the direction followed under Dashboard/Hardware for the I2C-interface:
- make under Dashboard/Hardware a main section called "Modbus_RS485"
- under that main section make subsections dedicated to a specific device in which the application script & settings per device-type can be called/set.

Because meanwhile for my configuration I have a working solution for 1 DDS238-1ZN kWh-meter in the form of a python-script applying the minimalmodbus-module, personnally the need for 'general support by Domoticz' has decreased.
Only action-item I presently foresee as 'common&meaningful' is a robust script to shift the Slave-address at installation of a module applying RS485-Modbus, to prevent beforehand any address-conflicts in a configuration with multiple devices on 1 RS485-bus.
(In my opinion) that shift-address-at-installation is a critical aspect for any specific script, and it has value that for each device in the list mentioned above, the shift-mechanism is functionally identical, because otherwise we can create new havoc .........
With necessary emphasis on 'functionally', because even for that 'basic' shift-command the various RS485-devices apply different protocol/message-layout.

Some 'brainwave' for a very first script (but open for any better ideas):
#1 at start checks how many devices of a certain type are on the RS485-bus and at which Slave-address(es)
#2 shows the Slave-addresses already occupied at the RS485-Bus for this type of device
#3 script has 2 fields for change of Slave-address, to insert the 'From'-Slave-address 'to be changed', respectively to insert the desired 'To'-Slave-address [just a basic change from '1' to 'X' (with X= 2~247] is not enough, because you must be able e.g. to change back. Some people make errors, or at a certain time want to modify configuration-settings ....]
#4 Basic fault checking =
1) 'From' and 'To' must be different, because otherwise you do not know whether the ordered change has become effective
2) if 'From' Slave-adress is empty, then the change is refused, because impossible to execute
3) if 'To' 'Slave-address is occupied, then the change is refused, because it means a conflict with another device
#5 at end of script show the actual list of devices on the RS485-bus.

Not sure, but at #2 there seems a risc if you have different types of devices on the same RS485-bus:
can the search made with the script for one type of device also 'see' another type of device?
If you apply only 1 type of device on the RS485-bus, the 'shift-script' could be simplified to:
#1 check which adresses are occupied on the RS485-bus
#2 if adress '1' is occupied, then shift that Slave to the next free address
#3 and #4 are optional and extensions for later time
#5 at end of script show the actual list of devices on the RS485-bus.

If that script is running, the devices got their separate, defined Slave-addresses, and the user can then apply his own 'home-made' application-software on devices with defined Slave-adress(es).

If that script is OK, then perhaps as a starter for the Domoticz' Dashboard/ Hardware we could introduce the main section mentioned above, and for each specific type of device in its subsection put the installation-functionality to shift for a device it's Slave-address to another position, IF required to deviate from 'default'. Not caring about different types of devices on the RS485-bus, leaving it to the user to prevent conflicts by 'clever' choices.

At later date such first steps might be followed by an upgrade within Domoticz, extending the subsections mentioned above, with a correlation-check for occupied Slave-addresses, enabling automatic resolution for conflicts between types of devices.
Probably it takes a year or more to arrive at that point .........

[Toulon7559]

Trying something .......
First step of the plan is to check where devices are on the RS485-bus.
With pieces of 'blunt' python-code stepwise I can set and reset my kWh-meter type DDS238-1ZN for the communication parameters.
But the planned script should be a bit more robust & more user-friendly with application of the Python-module minimalmodbus.

Following code should give indication whether on a certain address x a DDS238-1ZN-kWh-meter is present.

Code: Select all

   instrument = minimalmodbus.Instrument('/dev/ttyAMA0',x) # port name, slave address
   instrument.serial.baudrate = 9600
   instrument.serial.timeout = 0.5
   instrument.debug = True
   print instrument
   Status = instrument.read_register(21,0) # registernumber, number of decimals

In my configuration at this moment 1 kWh-meter assigned to address 1.
Not surprisingly, the call to Broadcast-adress x = 0 and the next call to Slave-address x = 1 confirm that 'occupation'.
Next call to Slave-address x=2 obviously gets no response.
The response from the script is below: the black part is the output for 'print instrument', and the white part is the Debug-info related to the next line requesting 'Status'.
Because my knowledge of Python is 'rather basic', then the question pops up how to deal with that 'no response' in a clean way.
A hint appreciated, enabling to move on to look at x = 3 etc.

Script_response at x=2

Knipsel_respons.PNG (14.37 KiB) Viewed 15456 times

[bobdvb]

I've taken the Chopper_Rob script and made my own Modbus version, it is still very hacked together and I don't recommend anyone use it yet but I wanted to share my progress so far:

Code: Select all

#!/usr/bin/python
#   Title: modbus.coilX.py
#   Author: bobdvb original code from Chopper_Rob
#   Date: 08-11-2016
#   Info: Checks the value of a modbus register and reports that to Domoticz
#   URL :
#   Version : 0.5

import sys
import datetime
import time
import os
import subprocess
import urllib2
import json
import base64

import minimalmodbus
import serial

instrument = minimalmodbus.Instrument('/dev/ttyUSB3', 1) # port name, slave address
instrument.serial.baudrate = 38400
instrument.timeout = 1
instrument.precalculate_read_size = False
instrument.debug = False

modfunccode = 2 # This sets if it is an input or an output that we are reading.

# Settings for the domoticz server
domoticzserver="192.168.1.2:8080"
domoticzusername = "admin"
domoticzpassword = "admin"
domoticzpasscode = "passphrase"

# If enabled. The script will log to the file _.log
# Logging to file only happens after the check for other instances, before that it only prints to screen.
log_to_file = False

# The script supports two types to check if another instance of the script is running.
# One will use the ps command, but this does not work on all machine (Synology has problems)
# The other option is to create a pid file named _.pid. The script will update the timestamp
# every interval. If a new instance of the script spawns it will check the age of the pid file.
# If the file doesn't exist or it is older then 3 * Interval it will keep running, otherwise is stops.
# Please chose the option you want to use "ps" or "pid", if this option is kept empty it will not check and just run.
check_for_instances = "pid"


# DO NOT CHANGE BEYOND THIS LINE
if len(sys.argv) != 5 :
  print ("Not enough parameters. Needs %register %IDX %Interval %Cooldownperiod.")
  sys.exit(0)

device=sys.argv[1]
switchid=sys.argv[2]
interval=sys.argv[3]
cooldownperiod=sys.argv[4]
previousstate=-1
lastsuccess=datetime.datetime.now()
lastreported=-1
base64string = base64.encodestring('%s:%s' % (domoticzusername, domoticzpassword)).replace('\n', '')
domoticzurl = 'http://'+domoticzserver+'/json.htm?type=devices&filter=all&used=true&order=Name'

if check_for_instances.lower() == "pid":
  pidfile = sys.argv[0] + '_' + sys.argv[1] + '.pid'
  if os.path.isfile( pidfile ):
    print datetime.datetime.now().strftime("%H:%M:%S") + "- pid file exists"
    if (time.time() - os.path.getmtime(pidfile)) < (float(interval) * 3):
      print datetime.datetime.now().strftime("%H:%M:%S") + "- script seems to be still running, exiting"
      print datetime.datetime.now().strftime("%H:%M:%S") + "- If this is not correct, please delete file " + pidfile
      sys.exit(0)
    else:
      print datetime.datetime.now().strftime("%H:%M:%S") + "- Seems to be an old file, ignoring."
  else:
    open(pidfile, 'w').close()

if check_for_instances.lower() == "ps":
  if int(subprocess.check_output('ps x | grep \'' + sys.argv[0] + ' ' + sys.argv[1] + '\' | grep -cv grep', shell=True)) > 2 :
    print (datetime.datetime.now().strftime("%H:%M:%S") + "- script already running. exiting.")
    sys.exit(0)

def log(message):
  print message
  if log_to_file == True:
    logfile = open(sys.argv[0] + '_' + sys.argv[1] + '.log', "a")
    logfile.write(message + "\n")
    logfile.close()

def domoticzstatus ():
  json_object = json.loads(domoticzrequest(domoticzurl))
  status = 0
  switchfound = False
  if json_object["status"] == "OK":
    for i, v in enumerate(json_object["result"]):
      if json_object["result"][i]["idx"] == switchid:
        switchfound = True
        if json_object["result"][i]["Status"] == "On":
          status = 0
        if json_object["result"][i]["Status"] == "Off":
          status = 1
  if switchfound == False: print (datetime.datetime.now().strftime("%H:%M:%S") + "- Error. Could not find switch idx in Domoticz response. Defaulting to switch off.")
  return status

def domoticzrequest (url):
  request = urllib2.Request(url)
  request.add_header("Authorization", "Basic %s" % base64string)
  response = urllib2.urlopen(request)
  return response.read()

log (datetime.datetime.now().strftime("%H:%M:%S") + "- script started.")

lastreported = domoticzstatus()
if lastreported == 0 :
  log (datetime.datetime.now().strftime("%H:%M:%S") + "- according to domoticz, " + device + " is online")
if lastreported == 1 :
  log (datetime.datetime.now().strftime("%H:%M:%S") + "- according to domoticz, " + device + " is offline")

while 1==1:
#  currentstate = subprocess.call('ping -q -c1 -W 1 '+ device + ' > /dev/null', shell=True)
  errcnt = 5
  while True:
    try:
      currentstate = instrument.read_bit(int(device), modfunccode)
      break
    except IOError:
      print("Failed to read from instrument, trying again. Error count: " + str(errcnt))
      time.sleep(1)
      errcnt = errcnt - 1
      if errcnt == 0 : break
    except ValueError:
      print("Unexpected value returned, trying again. Error count: " + str(errcnt))
      time.sleep(1)
      errcnt = errcnt - 1
      if errcnt == 0 : break
#  print "Read instrument"

  if currentstate == 0 : lastsuccess=datetime.datetime.now()
  if currentstate == 0 and currentstate != previousstate and lastreported == 1 :
    log (datetime.datetime.now().strftime("%H:%M:%S") + "- " + device + " off, no need to tell domoticz")
  if currentstate == 0 and currentstate != previousstate and lastreported != 1 :
    if domoticzstatus() == 0 :
      log (datetime.datetime.now().strftime("%H:%M:%S") + "- " + device + " tell domoticz it's off")
      domoticzrequest("http://" + domoticzserver + "/json.htm?type=command&param=switchlight&idx=" + switchid + "&switchcmd=Off&level=0" + "&passcode=" + domoticzpasscode)
    else:
      log (datetime.datetime.now().strftime("%H:%M:%S") + "- " + device + " off and domoticz already knew")
    lastreported=1

  if currentstate == 1 and currentstate != previousstate :
    log (datetime.datetime.now().strftime("%H:%M:%S") + "- " + device + " on, waiting")

  if currentstate == 1 and (datetime.datetime.now()-lastsuccess).total_seconds() > float(cooldownperiod) and lastreported != 0 :
    if domoticzstatus() == 1 :
      log (datetime.datetime.now().strftime("%H:%M:%S") + "- " + device + " on, tell domoticz it's on")
      domoticzrequest("http://" + domoticzserver + "/json.htm?type=command&param=switchlight&idx=" + switchid + "&switchcmd=On&level=0" + "&passcode=" + domoticzpasscode)
    else:
      log (datetime.datetime.now().strftime("%H:%M:%S") + "- " + device + " on, but domoticz already knew")
    lastreported=0

  time.sleep (float(interval))

  previousstate=currentstate
  if check_for_instances.lower() == "pid": open(pidfile, 'w').close()

This reads the state of a register in my Xlogic PLC via modbus and then sets that as a switch state.

I've also dealt with the errors that minimalmodbus library was throwing occasionally by adding a nasty try loop and a 1 sec sleep. I've tried this a few times and it seems to deal with the strange errors that I was getting with my PLC.

I hope that someone else might take an interest in this code and help me further? Should I put this on github or something?

Regards,

Bob

[bobdvb]

Also for reference here is my generic minimalmodbus probe python script which probes what I believe are the standard Modbus registers and values:

Code: Select all

#!/usr/bin/env python
import minimalmodbus
import serial
import time

inputCount = 12
outputCount = 6
regCount = 16 # Should be 256
middleCoil = 16 # Should be 256
flagCount = 32

instrument = minimalmodbus.Instrument('/dev/ttyUSB3', 1) # port name, slave address (in decimal)
instrument.serial.baudrate = 38400
instrument.timeout = 0.5
instrument.precalculate_read_size = False

#instrument.write_long(9, 12451840)

while (1):
        count = 0
        print '= Input Registers ='
        while (count < inputCount):
                coil = instrument.read_bit(count, 2) # Registernumber, number of decimals
                print 'input:', count+1, 'value:', coil
                count = count + 1
#       time.sleep(1)

        count = 0
        print '= Digital Outputs ='
        while (count < outputCount):
                coil = instrument.read_bit(count, 1) # Registernumber, number of decimals
                print 'coil:', count+1, 'value:', coil
                count = count + 1
#        time.sleep(1)

        count = 0
        print '= Middle Coil ='
        while (count < middleCoil):
                coil = instrument.read_bit((256 + count), 1) # Registernumber, number of decimals
                if (coil > 0):
                        print 'coil:', count+1, 'value:', coil
                count = count + 1

        count = 0
        print '= Digital Flag ='
        while (count < flagCount):
                coil = instrument.read_bit((768 + count), 1) # Registernumber, number of decimals
                if (coil > 0):
                        print 'value:', count+1, 'value:', coil
                count = count + 1

        count = 0

        print '= Holding Registers ='
        while (count < regCount):
                coil = instrument.read_long(count, 3) # Registernumber, number of decimals
                if (coil > 0):
                        print 'value:', count+1, 'value:', coil
                count = count + 1

        time.sleep(1)

print 'done'

[AndyDHill]

Modbus TCP/IP over Ethernet would be a bonus, we could then pick up Industry standard IO like Wago or Beckhoff and access thermocouples / Digial inputs/outputs also Items Like JMobile HMI's wil support Modbus TCP/IP so we could access then for Displays and interfaces.


Doing a quick search I see there is a Python Stack for Modbus TCP/IP (over cat 5) could some one imolement this into Domoticz ?

https://github.com/bashwork/pymodbus