Domoticz

Hello All,

I'm reaching out because my basic Python knowledge let's me down. I'm currently busy with a script to get output from the Kamstrup Multical 402. Which proves to be very difficult.

I tried several script and variables so far but i can't seem to get an output.

Hardware i use:
Raspberry Pi 3 with Domoticz
https://wiki.volkszaehler.org/hardware/ ... sb-ausgang
USB Extender via Ethernet
Kamstrup Multical 402

Software on the Pi:
Domoticz one of the latest beta's
Python
PySerial

Troubleshooting done:
To be sure that the IR read/write head works fine i found the official Kamstrup Metertool on the internet and installed that on a windows machine. I plugged in the USB extender, pushed a button on the Kamstrup meter to activate communication mode and i got output on my windows machine. So the hardware all works fine. After 30 minutes of read/write inactivity on the IR port the meter will shut down communication and a button press is needed.

Port Settings:
Baud 1200
Stop Bits: 2
No Parity
Bytesize: EIGHT

Example Script:

Spoiler: show

Code: Select all

#!/usr/local/bin/python

from __future__ import print_function

import serial

#######################################################################
# These are the variables I have managed to identify
# Submissions welcome.

kamstrup_402_var = {

 0x003C: "Energy",
 0x0050: "Power",
 0x0056: "Temp1",
 0x0057: "Temp2",
 0x0059: "Tempdiff",
 0x004A: "Flow",
 0x0044: "Volume",
 0x008D: "MinFlow_M",
 0x008B: "MaxFlow_M",
 0x008C: "MinFlowDate_M",
 0x008A: "MaxFlowDate_M",
 0x0091: "MinPower_M",
 0x008F: "MaxPower_M",
 0x0095: "AvgTemp1_M",
 0x0096: "AvgTemp2_M",
 0x0090: "MinPowerDate_M",
 0x008E: "MaxPowerDate_M",
 0x007E: "MinFlow_Y",
 0x007C: "MaxFlow_Y",
 0x007D: "MinFlowDate_Y",
 0x007B: "MaxFlowDate_Y",
 0x0082: "MinPower_Y",
 0x0080: "MaxPower_Y",
 0x0092: "AvgTemp1_Y",
 0x0093: "AvgTemp2_Y",
 0x0081: "MinPowerDate_Y",
 0x007F: "MaxPowerDate_Y",
 0x0061: "Temp1xm3",
 0x006E: "Temp2xm3",
 0x0071: "Infoevent",
 0x03EC: "HourCounter",

}


#######################################################################
# Kamstrup uses the "true" CCITT CRC-16
#

def crc_1021(message):
    poly = 0x1021
    reg = 0x0000
    for byte in message:
        mask = 0x80
        while(mask > 0):
            reg<<=1
            if byte & mask:
                reg |= 1
            mask>>=1
            if reg & 0x10000:
                reg &= 0xffff
                reg ^= poly
    return reg

#######################################################################
# Byte values which must be escaped before transmission
#

escapes = {
 0x06: True,
 0x0d: True,
 0x1b: True,
 0x40: True,
 0x80: True,
}

#######################################################################
# And here we go....
#
class kamstrup(object):

    def __init__(self, serial_port = "/dev/ttyUSB0"):
        self.debug_fd = open("kamstrup", "wb")
        self.debug_fd.write("\n\nStart\n")
        self.debug_id = None

        self.ser = serial.Serial(
            port = serial_port,
            baudrate = 2400,
            timeout = 20,
            bytesize = serial.EIGHTBITS,
            parity = serial.PARITY_NONE,
            stopbits = serial.STOPBITS_TWO)
#            xonxoff = 0,
#            rtscts = 0)
#           timeout = 20

    def debug(self, dir, b):
        for i in b:
            if dir != self.debug_id:
                if self.debug_id != None:
                    self.debug_fd.write("\n")
                    self.debug_fd.write(dir + "\t")
                    self.debug_id = dir
                    self.debug_fd.write(" %02x " % i)
                    self.debug_fd.flush()

    def debug_msg(self, msg):
        if self.debug_id != None:
            self.debug_fd.write("\n")
            self.debug_id = "Msg"
            self.debug_fd.write("Msg\t" + msg)
            self.debug_fd.flush()

    def wr(self, b):
        b = bytearray(b)
        self.debug("Wr", b);
        self.ser.write(b)

    def rd(self):
        a = self.ser.read(1)
        if len(a) == 0:
            self.debug_msg("Rx Timeout")
        return None
        b = bytearray(a)[0]
        self.debug("Rd", bytearray((b,)));
        return b

    def send(self, pfx, msg):
        b = bytearray(msg)
        b.append(0)
        b.append(0)
        c = crc_1021(b)
        b[-2] = c >> 8
        b[-1] = c & 0xff
        c = bytearray()
        c.append(pfx)
        for i in b:
            if i in escapes:
                c.append(0x1b)
                c.append(i ^ 0xff)
        else:
            c.append(i)
            c.append(0x0d)
            self.wr(c)
    
        def recv(self):
            b = bytearray()
        while True:
            d = self.rd()
        if d == None:
            return None
        if d == 0x40:
            b = bytearray()
            b.append(d)
        if d == 0x0d:
            #break
            c = bytearray()
            i = 1;
        while i < len(b) - 1:
            if b[i] == 0x1b:
                v = b[i + 1] ^ 0xff
            if v not in escapes:
                self.debug_msg(
                "Missing Escape %02x" % v)
                c.append(v)
                i += 2
        else:
            c.append(b[i])
            i += 1
            if crc_1021(c):
                self.debug_msg("CRC error")
        return c[:-2]
    
    def readvar(self, nbr):
        self.send(0x80, (0x3f, 0x10, 0x01, nbr >> 8, nbr & 0xff))
        b = self.recv()
        if b == None:
            return b
        if b[0] != 0x3f or b[1] != 0x10:
            return None
        if b[2] != nbr >> 8 or b[3] != nbr & 0xff:
            return None
            x = 0
        for i in b[7:]:
            x <<= 8
            x |= i
            s = ""
        for i in b[:4]:
            s += " %02x" % i
            s += " |"
        for i in b[4:7]:
            s += " %02x" % i
            s += " |"
        for i in b[7:]:
            s += " %02x" % i
            decimals = b[6] & 0x0f
        while decimals > 0:
            x *= .1
            decimals -= 1
            #print(s, "=", x)
        return x


if __name__ == "__main__":

    import time

foo = kamstrup()
for i in kamstrup_402_var:
    x = foo.readvar(i)
    #print("%-25s" % kamstrup_402_var[i], x)
    print(kamstrup_402_var[i]+":", x)

I have sniffed the serial interface on Windows but i don't have access to the logs currently. I can provide them if needed.

I know this should be possible but i'm pulling out my hairs right now, i have no idea what could be wrong, on the Pi the screen just stays black or no output is received.

What am i looking for?
I am looking for a collaboration where an advanced Python scripter can help me with this, either via mail, phone or skype. In the end i will place the script or whatever we've found here and on Github, and hopefully we can get it completely up and running in Domoticz with regards of the Gj and Energy levels used.

Hopefully there is someone who is able and willing to help me! My English is sufficient enough to communicate in that language, my native tongue is Dutch and I live in Purmerend in the Netherlands. Using City Heating / District heating with this meter.

Thanks!

Best Regards,
Paul

Any updates on this? I have the same meter and IR head, so maybe we can work together to make this work

Did you check:

https://www.domoticz.com/forum/viewtopi ... up#p119053

Hey Paul

I will start the same project next week as well. (same) Hardware already waiting for me
Living in Purmerend and Phyton knowledge available.

Cheers,

Frank

Hi Frank,

Good to hear! Let's team-up as we both live in the same City!

Regards, Paul

You are free of course to re-invent the wheel if you want, but if you check the link in my earlier post, you'll see a working Python script is available for the Kamstrup Meter Protocol.

I've been using it to measure heat usage and hot water usage from my Kamstrup for over a year now.

Hey Hesmink,

One of my good habits is that I am a lazy guy. And I already have taken a copy of the scripts in your link (lazy and not Crazy !)

Many thanks for the hint/link ! And will let you know the result soon.

Thanks

Frank

Hesmink wrote:Did you check:

viewtopic.php?f=42&t=11333&p=119053&hil ... up#p119053

Yes i checked that, thanks for linking!

But i can't get the scripts to work. No response is generated at all. I've tried everything.

When trying on Windows with the official tool the logger works fine so it's deffo not an HW issue.

@Hesmink

I have looked at the link and the script but there is ( not yet ) an output to Domoticz.
Only CSV and MySql

Are you willing to share how you integrated the script into domoticz ?
I guess a http call to a counter ?
How many counters and of which type ?

Or.... have I missed something?

Cheers,
Frank

I pressume he is talking about the pyMaster_single.py script, which you must run with Serial Port and IDX parameters and a Parameter which you want to have the data derived from. Then it will update an IDX in Domoticz, but haven't been successful in that.

Cletus wrote:I pressume he is talking about the pyMaster_single.py script, which you must run with Serial Port and IDX parameters and a Parameter which you want to have the data derived from. Then it will update an IDX in Domoticz, but haven't been successful in that.

Yes, can be found here: viewtopic.php?f=42&t=11333&p=119053&hil ... rup#p96112

Cletus wrote:

Hesmink wrote:Did you check:

viewtopic.php?f=42&t=11333&p=119053&hil ... up#p119053

Yes i checked that, thanks for linking!

But i can't get the scripts to work. No response is generated at all. I've tried everything.

When trying on Windows with the official tool the logger works fine so it's deffo not an HW issue.

First, I would take the script and modify it to just output to the screen.
Second, check for the correct port of the IR device, and make sure you are requesting a register with a meaningful value.
I don't know if the meter tool only does the KMP protocol, it's known that some Kamstrup meters use a much simpler form of communication.

Whaaaaaaa 1 huge step further for me

After some headbanging why it didn't work for me I found I have the Kamstup M66C....

Then I found this link :
http://www.smartmeterdashboard.nl/downloads
and downloaded the
http://www.smartmeterdashboard.nl/downl ... ects=0&d=1

Still no succes.. . pfffff

Then found that on the raspberry you put --comport 2 to point the script to /dev/ttyUSB1
So using 2 for USB1 ( pfffffff)

Still no succes.. #@$%#$%^@$%

RTFM
in the manual of Joost van der Linde it tells you to use python 3 !!
Running python -v gave me version 2.3 AHAHA !
Running python3 -v gives me version 3

so with

>>python3 M66C.py --comport 2

I got the following output :
An other evening of 'hacking' fun.. As you can see my post has a lot of emotion in it.. haha. Succes makes me smile !!
Tomorrow going to join Paul to get his version working on site and mine to integrate into Domoticz counters.

Many thanks for the help so far !!!

Frank

freijn wrote: RTFM
in the manual of Joost van der Linde it tells you to use python 3 !!
Running python -v gave me version 2.3 AHAHA !
Running python3 -v gives me version 3

Seriously?!?

Wow............ Haven't tried that yet. LOL!

See you this evening Frank!

So, in your case you use the 'simple' protocol, and not the KMP protocol.
My modifications are for the KMP protocol, since that works for my Kampstrup.

Confusing, that 2 protocols are used for the same type of hardware.

Hesmink

I had some spare moment and decided to run it with your script as well.

After changing
import urllib2
into
import urllib
and added
import urllib.request

then also change the url into
requestPost = urllib.request.urlopen( "http://127.0.0.1:8080/json.htm?type=com ... evice&idx=" + str(index) + "&svalue=" + str(v$

it dumps me this error.

Any clue ?

freijn wrote:

Spoiler: show

Hesmink

I had some spare moment and decided to run it with your script as well.

After changing
import urllib2
into
import urllib
and added
import urllib.request

then also change the url into
requestPost = urllib.request.urlopen( "http://127.0.0.1:8080/json.htm?type=com ... evice&idx=" + str(index) + "&svalue=" + str(v$

it dumps me this error.

Any clue ?

Code: Select all

pi@raspberrypi:~/domoticz/scripts/python $ nano kamstrup.py                                                                                  pi@raspberrypi:~/domoticz/scripts/python $ python3 kamstrup.py /dev/ttyUSB1 60 262
waarde:
None
Traceback (most recent call last):
  File "kamstrup.py", line 249, in <module>
    requestPost = urllib.request.urlopen( "http://127.0.0.1:8080/json.htm?type=command&param=udevice&idx=" + str(index) + "&svalue=" + str(value) )
  File "/usr/lib/python3.4/urllib/request.py", line 153, in urlopen
    return opener.open(url, data, timeout)
  File "/usr/lib/python3.4/urllib/request.py", line 461, in open
    response = meth(req, response)
  File "/usr/lib/python3.4/urllib/request.py", line 571, in http_response
    'http', request, response, code, msg, hdrs)
  File "/usr/lib/python3.4/urllib/request.py", line 499, in error
    return self._call_chain(*args)
  File "/usr/lib/python3.4/urllib/request.py", line 433, in _call_chain
    result = func(*args)
  File "/usr/lib/python3.4/urllib/request.py", line 579, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 401: Unauthorized
pi@raspberrypi:~/domoticz/scripts/python $

Hi Frank,

HTTP error 401 seems an unauthenticated error. Can you check if 127.0.0.1 is whitelisted without username and password? Alternatively give your network IP instead of the localhost one? (192*.*.*)

Cheers, Paul

Update: Frank visited my home and we got it to work!!! I will post the scripts we use down here.

Hopefully people have success with this!

Files:

Kamstrup.sh is the script to use selective stuff sending to IDX devices.
Kamstrup.py is screen logging only
Kamstrup_single.py is modified by someone else (left the names in the script! ) for single command usage.

This script is now running on my Raspberry Pi for a week now with success!

Bug: If a meter ends with 0 that 0 will be taken off, need to figure out how to fix this still.

Reagrds, Paul

Paul,

Not tested but it must be something like this, let me know if that works for you.


if __name__ == "__main__":

import time

comport = sys.argv[1]
command = int( sys.argv[2], 0)
index = int( sys.argv[3], 0)

foo = kamstrup( comport )
heat_timestamp=datetime.datetime.strftime(datetime.datetime.today(), "%Y-%m-%d %H:%M:%S" )

value,unit = foo.readvar( command )

#format the output
value = '{:06.2f}'.format(value)

requestPost = urllib.request.urlopen( "http://192.168.2.253:8080/json.htm?type ... evice&idx=" + str(index) + "&svalue=" + str(value) )

Hey Frank,

I tried it out but i got unexpected results as in that it trims the last digit now anyway.

Thanks!

Paul