Kamstrup Multical 402 Python Script

Python and python framework

Moderator: leecollings

Cletus
Posts: 54
Joined: Wednesday 17 August 2016 9:32
Target OS: Raspberry Pi / ODroid
Domoticz version: Latest
Location: Purmerend, Netherlands
Contact:

Kamstrup Multical 402 Python Script

Post by Cletus »

Hello All,

I'm reaching out because my basic Python knowledge let's me down. I'm currently busy with a script to get output from the Kamstrup Multical 402. Which proves to be very difficult.

I tried several script and variables so far but i can't seem to get an output.

Hardware i use:
Raspberry Pi 3 with Domoticz
https://wiki.volkszaehler.org/hardware/ ... sb-ausgang
USB Extender via Ethernet
Kamstrup Multical 402

Software on the Pi:
Domoticz one of the latest beta's
Python
PySerial

Troubleshooting done:
To be sure that the IR read/write head works fine i found the official Kamstrup Metertool on the internet and installed that on a windows machine. I plugged in the USB extender, pushed a button on the Kamstrup meter to activate communication mode and i got output on my windows machine. So the hardware all works fine. After 30 minutes of read/write inactivity on the IR port the meter will shut down communication and a button press is needed.

Port Settings:
Baud 1200
Stop Bits: 2
No Parity
Bytesize: EIGHT

Example Script:
Spoiler: show

Code: Select all

#!/usr/local/bin/python

from __future__ import print_function

import serial

#######################################################################
# These are the variables I have managed to identify
# Submissions welcome.

kamstrup_402_var = {

 0x003C: "Energy",
 0x0050: "Power",
 0x0056: "Temp1",
 0x0057: "Temp2",
 0x0059: "Tempdiff",
 0x004A: "Flow",
 0x0044: "Volume",
 0x008D: "MinFlow_M",
 0x008B: "MaxFlow_M",
 0x008C: "MinFlowDate_M",
 0x008A: "MaxFlowDate_M",
 0x0091: "MinPower_M",
 0x008F: "MaxPower_M",
 0x0095: "AvgTemp1_M",
 0x0096: "AvgTemp2_M",
 0x0090: "MinPowerDate_M",
 0x008E: "MaxPowerDate_M",
 0x007E: "MinFlow_Y",
 0x007C: "MaxFlow_Y",
 0x007D: "MinFlowDate_Y",
 0x007B: "MaxFlowDate_Y",
 0x0082: "MinPower_Y",
 0x0080: "MaxPower_Y",
 0x0092: "AvgTemp1_Y",
 0x0093: "AvgTemp2_Y",
 0x0081: "MinPowerDate_Y",
 0x007F: "MaxPowerDate_Y",
 0x0061: "Temp1xm3",
 0x006E: "Temp2xm3",
 0x0071: "Infoevent",
 0x03EC: "HourCounter",

}


#######################################################################
# Kamstrup uses the "true" CCITT CRC-16
#

def crc_1021(message):
    poly = 0x1021
    reg = 0x0000
    for byte in message:
        mask = 0x80
        while(mask > 0):
            reg<<=1
            if byte & mask:
                reg |= 1
            mask>>=1
            if reg & 0x10000:
                reg &= 0xffff
                reg ^= poly
    return reg

#######################################################################
# Byte values which must be escaped before transmission
#

escapes = {
 0x06: True,
 0x0d: True,
 0x1b: True,
 0x40: True,
 0x80: True,
}

#######################################################################
# And here we go....
#
class kamstrup(object):

    def __init__(self, serial_port = "/dev/ttyUSB0"):
        self.debug_fd = open("kamstrup", "wb")
        self.debug_fd.write("\n\nStart\n")
        self.debug_id = None

        self.ser = serial.Serial(
            port = serial_port,
            baudrate = 2400,
            timeout = 20,
            bytesize = serial.EIGHTBITS,
            parity = serial.PARITY_NONE,
            stopbits = serial.STOPBITS_TWO)
#            xonxoff = 0,
#            rtscts = 0)
#           timeout = 20

    def debug(self, dir, b):
        for i in b:
            if dir != self.debug_id:
                if self.debug_id != None:
                    self.debug_fd.write("\n")
                    self.debug_fd.write(dir + "\t")
                    self.debug_id = dir
                    self.debug_fd.write(" %02x " % i)
                    self.debug_fd.flush()

    def debug_msg(self, msg):
        if self.debug_id != None:
            self.debug_fd.write("\n")
            self.debug_id = "Msg"
            self.debug_fd.write("Msg\t" + msg)
            self.debug_fd.flush()

    def wr(self, b):
        b = bytearray(b)
        self.debug("Wr", b);
        self.ser.write(b)

    def rd(self):
        a = self.ser.read(1)
        if len(a) == 0:
            self.debug_msg("Rx Timeout")
        return None
        b = bytearray(a)[0]
        self.debug("Rd", bytearray((b,)));
        return b

    def send(self, pfx, msg):
        b = bytearray(msg)
        b.append(0)
        b.append(0)
        c = crc_1021(b)
        b[-2] = c >> 8
        b[-1] = c & 0xff
        c = bytearray()
        c.append(pfx)
        for i in b:
            if i in escapes:
                c.append(0x1b)
                c.append(i ^ 0xff)
        else:
            c.append(i)
            c.append(0x0d)
            self.wr(c)
    
        def recv(self):
            b = bytearray()
        while True:
            d = self.rd()
        if d == None:
            return None
        if d == 0x40:
            b = bytearray()
            b.append(d)
        if d == 0x0d:
            #break
            c = bytearray()
            i = 1;
        while i < len(b) - 1:
            if b[i] == 0x1b:
                v = b[i + 1] ^ 0xff
            if v not in escapes:
                self.debug_msg(
                "Missing Escape %02x" % v)
                c.append(v)
                i += 2
        else:
            c.append(b[i])
            i += 1
            if crc_1021(c):
                self.debug_msg("CRC error")
        return c[:-2]
    
    def readvar(self, nbr):
        self.send(0x80, (0x3f, 0x10, 0x01, nbr >> 8, nbr & 0xff))
        b = self.recv()
        if b == None:
            return b
        if b[0] != 0x3f or b[1] != 0x10:
            return None
        if b[2] != nbr >> 8 or b[3] != nbr & 0xff:
            return None
            x = 0
        for i in b[7:]:
            x <<= 8
            x |= i
            s = ""
        for i in b[:4]:
            s += " %02x" % i
            s += " |"
        for i in b[4:7]:
            s += " %02x" % i
            s += " |"
        for i in b[7:]:
            s += " %02x" % i
            decimals = b[6] & 0x0f
        while decimals > 0:
            x *= .1
            decimals -= 1
            #print(s, "=", x)
        return x


if __name__ == "__main__":

    import time

foo = kamstrup()
for i in kamstrup_402_var:
    x = foo.readvar(i)
    #print("%-25s" % kamstrup_402_var[i], x)
    print(kamstrup_402_var[i]+":", x)
I have sniffed the serial interface on Windows but i don't have access to the logs currently. I can provide them if needed.

I know this should be possible but i'm pulling out my hairs right now, i have no idea what could be wrong, on the Pi the screen just stays black or no output is received. :(

What am i looking for?
I am looking for a collaboration where an advanced Python scripter can help me with this, either via mail, phone or skype. In the end i will place the script or whatever we've found here and on Github, and hopefully we can get it completely up and running in Domoticz with regards of the Gj and Energy levels used.

Hopefully there is someone who is able and willing to help me! My English is sufficient enough to communicate in that language, my native tongue is Dutch and I live in Purmerend in the Netherlands. Using City Heating / District heating with this meter.

Thanks!

Best Regards,
Paul
This is our world now... the world of the electron and the switch, the beauty of the baud.
Vince88
Posts: 2
Joined: Thursday 22 June 2017 23:19
Target OS: Raspberry Pi / ODroid
Domoticz version:
Contact:

Re: Kamstrup Multical 402 Python Script

Post by Vince88 »

Any updates on this? I have the same meter and IR head, so maybe we can work together to make this work :)
Hesmink
Posts: 168
Joined: Monday 22 June 2015 10:48
Target OS: Raspberry Pi / ODroid
Domoticz version:
Location: The Netherlands
Contact:

Re: Kamstrup Multical 402 Python Script

Post by Hesmink »

freijn
Posts: 536
Joined: Friday 23 December 2016 16:40
Target OS: Raspberry Pi / ODroid
Domoticz version: Stable
Location: Netherlands Purmerend
Contact:

Re: Kamstrup Multical 402 Python Script

Post by freijn »

Hey Paul

I will start the same project next week as well. (same) Hardware already waiting for me :-)
Living in Purmerend and Phyton knowledge available.

Cheers,

Frank
Cletus
Posts: 54
Joined: Wednesday 17 August 2016 9:32
Target OS: Raspberry Pi / ODroid
Domoticz version: Latest
Location: Purmerend, Netherlands
Contact:

Re: Kamstrup Multical 402 Python Script

Post by Cletus »

Hi Frank,

Good to hear! Let's team-up as we both live in the same City! :D

Regards, Paul
This is our world now... the world of the electron and the switch, the beauty of the baud.
Hesmink
Posts: 168
Joined: Monday 22 June 2015 10:48
Target OS: Raspberry Pi / ODroid
Domoticz version:
Location: The Netherlands
Contact:

Re: Kamstrup Multical 402 Python Script

Post by Hesmink »

You are free of course to re-invent the wheel if you want, but if you check the link in my earlier post, you'll see a working Python script is available for the Kamstrup Meter Protocol.

I've been using it to measure heat usage and hot water usage from my Kamstrup for over a year now.
freijn
Posts: 536
Joined: Friday 23 December 2016 16:40
Target OS: Raspberry Pi / ODroid
Domoticz version: Stable
Location: Netherlands Purmerend
Contact:

Re: Kamstrup Multical 402 Python Script

Post by freijn »

Hey Hesmink,

One of my good habits is that I am a lazy guy. And I already have taken a copy of the scripts in your link :-) (lazy and not Crazy !)

Many thanks for the hint/link ! And will let you know the result soon.

Thanks

Frank
Cletus
Posts: 54
Joined: Wednesday 17 August 2016 9:32
Target OS: Raspberry Pi / ODroid
Domoticz version: Latest
Location: Purmerend, Netherlands
Contact:

Re: Kamstrup Multical 402 Python Script

Post by Cletus »

Yes i checked that, thanks for linking! :)

But i can't get the scripts to work. No response is generated at all. I've tried everything.

When trying on Windows with the official tool the logger works fine so it's deffo not an HW issue.
This is our world now... the world of the electron and the switch, the beauty of the baud.
freijn
Posts: 536
Joined: Friday 23 December 2016 16:40
Target OS: Raspberry Pi / ODroid
Domoticz version: Stable
Location: Netherlands Purmerend
Contact:

Re: Kamstrup Multical 402 Python Script

Post by freijn »

@Hesmink

I have looked at the link and the script but there is ( not yet ) an output to Domoticz.
Only CSV and MySql

Are you willing to share how you integrated the script into domoticz ?
I guess a http call to a counter ?
How many counters and of which type ?

Or.... have I missed something?

Cheers,
Frank
Cletus
Posts: 54
Joined: Wednesday 17 August 2016 9:32
Target OS: Raspberry Pi / ODroid
Domoticz version: Latest
Location: Purmerend, Netherlands
Contact:

Re: Kamstrup Multical 402 Python Script

Post by Cletus »

I pressume he is talking about the pyMaster_single.py script, which you must run with Serial Port and IDX parameters and a Parameter which you want to have the data derived from. Then it will update an IDX in Domoticz, but haven't been successful in that. :(
This is our world now... the world of the electron and the switch, the beauty of the baud.
Hesmink
Posts: 168
Joined: Monday 22 June 2015 10:48
Target OS: Raspberry Pi / ODroid
Domoticz version:
Location: The Netherlands
Contact:

Re: Kamstrup Multical 402 Python Script

Post by Hesmink »

Cletus wrote:I pressume he is talking about the pyMaster_single.py script, which you must run with Serial Port and IDX parameters and a Parameter which you want to have the data derived from. Then it will update an IDX in Domoticz, but haven't been successful in that. :(
Yes, can be found here: viewtopic.php?f=42&t=11333&p=119053&hil ... rup#p96112
Hesmink
Posts: 168
Joined: Monday 22 June 2015 10:48
Target OS: Raspberry Pi / ODroid
Domoticz version:
Location: The Netherlands
Contact:

Re: Kamstrup Multical 402 Python Script

Post by Hesmink »

Cletus wrote:
Yes i checked that, thanks for linking! :)

But i can't get the scripts to work. No response is generated at all. I've tried everything.

When trying on Windows with the official tool the logger works fine so it's deffo not an HW issue.
First, I would take the script and modify it to just output to the screen.
Second, check for the correct port of the IR device, and make sure you are requesting a register with a meaningful value.
I don't know if the meter tool only does the KMP protocol, it's known that some Kamstrup meters use a much simpler form of communication.
freijn
Posts: 536
Joined: Friday 23 December 2016 16:40
Target OS: Raspberry Pi / ODroid
Domoticz version: Stable
Location: Netherlands Purmerend
Contact:

Re: Kamstrup Multical 402 Python Script

Post by freijn »

Whaaaaaaa 1 huge step further for me :-)

After some headbanging why it didn't work for me I found I have the Kamstup M66C....

Then I found this link :
http://www.smartmeterdashboard.nl/downloads
and downloaded the
http://www.smartmeterdashboard.nl/downl ... ects=0&d=1

Still no succes.. . pfffff

Then found that on the raspberry you put --comport 2 to point the script to /dev/ttyUSB1
So using 2 for USB1 ( pfffffff)

Still no succes.. #@$%#$%^@$%

RTFM
in the manual of Joost van der Linde it tells you to use python 3 !!
Running python -v gave me version 2.3 AHAHA !
Running python3 -v gives me version 3 :-)

so with

>>python3 M66C.py --comport 2

I got the following output :

Code: Select all

pi@raspberrypi:~/domoticz/scripts/python $ python3 M66C.py  --comport 2
M66C.py V0.41
Non-Windows Mode
Python version 3.4.2
Control-C to abort

Startup parameters:
Output mode           : screen
COM-port              : 2 (/dev/ttyUSB1)
Initialise on 300 baud
Data transfer on 300 baud
Data transfer completed
Number of received elements: 6
Array of received elements: ['/KAM MC', '0.0(00004574909)', '6.8(0654.336*GJ)', '6.26(4823.206*m3)', '6.31(0118948*h)', 'n']
---------------------------------------------------------------------------------------
Kamstrup M66C telegram ontvangen op: 2017-07-13 22:35:11
Meter fabrikant/type: /KAM MC
 0. 0 - Meter identificatie: M66C_00004574909
 6. 8 - Meterstand Energie: 654.336 GJ
 6.26 - Meterstand Volume: 4823.206 m3
 6.31 - Meterstand Gebruiksduur: 118948.000 h
Einde M66C telegram
An other evening of 'hacking' fun.. As you can see my post has a lot of emotion in it.. haha. Succes makes me smile !!
Tomorrow going to join Paul to get his version working on site and mine to integrate into Domoticz counters.

Many thanks for the help so far !!!

Frank
Cletus
Posts: 54
Joined: Wednesday 17 August 2016 9:32
Target OS: Raspberry Pi / ODroid
Domoticz version: Latest
Location: Purmerend, Netherlands
Contact:

Re: Kamstrup Multical 402 Python Script

Post by Cletus »

freijn wrote: RTFM
in the manual of Joost van der Linde it tells you to use python 3 !!
Running python -v gave me version 2.3 AHAHA !
Running python3 -v gives me version 3 :-)
Seriously?!?

Wow............ Haven't tried that yet. LOL!

See you this evening Frank!
This is our world now... the world of the electron and the switch, the beauty of the baud.
Hesmink
Posts: 168
Joined: Monday 22 June 2015 10:48
Target OS: Raspberry Pi / ODroid
Domoticz version:
Location: The Netherlands
Contact:

Re: Kamstrup Multical 402 Python Script

Post by Hesmink »

So, in your case you use the 'simple' protocol, and not the KMP protocol.
My modifications are for the KMP protocol, since that works for my Kampstrup.

Confusing, that 2 protocols are used for the same type of hardware.
freijn
Posts: 536
Joined: Friday 23 December 2016 16:40
Target OS: Raspberry Pi / ODroid
Domoticz version: Stable
Location: Netherlands Purmerend
Contact:

Re: Kamstrup Multical 402 Python Script

Post by freijn »

Hesmink

I had some spare moment and decided to run it with your script as well.

After changing
import urllib2
into
import urllib
and added
import urllib.request

then also change the url into
requestPost = urllib.request.urlopen( "http://127.0.0.1:8080/json.htm?type=com ... evice&idx=" + str(index) + "&svalue=" + str(v$

it dumps me this error.

Any clue ?

Code: Select all

pi@raspberrypi:~/domoticz/scripts/python $ nano kamstrup.py                                                                                  pi@raspberrypi:~/domoticz/scripts/python $ python3 kamstrup.py /dev/ttyUSB1 60 262
waarde:
None
Traceback (most recent call last):
  File "kamstrup.py", line 249, in <module>
    requestPost = urllib.request.urlopen( "http://127.0.0.1:8080/json.htm?type=command&param=udevice&idx=" + str(index) + "&svalue=" + str(value) )
  File "/usr/lib/python3.4/urllib/request.py", line 153, in urlopen
    return opener.open(url, data, timeout)
  File "/usr/lib/python3.4/urllib/request.py", line 461, in open
    response = meth(req, response)
  File "/usr/lib/python3.4/urllib/request.py", line 571, in http_response
    'http', request, response, code, msg, hdrs)
  File "/usr/lib/python3.4/urllib/request.py", line 499, in error
    return self._call_chain(*args)
  File "/usr/lib/python3.4/urllib/request.py", line 433, in _call_chain
    result = func(*args)
  File "/usr/lib/python3.4/urllib/request.py", line 579, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 401: Unauthorized
pi@raspberrypi:~/domoticz/scripts/python $
Cletus
Posts: 54
Joined: Wednesday 17 August 2016 9:32
Target OS: Raspberry Pi / ODroid
Domoticz version: Latest
Location: Purmerend, Netherlands
Contact:

Re: Kamstrup Multical 402 Python Script

Post by Cletus »

freijn wrote:
Spoiler: show
Hesmink

I had some spare moment and decided to run it with your script as well.

After changing
import urllib2
into
import urllib
and added
import urllib.request

then also change the url into
requestPost = urllib.request.urlopen( "http://127.0.0.1:8080/json.htm?type=com ... evice&idx=" + str(index) + "&svalue=" + str(v$

it dumps me this error.

Any clue ?

Code: Select all

pi@raspberrypi:~/domoticz/scripts/python $ nano kamstrup.py                                                                                  pi@raspberrypi:~/domoticz/scripts/python $ python3 kamstrup.py /dev/ttyUSB1 60 262
waarde:
None
Traceback (most recent call last):
  File "kamstrup.py", line 249, in <module>
    requestPost = urllib.request.urlopen( "http://127.0.0.1:8080/json.htm?type=command&param=udevice&idx=" + str(index) + "&svalue=" + str(value) )
  File "/usr/lib/python3.4/urllib/request.py", line 153, in urlopen
    return opener.open(url, data, timeout)
  File "/usr/lib/python3.4/urllib/request.py", line 461, in open
    response = meth(req, response)
  File "/usr/lib/python3.4/urllib/request.py", line 571, in http_response
    'http', request, response, code, msg, hdrs)
  File "/usr/lib/python3.4/urllib/request.py", line 499, in error
    return self._call_chain(*args)
  File "/usr/lib/python3.4/urllib/request.py", line 433, in _call_chain
    result = func(*args)
  File "/usr/lib/python3.4/urllib/request.py", line 579, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 401: Unauthorized
pi@raspberrypi:~/domoticz/scripts/python $
Hi Frank,

HTTP error 401 seems an unauthenticated error. Can you check if 127.0.0.1 is whitelisted without username and password? Alternatively give your network IP instead of the localhost one? (192*.*.*)

Cheers, Paul
This is our world now... the world of the electron and the switch, the beauty of the baud.
Cletus
Posts: 54
Joined: Wednesday 17 August 2016 9:32
Target OS: Raspberry Pi / ODroid
Domoticz version: Latest
Location: Purmerend, Netherlands
Contact:

Re: Kamstrup Multical 402 Python Script

Post by Cletus »

Update: Frank visited my home and we got it to work!!! I will post the scripts we use down here.

Hopefully people have success with this!

Files:

Kamstrup.sh is the script to use selective stuff sending to IDX devices.
Kamstrup.py is screen logging only
Kamstrup_single.py is modified by someone else (left the names in the script! :)) for single command usage.

This script is now running on my Raspberry Pi for a week now with success!

Bug: If a meter ends with 0 that 0 will be taken off, need to figure out how to fix this still.

Reagrds, Paul
Attachments
kamstrup_402.zip
(6.26 KiB) Downloaded 464 times
This is our world now... the world of the electron and the switch, the beauty of the baud.
freijn
Posts: 536
Joined: Friday 23 December 2016 16:40
Target OS: Raspberry Pi / ODroid
Domoticz version: Stable
Location: Netherlands Purmerend
Contact:

Re: Kamstrup Multical 402 Python Script

Post by freijn »

Paul,

Not tested but it must be something like this, let me know if that works for you.


if __name__ == "__main__":

import time

comport = sys.argv[1]
command = int( sys.argv[2], 0)
index = int( sys.argv[3], 0)

foo = kamstrup( comport )
heat_timestamp=datetime.datetime.strftime(datetime.datetime.today(), "%Y-%m-%d %H:%M:%S" )

value,unit = foo.readvar( command )

#format the output
value = '{:06.2f}'.format(value)

requestPost = urllib.request.urlopen( "http://192.168.2.253:8080/json.htm?type ... evice&idx=" + str(index) + "&svalue=" + str(value) )
Cletus
Posts: 54
Joined: Wednesday 17 August 2016 9:32
Target OS: Raspberry Pi / ODroid
Domoticz version: Latest
Location: Purmerend, Netherlands
Contact:

Re: Kamstrup Multical 402 Python Script

Post by Cletus »

Hey Frank,

I tried it out but i got unexpected results as in that it trims the last digit now anyway.

Thanks!

Paul
This is our world now... the world of the electron and the switch, the beauty of the baud.
Post Reply

Who is online

Users browsing this forum: No registered users and 1 guest