GPIO meter pulse counting with instant power calculation

Moderator: leecollings

RidingTheFlow
Posts: 72
Joined: Friday 11 March 2016 18:23
Target OS: Raspberry Pi / ODroid
Domoticz version:
Location: Essex, UK
Contact:

GPIO meter pulse counting with instant power calculation

Post by RidingTheFlow »

After some testing I've done (and it seems robust), this is Python script to count pulses from power and gas meter. Could be easily extended/changed for water meter too.

This script is special in fashion that it does not just count pulses, it actually calculates your current electric power usage in watts, and pretty accurately too - at least for usual electric meters which blink once per Wh.
It uses new gpiozero library which is a lot better protection against false triggers/cable noise than standard wiringPi. This should help you if you did get discrepancy in pulse counts/pulses counting twice, etc.
It uses interrupts so it should never miss pulses unlike polling, and won't use CPU unnecessarily.

Installation instructions: please make sure you've done them before complaining that script doesn't work
- install gpiozero by sudo apt-get install python-gpiozero
- if you haven't already, create a "Dummy hardware" in Domoticz
- on Dummy hardware click "Create Virtual Sensors"
- create a sensor with any name you like and type "Electric: Instant + Counter"
- create a sensor type "Gas"
- go to "Devices" menu. Write down the "Idx" column for each counters you just created. Edit the script to put these numbers in GAS_IDX= and ELEC_IDX= at the script beginning
- Set GAS_GPIO= and ELEC_GPIO= at script beginning to actual GPIO numbers your sensors connected to

Code: Select all

#!/usr/bin/env python

import time
import json
import urllib2
import threading
import logging
import logging.handlers
import os
from gpiozero import DigitalInputDevice

GET_URL = 'http://127.0.0.1:8080/json.htm?type=devices&rid=%d'
SET_URL = 'http://127.0.0.1:8080/json.htm?type=command&param=udevice&idx=%d&svalue=%d'
GAS_DELTA = 0
GAS_IDX =
GAS_GPIO = 
GAS_COUNTER_LOCK = threading.Lock()
ELEC_DELTA = 0
ELEC_IDX =
ELEC_GPIO = 
ELEC_COUNTER_LOCK = threading.Lock()
ELEC_LAST_TIME = 0
ELEC_POST_TIME = 0

def gas_intr():
    global GAS_DELTA
    with GAS_COUNTER_LOCK:
        GAS_DELTA += 10
    logging.debug( 'Gas counter tick: %d' % GAS_DELTA )

def elec_intr():
    tme = time.time()
    global ELEC_DELTA
    global ELEC_LAST_TIME
    global ELEC_POST_TIME
    with ELEC_COUNTER_LOCK:
        ELEC_LAST_TIME = tme
        if ELEC_POST_TIME == 0:
            ELEC_POST_TIME = ELEC_LAST_TIME
        else:
            ELEC_DELTA += 1
    logging.debug( 'Electricity counter tick: %d' % ELEC_DELTA )
    
def main():
    global GAS_DELTA
    global GAS_COUNTER
    global ELEC_DELTA
    global ELEC_LAST_TIME
    global ELEC_POST_TIME
    global ELEC_COUNTER

    syslog = logging.handlers.SysLogHandler(address='/dev/log', facility='local1')
    syslog.setFormatter(logging.Formatter('local_sensors.py: %(levelname)s %(message)s'))
    logging.getLogger().addHandler(syslog)
    logging.getLogger().setLevel(logging.INFO)

    while True:
        try:
            res = json.load(urllib2.urlopen(GET_URL % GAS_IDX))
            if res['status'] != 'OK':
                raise Exception('Domoticz json error')
            break
        except Exception as e:
            logging.warning( e )
        time.sleep(30.0)

    GAS_COUNTER = int(float(res['result'][0]['Data']) * 1000)
#    GAS_COUNTER = <Your initial count here * 1000, don't forget to remove after Domoticz updated!>
    logging.info( 'Current gas counter is: %d' % GAS_COUNTER )

    while True:
        try:
            res = json.load(urllib2.urlopen(GET_URL % ELEC_IDX))
            if res['status'] != 'OK':
                raise Exception('Domoticz json error')
            break
        except Exception as e:
            logging.warning( e )
        time.sleep(30.0)

    ELEC_COUNTER = int(float(res['result'][0]['Data'][:-4]) * 1000)
#    ELEC_COUNTER = <Your initial count here * 1000, don't forget to remove after Domoticz updated!>
    logging.info( 'Current electricity counter is: %d' % ELEC_COUNTER )

    elecSensor = DigitalInputDevice(ELEC_GPIO, pull_up=True)
    elecSensor.when_deactivated = elec_intr

    gasSensor = DigitalInputDevice(GAS_GPIO, pull_up=True)
    gasSensor.when_activated = gas_intr  
    
    os.nice(-20)

    logging.info('Polling loop starting')

    while True:
        time.sleep(60)

        GAS_DELTA_POST = GAS_DELTA

        if GAS_DELTA_POST != 0:
            GAS_COUNTER += GAS_DELTA_POST
            with GAS_COUNTER_LOCK:
                GAS_DELTA -= GAS_DELTA_POST
            try:
                res = json.load(urllib2.urlopen(SET_URL % (GAS_IDX, GAS_COUNTER)))
                if res['status'] != 'OK':
                    raise Exception('Domoticz json error')
                logging.info( 'Gas counter %d' % GAS_COUNTER )
            except Exception as e:
                logging.warning( e )

        with ELEC_COUNTER_LOCK:
            if ELEC_LAST_TIME > ELEC_POST_TIME:
                ELEC_LOAD = ELEC_DELTA * 3600 / ( ELEC_LAST_TIME - ELEC_POST_TIME )
            else:
                ELEC_LOAD = 0

            ELEC_COUNTER += ELEC_DELTA

            ELEC_DELTA = 0
            ELEC_POST_TIME = ELEC_LAST_TIME

        if ELEC_LOAD != 0:
            try:
                res = json.load(urllib2.urlopen((SET_URL+';%d') % (ELEC_IDX, int(ELEC_LOAD), ELEC_COUNTER)))
                if res['status'] != 'OK':
                    raise Exception('Domoticz json error')
                logging.info('Elec load %.2f counter %d' % (ELEC_LOAD, ELEC_COUNTER) )
            except Exception as e:
                logging.warning( e )

if __name__=="__main__":
    main()  
If you want, you can also put down initial meter values in the script, just don't forget to remove them after script updated the counters in Domoticz, otherwise on each script run it will reset meter values back to this value and forget current one.
I recommend you run script on root user since it will try to raise its own priority to prevent losing pulses if CPU is busy by some other process. If you don't want to run as root, you can remove os.nice line.
Script uses syslog on local1 facility, so you may change this for dedicated file or raise logging level to DEBUG if you want to debug when pulses are counters.

The hardware I've used:
Optical sensor module with photodiode (I recommend using photodiode and not photoresistor since diode is better for quick pulses) - http://www.ebay.co.uk/itm/371350785169? ... EBIDX%3AIT. Note that it signals by pulling line *low* (so its Vcc output when there is *no light detected*, and drops to zero when light detected). Please make sure you tune it properly, this is critical! First rotate potentiometer slowly to find where it just switches to "light detected" always, even when meter is not blinking and output stays steady low, note the screw position. Then rotate potentiometer until it *always no light detected" and it stops seeing the pulses, note the screw position. Then set screw roughly *in the middle between these two extremes*. This will give you most reliable pulse detection even if meter blinks very quickly. This sensor outputs at same voltage as Vcc, so make sure you power it from 3.3V pin, don't use 5V power since this will damage your GPIO!.

For gas meter I detected a magnet which was located *after* the last digit disk. I've used this reed switch - http://uk.farnell.com/hamlin/miti-3v1-6 ... 7-00001003. If you want to use other switch, note that you better choose one which is sensitive enough (lower "AT activation" number, the better). You simply connect this to connect GPIO to ground, as with usual switch.

Note that I strongly recommend to use around 2K pullup resistors just around start of your cable between each GPIO signal line and 3.3V. This will greatly reduce interference and possibility of false ticks. But I also recommend to use shielded cable between Pi and sensor - for example Cat7 Ethernet is a good and cheap choice (e.g. you could just solder RJ45 sockets at sensor and to 40Pin cable near Pi and use store-bought Cat7 patch cables to connect things neatly).

This gives very cheap but very functional setup to keep track of your utility cost even if you don't have smart meters.
Pinguin
Posts: 9
Joined: Thursday 10 March 2016 17:38
Target OS: NAS (Synology & others)
Domoticz version:
Location: Switzerland
Contact:

Re: GPIO meter pulse counting with instant power calculation

Post by Pinguin »

Thanks for your answer, I will try it in a few days.
I started with Domoticz end of last year on a synology Ds213. I have to figure how to insert your script. I use ESP 8266 boards with ESP easy because with a few clicks it configured and ready to use; adding a sensor is made in 3 minutes
As I'm an electronic engineer, I'm don't have any problems with the sensors, I never use the LDR, I prefer IR diodes as they are not sensitive to ambient light.
Concerning the gas meter, I 've asked the gas company to provide me the original sensor for the gas meter. They installed me one for free. If you are interested how is made the counting system of a gas counter, I have disassembled an old one and we can see the magnet..
I'll keep you updated in a few weeks.
best regards
Pinguin
Posts: 9
Joined: Thursday 10 March 2016 17:38
Target OS: NAS (Synology & others)
Domoticz version:
Location: Switzerland
Contact:

Re: GPIO meter pulse counting with instant power calculation

Post by Pinguin »

just one more remark about he sensor you use. I bought the same one and had some high frequency oscillations with it. This is due to the fact there is no hysteresis on the comparator. this can be modified by adding a simple resistor.
some tools:
http://www.daycounter.com/Calculators/C ... ator.phtml
http://sim.okawa-denshi.jp/en/compkeisan.htm


enjoy
User avatar
Marci
Posts: 531
Joined: Friday 22 January 2016 18:00
Target OS: Raspberry Pi / ODroid
Domoticz version: 3.8153
Location: Wakefield, West Yorkshire UK
Contact:

Re: GPIO meter pulse counting with instant power calculation

Post by Marci »

Have a look at https://github.com/PatchworkBoy/gaspuls ... r/gaspulse for ideas to expand/improve... This monitors interrupts, applies software debouncing to deal with hi-freq oscillations, writes to backup log files, and provides proper watt/kW conversions from m3 gas units.
Extended Domoticz homebridge-plugin for latest Homebridge - adds temp/humidity/pressure sensors, power consumption sensors, DarkSkies virtual weather station support, YouLess Meter support, general % usage support & switch/lamp status checking!
RidingTheFlow
Posts: 72
Joined: Friday 11 March 2016 18:23
Target OS: Raspberry Pi / ODroid
Domoticz version:
Location: Essex, UK
Contact:

Re: GPIO meter pulse counting with instant power calculation

Post by RidingTheFlow »

Pinguin wrote:Thanks for your answer, I will try it in a few days.
I started with Domoticz end of last year on a synology Ds213. I have to figure how to insert your script. I use ESP 8266 boards with ESP easy because with a few clicks it configured and ready to use; adding a sensor is made in 3 minutes
Personally I wouldn't use ESP since it requires exposing Domoticz over WiFi, but to each their own I guess. But if you use the ESP8266, you don't need this script. You just run counter/calc/debouncing directly on ESP, and ESP directly submits counter values to Domoticz.
This script was made specifically because my meters are in utility cupboard ~1 meter away from my Pi hub and adding extra controllers for them (which will need wires for power anyway) are rather silly.
I prefer IR diodes as they are not sensitive to ambient light.
I didn't use IR diode because I was not sure how much of IR component is in my meter's pulse diode and didn't want to buy extra stuff if IR didn't work reliably. Ambient light is not an issue here since this photodiode is focused and when mounted "face to face" with meters emitter inside a mounting bracket does not really take side light.
Pinguin wrote:just one more remark about he sensor you use. I bought the same one and had some high frequency oscillations with it. This is due to the fact there is no hysteresis on the comparator. this can be modified by adding a simple resistor.
some tools:
http://www.daycounter.com/Calculators/C ... ator.phtml
http://sim.okawa-denshi.jp/en/compkeisan.htm
enjoy
Oh, thanks, I now understand why this sensor initially gave me double triggers on each pulse. Luckily gpiozero has very good software filtering and eliminated these without any need to tweak actual sensor. I needed gpiozero filtering anyway because without it I was getting false pulses due to RF noise in my elec cupboard when some high loads switched on/off.
Reinz
Posts: 5
Joined: Wednesday 27 May 2015 11:29
Target OS: Raspberry Pi / ODroid
Domoticz version: 3.5877
Location: Austria
Contact:

Re: GPIO meter pulse counting with instant power calculation

Post by Reinz »

Could you explain your script a little, I would like to adapt it for my meter with 75 revolutions per kWh.
Also I tried to comment out the gas metering but since I have no python experience I totally broke it ;)

thanks in advance
RidingTheFlow
Posts: 72
Joined: Friday 11 March 2016 18:23
Target OS: Raspberry Pi / ODroid
Domoticz version:
Location: Essex, UK
Contact:

Re: GPIO meter pulse counting with instant power calculation

Post by RidingTheFlow »

If you want to change pulse ratio, you'll need to change power calculation like

Code: Select all

ELEC_LOAD = ELEC_DELTA * 3600 * 1000 /  ( 75 * ( ELEC_LAST_TIME - ELEC_POST_TIME ) )
It won't be very accurate as 1 pulse per wh but still would give an idea of current total wattage, esp. for hungry loads like heaters/cookers.

Also you will need to change electricity counter divider in Domoticz settings to 75, or multiply by 1000/75 when data is submitted to domoticz.
Reinz
Posts: 5
Joined: Wednesday 27 May 2015 11:29
Target OS: Raspberry Pi / ODroid
Domoticz version: 3.5877
Location: Austria
Contact:

Re: GPIO meter pulse counting with instant power calculation

Post by Reinz »

Thank you, I'll give it a try
JarekP
Posts: 15
Joined: Monday 16 May 2016 10:16
Target OS: Raspberry Pi / ODroid
Domoticz version:
Contact:

Re: GPIO meter pulse counting with instant power calculation

Post by JarekP »

Forgive a question from newbie, but what should I do witch such script? :)
RidingTheFlow
Posts: 72
Joined: Friday 11 March 2016 18:23
Target OS: Raspberry Pi / ODroid
Domoticz version:
Location: Essex, UK
Contact:

Re: GPIO meter pulse counting with instant power calculation

Post by RidingTheFlow »

Run it as Linux background process, for example start when system boots up and it will sit here counting pulses & submitting data to domoticz.
JarekP
Posts: 15
Joined: Monday 16 May 2016 10:16
Target OS: Raspberry Pi / ODroid
Domoticz version:
Contact:

Re: GPIO meter pulse counting with instant power calculation

Post by JarekP »

I tried to do it, but something is wrong, please see my topic:

http://www.domoticz.com/forum/viewtopic ... 654c1986b2
User avatar
Veikko
Posts: 4
Joined: Tuesday 20 December 2016 3:00
Target OS: Raspberry Pi / ODroid
Domoticz version: V3.5877
Contact:

Re: GPIO meter pulse counting with instant power calculation

Post by Veikko »

Hi!

Thak you very much for that script.
I got this working and also I can see data in Domoticz.
Unfortunately it seems that it shows only 1/10 of real usage.
I have a meter where 1000 impulses is one Kw so your script should be correct also for me.

Do you have any suggestions what should I check?

Thanks in advance and a Merry Christmast you all!
Dr. Lanning: "One day they'll have secrets, one day they'll have dreams..."
RidingTheFlow
Posts: 72
Joined: Friday 11 March 2016 18:23
Target OS: Raspberry Pi / ODroid
Domoticz version:
Location: Essex, UK
Contact:

Re: GPIO meter pulse counting with instant power calculation

Post by RidingTheFlow »

Veikko wrote:Hi!

Unfortunately it seems that it shows only 1/10 of real usage.
I have a meter where 1000 impulses is one Kw so your script should be correct also for me.

Do you have any suggestions what should I check?
In Domoticz settings check Meter/Counters => RFXMeter/Counter Dividers, maybe its not 1000 here.
User avatar
Veikko
Posts: 4
Joined: Tuesday 20 December 2016 3:00
Target OS: Raspberry Pi / ODroid
Domoticz version: V3.5877
Contact:

Re: GPIO meter pulse counting with instant power calculation

Post by Veikko »

It was inside the script. got it working.
Dr. Lanning: "One day they'll have secrets, one day they'll have dreams..."
Abbadon
Posts: 40
Joined: Thursday 01 October 2015 8:25
Target OS: Raspberry Pi / ODroid
Domoticz version: beta
Location: Poland, Wrocław
Contact:

Re: GPIO meter pulse counting with instant power calculation

Post by Abbadon »

i modiffied you script with my idx, changed ip i run as root, i can see gpio change with 'watch -d -n 0,1 gpio readall' but nothing happens, in log file i can see:

Code: Select all

pi@raspberrypi:~$ cat /var/log/syslog |grep local_sensors.py
Mar 10 20:19:40 raspberrypi local_sensors.py: INFO Current gas counter is: 1000
Mar 10 20:19:40 raspberrypi local_sensors.py: INFO Current electricity counter is: 0
Mar 10 20:22:31 raspberrypi local_sensors.py: INFO Current gas counter is: 1000
Mar 10 20:22:31 raspberrypi local_sensors.py: INFO Current electricity counter is: 0
Mar 10 20:22:39 raspberrypi local_sensors.py: INFO Current gas counter is: 1000
Mar 10 20:22:39 raspberrypi local_sensors.py: INFO Current electricity counter is: 0
Mar 10 20:22:39 raspberrypi local_sensors.py: INFO Polling loop starting
Mar 10 20:24:59 raspberrypi local_sensors.py: INFO Current gas counter is: 1000
Mar 10 20:24:59 raspberrypi local_sensors.py: INFO Current electricity counter is: 0
Mar 10 20:24:59 raspberrypi local_sensors.py: INFO Polling loop starting
Mar 10 20:27:11 raspberrypi local_sensors.py: INFO Current gas counter is: 1000
Mar 10 20:27:11 raspberrypi local_sensors.py: INFO Current electricity counter is: 0
Mar 10 20:27:11 raspberrypi local_sensors.py: INFO Polling loop starting
Mar 10 20:29:29 raspberrypi local_sensors.py: INFO Current gas counter is: 1000
Mar 10 20:29:30 raspberrypi local_sensors.py: INFO Current electricity counter is: 0
Mar 10 20:29:30 raspberrypi local_sensors.py: INFO Polling loop starting
Mar 10 20:33:50 raspberrypi local_sensors.py: INFO Current gas counter is: 1000
Mar 10 20:33:50 raspberrypi local_sensors.py: INFO Current electricity counter is: 0
Mar 10 20:33:50 raspberrypi local_sensors.py: INFO Polling loop starting
Mar 10 20:34:48 raspberrypi local_sensors.py: INFO Current gas counter is: 1000
Mar 10 20:34:48 raspberrypi local_sensors.py: INFO Current electricity counter is: 0
Mar 10 20:34:48 raspberrypi local_sensors.py: INFO Polling loop starting
Mar 10 20:39:24 raspberrypi local_sensors.py: INFO Current gas counter is: 0
Mar 10 20:39:24 raspberrypi local_sensors.py: INFO Current electricity counter is: 0
Mar 10 20:40:05 raspberrypi local_sensors.py: INFO Current gas counter is: 0
Mar 10 20:40:05 raspberrypi local_sensors.py: INFO Current electricity counter is: 0
Mar 10 20:40:05 raspberrypi local_sensors.py: INFO Polling loop starting
Mar 10 20:43:18 raspberrypi local_sensors.py: INFO Current gas counter is: 0
Mar 10 20:43:18 raspberrypi local_sensors.py: INFO Current electricity counter is: 0
Mar 10 20:43:18 raspberrypi local_sensors.py: INFO Polling loop starting
Mar 10 20:46:35 raspberrypi local_sensors.py: INFO Current gas counter is: 0
Mar 10 20:46:35 raspberrypi local_sensors.py: INFO Current electricity counter is: 0
Mar 10 20:46:35 raspberrypi local_sensors.py: INFO Polling loop starting
Mar 10 20:47:33 raspberrypi local_sensors.py: INFO Current gas counter is: 0
Mar 10 20:47:33 raspberrypi local_sensors.py: INFO Current electricity counter is: 1200
Mar 10 20:47:33 raspberrypi local_sensors.py: INFO Polling loop starting

RidingTheFlow
Posts: 72
Joined: Friday 11 March 2016 18:23
Target OS: Raspberry Pi / ODroid
Domoticz version:
Location: Essex, UK
Contact:

Re: GPIO meter pulse counting with instant power calculation

Post by RidingTheFlow »

Abbadon wrote:i modiffied you script with my idx, changed ip i run as root, i can see gpio change with 'watch -d -n 0,1 gpio readall' but nothing happens, in log file i can see:
Try to change logging level to debug

logging.getLogger().setLevel(logging.DEBUG)

And see if gpio interrupts actually get called.
Abbadon
Posts: 40
Joined: Thursday 01 October 2015 8:25
Target OS: Raspberry Pi / ODroid
Domoticz version: beta
Location: Poland, Wrocław
Contact:

Re: GPIO meter pulse counting with instant power calculation

Post by Abbadon »

idk what was the problem nut now it's working ;o
mosjonathan
Posts: 40
Joined: Friday 24 February 2017 21:20
Target OS: Linux
Domoticz version: Beta
Location: Netherlands
Contact:

Re: GPIO meter pulse counting with instant power calculation

Post by mosjonathan »

sorry for opening a old post,

i try to get this script working but i get the following error:
GAS_COUNTER = int(float(res['result'][0]['Data']))
KeyError: 'result'
mosjonathan
Posts: 40
Joined: Friday 24 February 2017 21:20
Target OS: Linux
Domoticz version: Beta
Location: Netherlands
Contact:

Re: GPIO meter pulse counting with instant power calculation

Post by mosjonathan »

i fixed the error by updating the couters with json before running the script
loyske
Posts: 12
Joined: Wednesday 19 August 2015 13:43
Target OS: Raspberry Pi / ODroid
Domoticz version:
Location: Netherlands
Contact:

Re: GPIO meter pulse counting with instant power calculation

Post by loyske »

Thanks for the script!

I use this script to count pulses via S0 from a Finder kwh meter, which logs the generated power and energy from my Steca500 solar inverter.
The only problem i see, is that if there are no pulses anymore, the calculated power stays at the last value seen.
Can someone explain how to modify the script, that if for 15 minutes no pulse, then power = 0 watts?

Thanks!
Post Reply

Who is online

Users browsing this forum: No registered users and 0 guests