TP-Link smart plug HS100/HS110

[cerocca]

@ajay100 : thanks for your "detailed" instructions, I followed them (maybe you can put them in the github repository) and everything worked fine.

BUT it worked fine for 2 days... today I went checking power consumption - I use the HS110 to monitor consumption of my water heater - and I saw no data for last day... it shown "last seen" 1 day ago, as if the HS was dead.
Of course it wasn't, neither it was switched off. I can see it from the Kasa app (and from the "TPLink Energy Monitor" I have implemented with this dashboard: https://github.com/jamesbarnett91/tplink-energy-monitor . It is nice but it doesn't keep history, this is why I want to use also your code)
I rebooted the raspberry and now again all seems fine, so my question is: is there a way to avoid these "dead" periods?
Who can have caused the interruption of data from the HS to the Domoticz server? The "TPLink Energy Monitor" ? I think not, but I cannot be 100% sure

[ajay100]

@cerocca, that would frustrate me too. My HS110 has been solid, so I can't guess what the problem might be. Can you check your logs to see if they show what happened? In my code (logger_name = "hs110-1"), they are named hs110-1_data.csv and hs110-1.log.

You may need to change from debug_level="INFO" to debug_level="DEBUG" to get more information.

Cheers - Andrew

[cerocca]

Hi ajay100,
don't ask me why but the script was already in DEBUD mode, this is the .log file between the last received data an my reboot of today (when I saw it was not recording anymore). Do you see something strange?
For your further reference I attach the whole .csv file, is there any more feedback i can give you (other debug modes) ?

thanks a lot!

Code: Select all

2018-10-18 23:54:55,657:__main__:DEBUG:Command: {"system":{"get_sysinfo":{}}}
2018-10-18 23:54:55,673:__main__:DEBUG:Sent:     {"system":{"get_sysinfo":{}}}
2018-10-18 23:54:55,675:__main__:DEBUG:Received: {"system":{"get_sysinfo":{"err_code":0,"sw_ver":"1.2.5 Build 171213 Rel.101523","hw_ver":"1.0","type":"IOT.SMARTPLUGSWITCH","model":"HS110(EU)","mac":"70:4F:57:1B:DB:37","deviceId":"800657AADA2A0C8635066D6ED760C98B18ECE692","hwId":"45E29DA8382494D2E82688B52A0B2EB5","fwId":"00000000000000000000000000000000","oemId":"3D341ECE302C0642C99E31CE2430544B","alias":"CiruPluggr","dev_name":"Wi-Fi Smart Plug With Energy Monitoring","icon_hash":"","relay_state":1,"on_time":973240,"active_mode":"schedule","feature":"TIM:ENE","updating":0,"rssi":-57,"led_off":0,"latitude":43.100680,"longitude":12.384625}}}
2018-10-18 23:54:55,677:__main__:DEBUG:URL: http://localhost:8080/json.htm?type=command&param=udevice&idx=22&nvalue=1
2018-10-18 23:55:10,724:__main__:DEBUG:Command: {"emeter":{"get_realtime":{}}}
2018-10-18 23:55:10,791:__main__:DEBUG:Sent:     {"emeter":{"get_realtime":{}}}
2018-10-18 23:55:10,793:__main__:DEBUG:Received: {"emeter":{"get_realtime":{"current":0.022858,"voltage":230.357978,"power":1.216032,"total":41.578000,"err_code":0}}}
2018-10-18 23:55:10,797:__main__:DEBUG:IDX: 23, Item: voltage, Value: 230.357978
2018-10-18 23:55:10,800:__main__:DEBUG:URL: http://localhost:8080/json.htm?type=command&param=udevice&nvalue=0&idx=23&svalue=230.357978
2018-10-18 23:55:10,834:__main__:DEBUG:IDX: 24, Item: current, Value: 0.022858
2018-10-18 23:55:10,838:__main__:DEBUG:URL: http://localhost:8080/json.htm?type=command&param=udevice&nvalue=0&idx=24&svalue=0.022858
2018-10-18 23:55:10,870:__main__:DEBUG:IDX: 25, Item: power, Value: 1.216032
2018-10-18 23:55:10,876:__main__:DEBUG:URL: http://localhost:8080/json.htm?type=command&param=udevice&nvalue=0&idx=25&svalue=1.216032
2018-10-18 23:55:10,908:__main__:DEBUG:IDX: 26, Items: Power (W), Usage (kWhr), Values: 1.216032;41.578
2018-10-18 23:55:10,910:__main__:DEBUG:URL: http://localhost:8080/json.htm?type=command&param=udevice&nvalue=0&idx=26&svalue=1.216032;41578.0
2018-10-20 09:31:27,209:__main__:INFO:cirhs110-log version 0.5 has started...
2018-10-20 09:31:28,351:__main__:DEBUG:Command: {"emeter":{"get_realtime":{}}}
2018-10-20 09:31:28,383:__main__:DEBUG:Sent:     {"emeter":{"get_realtime":{}}}
2018-10-20 09:31:28,386:__main__:DEBUG:Received: {"emeter":{"get_realtime":{"current":0.022443,"voltage":229.664393,"power":1.054664,"total":42.151000,"err_code":0}}}
2018-10-20 09:31:28,396:__main__:DEBUG:IDX: 23, Item: voltage, Value: 229.664393
2018-10-20 09:31:28,399:__main__:DEBUG:URL: http://localhost:8080/json.htm?type=command&param=udevice&nvalue=0&idx=23&svalue=229.664393
2018-10-20 09:31:28,497:__main__:DEBUG:IDX: 24, Item: current, Value: 0.022443
2018-10-20 09:31:28,501:__main__:DEBUG:URL: http://localhost:8080/json.htm?type=command&param=udevice&nvalue=0&idx=24&svalue=0.022443
2018-10-20 09:31:28,548:__main__:DEBUG:IDX: 25, Item: power, Value: 1.054664
2018-10-20 09:31:28,551:__main__:DEBUG:URL: http://localhost:8080/json.htm?type=command&param=udevice&nvalue=0&idx=25&svalue=1.054664
2018-10-20 09:31:28,595:__main__:DEBUG:IDX: 26, Items: Power (W), Usage (kWhr), Values: 1.054664;42.151
2018-10-20 09:31:28,597:__main__:DEBUG:URL: http://localhost:8080/json.htm?type=command&param=udevice&nvalue=0&idx=26&svalue=1.054664;42151.0
2018-10-20 09:31:28,656:__main__:DEBUG:Command: {"system":{"get_sysinfo":{}}}
2018-10-20 09:31:28,673:__main__:DEBUG:Sent:     {"system":{"get_sysinfo":{}}}
2018-10-20 09:31:28,675:__main__:DEBUG:Received: {"system":{"get_sysinfo":{"err_code":0,"sw_ver":"1.2.5 Build 171213 Rel.101523","hw_ver":"1.0","type":"IOT.SMARTPLUGSWITCH","model":"HS110(EU)","mac":"70:4F:57:1B:DB:37","deviceId":"800657AADA2A0C8635066D6ED760C98B18ECE692","hwId":"45E29DA8382494D2E82688B52A0B2EB5","fwId":"00000000000000000000000000000000","oemId":"3D341ECE302C0642C99E31CE2430544B","alias":"CiruPluggr","dev_name":"Wi-Fi Smart Plug With Energy Monitoring","icon_hash":"","relay_state":1,"on_time":1094233,"active_mode":"schedule","feature":"TIM:ENE","updating":0,"rssi":-57,"led_off":0,"latitude":43.100680,"longitude":12.384625}}}
2018-10-20 09:31:28,677:__main__:DEBUG:URL: http://localhost:8080/json.htm?type=command&param=udevice&idx=22&nvalue=1

[cerocca]

@ajay100,

just to tell you that this morning... same problem

Basically the script stops collecting data at the same time of yesterday, 23:55
Of course also in this case I had to reboot the raspberry
I attach the updated CSV file, I really don't know which can be the problem of my system... any suggestion? It would be greatly appreciated!

[ajay100]

Hi @cerocca,

It looks like the script is failing at the time it should be writing the daily file. Try setting text_logging = False to see how that goes.

You may not need a text log file anyway if you are happy with the history in Domoticz. The reason I added the text logging was so that I had historic values for each logging period, rather than just the minimum, maximum and average that Domoticz provides after one day. I hope to do other graphing with this data over time.

If you would like to keep the text logging, see if the file ending with '_daily.csv' exists. The code that sets up the daily file is in this section

Code: Select all

    if text_logging:
      self.next_daily_time = datetime.datetime.combine(datetime.date.today(),datetime.time(23,55,0))
      # Set up headers for log and daily files
      if not os.path.isfile(datafile):
        self.write_file(datafile,"w",datafile_columns + "\n")
      if not os.path.isfile(dailyfile):
        self.write_file(dailyfile,"w",dailyfile_columns + "\n")

The code that writes the daily log file is:

Code: Select all

      if datetime.datetime.now() > self.next_daily_time:
        self.next_daily_time = datetime.datetime.combine(datetime.date.today() + datetime.timedelta(days=1),datetime.time(23,55,0))
        out = time.strftime("%Y-%m-%d %H:%M:%S") + "," + str(usage) + "\n"
        self.write_file(dailyfile, "a", out)

I have set the arbitrary time of 11:55pm as the time to write the daily log file. There could be something about my date/time handling that doesn't work in your region (mine is Australia). As I'm not a python expert, I'm not sure.

Cheers - Andrew

[cerocca]

Hi @ajay100,

perfect ! I've set the text_logging as False, now no more failures, your script works like a charm for me.
Again congrats for your job! And thanks for your precious support

[vamp]

Hi there,

This plugin work with HS100?

https://github.com/dahallgren/domoticz-tplink-smartplug

It is a on/off plugin, or show the energy usage?

somebody able to test it?

HI!

Somebody test it?

[ajay100]

@vamp, I installed it but the plugin is not working for me at all (not showing in Domoticz). I think this is to do with my default Python being v2.7 on the RPi. I have v3 installed but I'm not sure how to force the plugin to use it. I don't want to make python v3 default for the system. I will keep investigating.

Cheers - Andrew

[MikeF]

I have installed the plugin, and got it working (with a HS110) after making changes following repeated KeyError messages - for which I have raised an issue on Github (q.v.). As with all plugins, you need to reboot Domoticz after saving the plugin file, and ‘Allow new devices for 5 minutes’ (in Settings). Incidentally my default version of Python (on RPi) is 2.7.

When configuring the plugin, you can specify if it is for a HS100 or HS110 - only the latter supports energy readings (power, voltage, current), whereas the HS100 only supports on / off.

[sadrov]

after making changes following repeated KeyError messages - for which I have raised an issue on Github (q.v.).

Hey Mike,
which changes did you make exactly? I was trying the plugin, was able to add the hardware but I get this from the log:
2018-11-08 20:58:37.556 Error: (hs110) ----> Line 262 in /home/pi/domoticz/plugins/tplink-smartplug-master/plugin.py, function onHeartbeat
2018-11-08 20:58:37.557 Error: (hs110) ----> Line 136 in /home/pi/domoticz/plugins/tplink-smartplug-master/plugin.py, function onHeartbeat
2018-11-08 20:58:37.557 Error: (hs110) ----> Line 198 in /home/pi/domoticz/plugins/tplink-smartplug-master/plugin.py, function update_emeter_values

I'm using the HS110 with firmware v1.5.4, python version on my domoticz on raspberry pi is 2.7.13
Do you have any advice please?
thanks a lot in advance

[sadrov]

I have now created a python script to send energy data from the TP-Link HS110 Smart Plug to Domoticz:

Spoiler: show

Code: Select all

#!/usr/bin/env python
# 
# TP-Link Wi-Fi Smart Plug Protocol Client
# For use with TP-Link HS110: energy monitor
# 
# Gets current power (W) and cumulative energy (kWh)
# and sends to Domoticz

import socket
import argparse
import json
import urllib
import urllib2

# ip, port, and command for HS110
ip = '<IP of HS110>'
port = 9999
cmd = '{"emeter":{"get_realtime":{}}}'

# Domoticz command stub and IDx of HS110
baseURL = 'http://<Domoticz IP:port>/json.htm?type=command&param=udevice&nvalue=0'
HSIdx = <Domoticz IDx of HS110>

# Encryption and Decryption of TP-Link Smart Home Protocol
# XOR Autokey Cipher with starting key = 171
def encrypt(string):
	key = 171
	result = "\0\0\0\0"
	for i in string: 
		a = key ^ ord(i)
		key = a
		result += chr(a)
	return result

def decrypt(string):
	key = 171 
	result = ""
	for i in string: 
		a = key ^ ord(i)
		key = ord(i) 
		result += chr(a)
	return result

def domoticzrequest (url):
   request = urllib2.Request(url)
#   request.add_header("Authorization", "Basic %s" % base64string)
   response = urllib2.urlopen(request)
   return None;

# Send command and receive reply 
try:
	sock_tcp = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
	sock_tcp.connect((ip, port))
	sock_tcp.send(encrypt(cmd))
	data = sock_tcp.recv(2048)
	sock_tcp.close()
	
#	print "Sent:     ", cmd
	result = decrypt(data[4:])
	jsonData = json.loads(result)
#	print "Received: "
#	print json.dumps(jsonData, indent=4, sort_keys=True)
	power = jsonData['emeter']['get_realtime']['power']
	total = jsonData['emeter']['get_realtime']['total'] * 1000
#	print power, total
except socket.error:
	quit("Cound not connect to host " + ip + ":" + str(port))

# Send data to Domoticz
try:
    url = baseURL + "&idx=%s&svalue=%s;%s" % (HSIdx, power, total)
    domoticzrequest(url)
except urllib2.URLError, e:
	print e.code

Create a virtual sensor with sensor type 'Electric (Instant+Counter)', and insert its IDx and the IP addresses of your HS110 and Domoticz server in the script. Then set up a crontab entry to run the script every 5 or 10 minutes, as you prefer.

It will appear in the Utilities page - here's an example:

Hi again Mike,
my HS110 is being recognized and I updated the script with the necessary variables but when I run it, I get the following:
Traceback (most recent call last):

File "./energydata.py", line 60, in <module>
jsonData = json.loads(result)
File "/usr/lib/python2.7/json/__init__.py", line 339, in loads
return _default_decoder.decode(s)
File "/usr/lib/python2.7/json/decoder.py", line 364, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/lib/python2.7/json/decoder.py", line 382, in raw_decode
raise ValueError("No JSON object could be decoded")
ValueError: No JSON object could be decoded

Have you got any idea where it's going wrong here? I have almost no clue about python and scripting so apologies if this is a stupid question.
I hope you can help me out.
Thanks a lot in advance.

[ajay100]

I have now got the HS100/110 Domoticz plugin (https://github.com/dahallgren/domoticz-tplink-smartplug) working on my RPi. Thanks to @MikeF for the tips!

The plugin was initially not recognised at all. After following the plugin guide, checking python versions, etc. over several days with no luck, I installed the Python Plugin Manager. Immediately both plugins appeared after the Domoticz restart! This could have been an issue specific to my system, but it was following a clean install of Domoticz v4.9700 on Debian Stretch.

After the plugin was recognised, I got the same error that @sandrov and MikeF reported. First up I changed to 'nValue=0 in lines 198-200 as per @MikeF's post. I turned debug on for the HS110 plugin and the following appeared in the Domoticz log:

2018-11-09 16:27:29.051 (HS110 Plugin) Pushing 'onHeartbeatCallback' on to queue
2018-11-09 16:27:29.095 (HS110 Plugin) Processing 'onHeartbeatCallback' message
2018-11-09 16:27:29.095 (HS110 Plugin) Calling message handler 'onHeartbeat'.
2018-11-09 16:27:29.285 (HS110 Plugin) data len: 114
2018-11-09 16:27:29.288 (HS110 Plugin) got response: {'emeter': {'get_realtime': {'voltage_mv': 242201, 'total_wh': 1560, 'err_code': 0, 'power_mw': 1646, 'current_ma': 60}}}
2018-11-09 16:27:29.288 Error: (HS110 Plugin) 'onHeartbeat' failed 'KeyError'.
2018-11-09 16:27:29.288 Error: (HS110 Plugin) ----> Line 262 in /home/pi/domoticz/plugins/hs110/plugin.py, function onHeartbeat
2018-11-09 16:27:29.288 Error: (HS110 Plugin) ----> Line 136 in /home/pi/domoticz/plugins/hs110/plugin.py, function onHeartbeat
2018-11-09 16:27:29.288 Error: (HS110 Plugin) ----> Line 198 in /home/pi/domoticz/plugins/hs110/plugin.py, function update_emeter_values

You can see from the JSON response that my HS110 is returning mV, mA and mW instead of V, A and W. As per the problem @Marc had with my code in this post: viewtopic.php?f=56&t=13290&start=100#p193180, this is the second cause of the KeyError.

Here are my working lines 198-200:

Code: Select all

                Devices[2].Update(nValue=0, sValue=str(realtime_result['current_ma'] / 1000))
                Devices[3].Update(nValue=0, sValue=str(realtime_result['voltage_mv'] / 1000))
                Devices[4].Update(nValue=0, sValue=str(realtime_result['power_mw'] / 1000)) 

The plugin is working well, so thanks also to @dahallgren for creating it. This will be very helpful if I decide to change my other monitoring code to plugins.

[sadrov]

thanks @ajay100 ! this did the trick!
after reading this topic all over again, I found the solutions to the problems I stated yesterday.
I guess the latest firmware of the HS110 made the scripts of pre July 2018 outdated.
Also thanks to @Olfer 's post

You'll have to...
* replace the encrypt and decrypt functions with new ones
* add the 'from struct import pack' line
* alter the json query to... ']['power_mw'] / 1000
* alter the json query to...']['total_wh']

I managed to make @MikeF 's energy data script work.
This is really fun
thanks to all the wonderful contributors out here!

[crazynight]

Having a real struggle getting the TPlink sockets working with domoticz, the plugin worked but did not report/update the current state of the socket which is what I need as there is a chance they are controlled by alexa or at kaza app.

I have tried https://github.com/ajay10000/TP-Link-HS110 by ajay10000 but the documentation I have found does not seem to match the file I downloaded 5 IDX's needed yet the documentation calls for just 4.

basic commands work fine for control of the socket.

python tplink_hs110.py -c off
Sent: {"system":{"set_relay_state":{"state":0}}}
Received: {"system":{"set_relay_state":{"err_code":0}}

but the status is not updated.

after near 10hrs on this starting to pull my hair out if you can help it would be much appreciated.

[liderbug]

Flame me all you want, but ... (kiss)

#!/bin/bash
cmdON="AAAAKtDygfiL/5r31e+UtsWg1Iv5nPCR6LfEsNGlwOLYo4HyhueT9tTu36Lfog=="
cmdOFF="AAAAKtDygfiL/5r31e+UtsWg1Iv5nPCR6LfEsNGlwOLYo4HyhueT9tTu3qPeow=="

case $1 in
on) echo -n "$cmdON" | base64 -d | nc sg104 9999 ;;
off) echo -n "$cmdOFF" | base64 -d | nc sg104 9999 ;;
*) echo "wtfo" ;;
esac

# myhs on
# myhs off

[ajay100]

Hi @crazynight,

In both python2 and python3 versions, there are four Domoticz indexes for the HS110 monitoring functions and the monitor_list array maps them;

Code: Select all

monitor_list = ["voltage","current","power","usage"]
domoticz_idx = [90,91,108,93]

Then there is the single index for the switch function, which feeds back the switch status to Domoticz:

Code: Select all

hs110_switch_idx = 107

Those index numbers apply to my Domoticz devices, but yours will probably be different. There is some commentary on the code in previous posts.

This is the code that sends the switch state to Domoticz:

Code: Select all

      if self.read_state:
        # For getting the switch state only
        state = json_data['system']['get_sysinfo']['relay_state']
        full_url = domain + "json.htm?type=command&param=udevice&idx={}&nvalue={}".format(hs110_switch_idx,state)
        logger.debug("URL: {}".format(full_url))
        # Send the json string
        req = urllib.request.Request(full_url)
        with urllib.request.urlopen(req) as response:
          result = response.read()
        logger.debug("Logger response: {}".format(result))

The switch device in Domoticz is set up to send on/off commands to the HS110. The python script can send the *actual* state of the switch back to Domoticz if it has been changed by another application or manually switched on the device itself.

Cheers - Andrew

[crazynight]

Code: Select all

python tplink_hs110.py -c state

gives me

Code: Select all

Traceback (most recent call last):
  File "tplink_hs110.py", line 238, in <module>
    hs.read_hs110()
  File "tplink_hs110.py", line 224, in read_hs110
    self.send_json(received_data)
  File "tplink_hs110.py", line 155, in send_json
    voltage = round(float(json_data['emeter']['get_realtime']['voltage_mv']) / 1000,2)
KeyError: 'emeter'

but does not update any of the switches in domoticz.

the on / off command does make the plug change state so I know comms to that is ok but again does not update domesticz.

this is the user editable section.

Code: Select all

# Begin user editable variables
version = 0.5
logger_name = "hs110-1"  #used for log file names, messages, etc
debug_level="INFO"  # debug options DEBUG, INFO, WARNING, ERROR, CRITICAL
delay_time = 15 #update time in seconds
domain="http://192.168.1.169:8080/"
base_url = domain + "json.htm?type=command&param=udevice&nvalue=0"
monitor_list = ["voltage","current","power","usage"]
domoticz_idx = [115,116,117,118]
hs110_ip = "192.168.1.180"
text_logging = True
track_state = True
hs110_switch_idx = 114
datafile_columns = "Time,Voltage,Current,Power (W),Usage (kWHr)"
dailyfile_columns = "Date-Time,Usage (kWHr)"
# End user editable variables

[ajay100]

@crazynight, can you post the result of

Code: Select all

python tplink_hs110.py -c info

[crazynight]

@ajay100

Sent: {"system":{"get_sysinfo":{}}}
Received: {"system":{"get_sysinfo":{"err_code":0,"sw_ver":"1.2.5 Build 171213 Rel.095335","hw_ver":"1.0","type":"IOT.SMARTPLUGSWITCH","model":"HS110(UK)","mac":"50:C7:BF:5B:12:47","deviceId":"800621A648D627AA69879D4B292084CC1878E40E","hwId":"2448AB56FB7E126DE5CF876F84C6DEB5","fwId":"00000000000000000000000000000000","oemId":"90AEEA7AECBF1A879FCA3C104C58C4D8","alias":"motorhome","dev_name":"Wi-Fi Smart Plug With Energy Monitoring","icon_hash":"","relay_state":0,"on_time":0,"active_mode":"schedule","feature":"TIM:ENE","updating":0,"rssi":-62,"led_off":0,"latitude":53.210092,"longitude":-2.917401}}}

[ajay100]

Any errors and/or logs for

Code: Select all

python tplink_hs110.py -c energy