Domoticz

Posted: **Monday 27 November 2017 18:44**

Hi,

I'm trying to integrate the stats from my Solar Hot Water system's "Resol DataLogger DL2" (http://amzn.to/2k4y1mm) into Domoticz. Well, "integrate" may be overstating it - I'm just scraping the stats off the live data/stats webpage (https://resoldl2.furrie.net/dl2/live/data) & pushing the data into some dummy/virtual sensors.

My shell script wgets the page & then processes the text into a state where I have a list of CSVs, which I then PUT into Domoticz on my Raspberry Pi, using curl.

Here's the wget (expanded, to aid visual delight): -

Code: Select all

wget --no-check-certificate --secure-protocol=TLSv1 -q -O - https://resoldl2.furrie.net/dl2/live/data \
  | grep "<td>" \
  | sed -e 's/^.*<td>//g' -e 's/&nbsp;//g' -e 's|</td>||g' \
  | dos2unix \
  | sed -e ':a;N;$!ba;s/:\n/,/g' \
  | grep -v 'relay 2' \
  | grep -v defective \
  | grep -v temperature \
  | grep -v operation \
  | sed -e 's/Temperature sensor 1/145,Solar Collector Temp (°C)/' \
    -e 's/Temperature sensor 2/146,Hot-Water Tank Bottom Temp (°C)/' \
    -e 's/Temperature sensor 3/147,Hot-Water Tank Top (°C)/' \
    -e 's/Temperature sensor 4/148,Pump Station Return Temp (°C)/' \
    -e 's/Pump speed relay 1/149,Pump Duty-Cycle (%)/' \
    -e 's/Operating hours relay 1/150,Pump Duration (h)/' \
    -e 's/Heat quantity/151,Heat Generation (kWh)/' \
  | sed -e 's/,\(.*\) .*$/,\1/g' \
  | grep -v DL2 \
  | grep -v Status \
  | grep -v Programme \
  | grep -v Version

Here's the curl output that is almost 100% successful: -

Code: Select all

145,Solar Collector Temp (°C),22.1
/usr/bin/curl -S -X PUT http://domoticz.furrie.net/json.htm?type=command&param=udevice&idx=145&nvalue=0&svalue=22.1
{
   "status" : "OK",
   "title" : "Update Device"
}

146,Hot-Water Tank Bottom Temp (°C),23.8
/usr/bin/curl -S -X PUT http://domoticz.furrie.net/json.htm?type=command&param=udevice&idx=146&nvalue=0&svalue=23.8
{
   "status" : "OK",
   "title" : "Update Device"
}

147,Hot-Water Tank Top (°C),46.0
/usr/bin/curl -S -X PUT http://domoticz.furrie.net/json.htm?type=command&param=udevice&idx=147&nvalue=0&svalue=46.0
{
   "status" : "OK",
   "title" : "Update Device"
}

148,Pump Station Return Temp (°C),15.2
/usr/bin/curl -S -X PUT http://domoticz.furrie.net/json.htm?type=command&param=udevice&idx=148&nvalue=0&svalue=15.2
{
   "status" : "OK",
   "title" : "Update Device"
}

149,Pump Duty-Cycle (%),0
/usr/bin/curl -S -X PUT http://domoticz.furrie.net/json.htm?type=command&param=udevice&idx=149&nvalue=0&svalue=0
{
   "status" : "OK",
   "title" : "Update Device"
}

150,Pump Duration (h),9307
/usr/bin/curl -S -X PUT http://domoticz.furrie.net/json.htm?type=command&param=udevice&idx=150&nvalue=0&svalue=9307
{
   "status" : "OK",
   "title" : "Update Device"
}

151,Heat Generation (kWh),106677.31 
/usr/bin/curl -S -X PUT http://domoticz.furrie.net/json.htm?type=command&param=udevice&idx=151&nvalue=0&svalue=106677.31 
<html><head><title>Bad Request</title></head><body><h1>400 Bad Request</h1></body></html>

This is infuriating, because the last command that fails in the script, actually does work if I issue it manually, a la: -

Code: Select all

% /usr/bin/curl -S -X PUT http://domoticz.furrie.net/json.htm\?type\=command\&param\=udevice\&idx\=151\&nvalue\=0\&svalue\=106677.31 
{
   "status" : "OK",
   "title" : "Update Device"
}

or with single quotes instead of the escapes: -

Code: Select all

% /usr/bin/curl -S -X PUT 'http://domoticz.furrie.net/json.htm?type=command&param=udevice&idx=151&nvalue=0&svalue=106677.31'           
{
   "status" : "OK",
   "title" : "Update Device"
}

I reverse-sorted the wget output, to test if it was the for loop that was borking on the last iteration, but the first one fails.

I have enabled -v and -S on curl and can see no real difference in the successful & unsuccessful requests. Is there something I can do on the Domoticz side, to figure out what's occurring? I wondered if the value I'm inserting is such a high number of kWh, but it DOES work manually, so probably not. Anyway, looking forward to any advice, cheers!

Here's some screenshots of the stats as shown in Domoticz

: DomoticzSolarHWUtility.png (38.66 KiB) Viewed 561 times

: DomoticzSolarHWTemps.png (54.8 KiB) Viewed 561 times

: DomoticzSolarHWDevices.png (80.88 KiB) Viewed 561 times

Posted: **Tuesday 05 December 2017 23:07**

Well, this was stupidly easy to resolve, although quite the swine to actually diagnose. Long story short, I highlighted the screen output & noticed some trailing whitespace on the errant element. The DL2's '/dl2/live/data' page places 2 spaces between the value and the unit - ONLY on this value! I fixed this by adding this sed operation, before the final (existing) sed operation; all it does is turn any double-spaces into a single space - which then gets handled by the previous script's final sed operation: -

Code: Select all

-e 's/  / /g' \

This is my script, which I'm running from crontab, every 5 minutes (with ' 2>&1 >/dev/null', to prevent cron emails); I understand the sun doesn't shine at night, but the tank temperatures are still subject to change - it gets cold/rains/snows/etc and more importantly, I have a gas boiler that comes on to bring the top of the tank to the required temp & that's never been monitored - this costs me money, so I want to stay abreast of any issues.

Code: Select all

#!/bin/zsh
#
## SolarHW-to-Domoticz

SolarHW='https://<DL2Hostname:Port>/dl2/live/data'
DomoticzHost='<DomoticzHostname:Port>'

wget=`which wget`
curl=`which curl`

## Tweak IFS so that we separate on newline, not spaces
## Record the old value
OLDIFS="$IFS"
## Tweak it
IFS=$'\n'

for stat in $(
    $wget --no-check-certificate --secure-protocol=TLSv1 -q -O - $SolarHW \
        | grep "<td>" \
        | sed -e 's/^.*<td>//g' \
            -e 's/&nbsp;//g' \
            -e 's|</td>||g' \
        | dos2unix \
        | sed -e ':a;N;$!ba;s/:\n/,/g' \
        | grep -v 'relay 2' \
        | grep -v defective \
        | grep -v temperature \
        | grep -v operation \
        | grep -v DL2 \
        | grep -v Status \
        | grep -v Programme \
        | grep -v Version \
        | sed -e 's/Temperature sensor 1/145,Solar Collector Temp (°C)/' \
            -e 's/Temperature sensor 2/146,Hot-Water Tank Bottom Temp (°C)/' \
            -e 's/Temperature sensor 3/147,Hot-Water Tank Top (°C)/' \
            -e 's/Temperature sensor 4/148,Pump Station Return Temp (°C)/' \
            -e 's/Pump speed relay 1/149,Pump Duty-Cycle (%)/' \
            -e 's/Operating hours relay 1/150,Pump Duration (h)/' \
            -e 's/Heat quantity/151,Heat Generation (kWh)/' \
        | sed -e 's/  / /g' \
            -e 's/,\(.*\) .*$/,\1/g'
)
do
    echo -e "\e[1;4;35m"
    printf '%s\n' "$stat"
    IDX=`echo $stat | cut -f1 -d,`
    STAT=`echo $stat | cut -f3 -d,`
    echo -e -n "\e[1;4;33m"
    echo $curl -s -X PUT "http://$DomoticzHost/json.htm?type=command&param=udevice&idx=$IDX&nvalue=0&svalue=$STAT"
    echo -e -n "\e[0m"
    $curl -s -X PUT "http://$DomoticzHost/json.htm?type=command&param=udevice&idx=$IDX&nvalue=0&svalue=$STAT"
done
echo ''

## Return IFS to normal
IFS="$OLDIFS"

exit 0

#EOF

I've published the code-proper, on Github: https://github.com/furriephillips/ResolDL2-to-Domoticz

Domoticz

[SOLVED] Integrating DeltaSol DL2

[SOLVED] Integrating DeltaSol DL2

Re: Integrating DeltaSol DL2