Issue with Darksky and email notifications when using Domoticz Beta version

[meal]

Hello,

I am a new comer on the forum and I appreciate a lot Domoticz as a Home Automation System.

1 – Some elements of context

1.1 I have a Raspberry PI3 running perfectly Domoticz Version: 4.9700 using database domoticz.db with RFXCOM and ZWAVE, Darksky wheather. I also use notifications by email.
In the document I will call it PI PROD.
This the configuration of the Domoticz Version:
- Build Hash: a3a45906
- Compile Date: 2018-06-23 16:24:51
- dzVents Version: 2.4.6
- Python Version: 3.5.3 (default, Sep 27 2018, 17:25:39) [GCC 6.3.0 20170516]
And of the OS :
- Distributor ID: Raspbian
- Description: Raspbian GNU/Linux 9.6 (stretch)
- Release: 9.6
- Codename: stretch

1.2 I have a second Raspberry PI3 used as a sandbox where I want to install the latest Domoticz Beta Version.
In the document I will call it PI SANDBOX.
I first upgraded the OS to release 9.8.
Then I built from scratch Domoticz Beta version on 28/2/2019 (date of source clones)
using the following procedure
Step Command
1 sudo apt update
2 sudo apt upgrade
3 cat /etc/debian_version
4 cd /home/pi
5 mkdir DEV
6 cd /home/pi/DEV
7 sudo apt-get install libudev-dev
8 git clone

9 ln -s open-zwave open-zwave-read-only
10 cd open-zwave
11 Make
12 cd /usr
13 sudo chmod o+w lib
14 Copy of openzwave.a et so to /usr/lib
15 cd /etc/apt
16 sudo chmod o+w sources.list
17 using sudo nano sources.list Add `deb http://download.telldus.com/debian/ stable main` to the end of file `/etc/apt/sources.list
18 wget -q http://download.telldus.se/debian/telldus-public.key -O- | sudo apt-key add -
19 sudo apt-get update
20 sudo apt-get install libtelldus-core-dev
21 sudo apt-get install build-essential -y
22 sudo apt autoremove
23 sudo apt-get install cmake libboost-dev libboost-thread-dev libboost-system-dev libsqlite3-dev subversion curl libcurl4-openssl-dev libusb-dev libudev-dev zlib1g-dev libssl-dev
24 cd /home/pi/DEV
25 git clone https://github.com/domoticz/domoticz.git domoticz
cd domoticz
git pull
26 cmake -DCMAKE_BUILD_TYPE=Release CMakeLists.txt
27 make -j 1
28 Copy of pi/DEV/domoticz in pi/domoticz via filezilla
29 Sudo reboot
30 Putty
31 Dmesg and dpkg-l

In the end after a couple of hours, Domoticz BETA version is generated correctly and PI SANDBOX runs the previous domoticz.db.

2 - Description of the Issue
The Beta Domoticz application PI SANDBOX runs the previous domoticz.db almost perfectly except for 2 things
- Darksky cannot access to Darksky Server and get the error :
o darksky error getting http data
- I cannot send email from Domoticz platform and get the following messages
o Error: SMTP Mailer: Error sending Email to:
o Error: Failed to send Email notification
o Error: Notification sent (email) => Failed

3 – Some diagnosis I made
The issue is not linked to the router which is configured the same way for both cases PI PROD and PI SANDBOX.

I compared “dpkg – l” command outcomes between PI PROD and PI SANDBOX and noticed some discrepancies
between PI SANDBOX and PI PROD

in PI SANDBOX the following libraries were installed

libapr1:armhf 1.5.2-5 armhf Apache Portable Runtime Library
libaprutil1:armhf 1.5.4-3
libjsoncpp1:armhf 1.7.4-3
liblzo2-2:armhf 2.08-1.2
libssl-dev:armhf 1.1.0j-1~deb9u1 armhf Secure Sockets Layer toolkit - development files
libssl-doc 1.1.0j-1~deb9u1 all Secure Sockets Layer toolkit - development documentation
libssl1.0.0:armhf 1.0.1t-1+deb8u11 armhf Secure Sockets Layer toolkit - shared libraries
libssl1.0.2:armhf 1.0.2r-1~deb9u1
libcurl4-openssl-dev:armhf 7.52.1-5+deb9u9

In PI PROD I had only
ii libssl1.0.2:armhf 1.0.2l-2+deb9u3

If you managed to build Domoticz executable from the beta source recently and are not facing the issues with the HTTPS access and email notification I would appreciate you revert to me with your experience.
If you are facing the same issue or have faced the same issue when rebuilding Domoticz Beta version, please let me know and share the fixes or workarounds you have made/identified to solve the problem.
Best regards
meal

[dekkerjj]

I'm having darksky issues as well.
Also 'Winddelen' won't work so it looks like a similar issue.

Today I got the tip to use CMAKE slightly different because that would solve https (and notification) issues:

cmake -DCMAKE_BUILD_TYPE=Release CMakelists.txt -DUSE_OPENSSL_STATIC="NO" -DCMAKE_LIBRARY_PATH=open-zwave-read-only

Apparently the tip came from this forum, but I could not find it here..

Trying tonight, I'll let you know!

[meal]

Hello,

I thank you for your reply.

Meanwhile I came across some more information in GitHub more precisely in the link below:

https://github.com/domoticz/domoticz/bl ... travis.yml

As you will see it refers to the cmake command you already mentionned

cmake -DCMAKE_BUILD_TYPE=Release -DUSE_OPENSSL_STATIC="NO" -DCMAKE_LIBRARY_PATH=open-zwave-read-only CMakeLists.txt

But also to the boost libraries to update, some actions to perform with raspi-config to extend the file space.

I have not had the time to try it yet but let me know the outcomes of your investigation. I will do the same on my side.

Best regards

Meal

[dekkerjj]

I've compiled using cmake -DCMAKE_BUILD_TYPE=Release CMakelists.txt -DUSE_OPENSSL_STATIC="NO" -DCMAKE_LIBRARY_PATH=open-zwave-read-only just now and the issue is gone ..

Apparently the -DUSE_OPENSSL_STATIC="NO" statement prevents something in HTTPs breaking


Problem solved

P.S.
Not sure what the -DCMAKE_LIBRARY_PATH=open-zwave-read-only is for, as far as I can tell that directory is read automatically from CMakelists.txt.
But.. Better safe then sorry so I've included it.

[meal]

Hello

I thank you for the update. I will do the same in a couple of days.

By the way which version of Boost Libraries are you working with ?

Would it be possible to share the prequisite actions prior to the generation of Domoticz as i did in my first message.

Many thanks in advance

BR

meal

[dekkerjj]

I've left the Boost libraries as they were (installed rasbian [stretch]-image from https://www.raspberrypi.org/downloads/raspbian/
Pretty sure that's 1.62. AFAIK that's the latest stable version for raspbian

Steps after that:
# With RASPI-CONFIG set password, locale, expand disk etc.
# Update system
sudo apt-get update
sudo apt-get upgrade

For the building itself I've created a script:

Code: Select all

#!/bin/bash
# Script to build domoticz from scratch - JD2019


#Parameters / Variables
DEVDIR="/home/pi/dev"				#Development folder
DESTDIR=$DEVDIR/domoticz			#Subfolder for domoticz
OPENZWAVE=/home/pi/dev/open-zwave-read-only	#Subfolder for openzwave
LOGFILE="/home/pi/dev/Build.log"		#Logfile for this script 
CONF_SWAPFILE=1024

echo $(date)  " - Build-script started" >> $LOGFILE

# Update Packages-list
sudo apt-get update

#Install GIT
sudo apt install git-all >>$LOGFILE

#Install  prereqs domoticz
sudo apt-get install build-essential -y >>$LOGFILE
sudo apt-get install cmake libboost-dev libboost-thread-dev libboost-system-dev libsqlite3-dev subversion curl libcurl4-openssl-dev libusb-dev zlib1g-dev libssl-dev git  >>$LOGFILE
sudo apt-get install python3-dev >>$LOGFILE

# install prereqs Open-Zwave
sudo apt-get install libudev-dev >> $LOGFILE



############# Start compiling openzwave


cd $DEVDIR

# Check if openzwave already pulled else pull from Git
if [ ! -d "$OPENZWAVE" ]; then
	echo $(date)  " - pulling Open-zwave from Github" >> $LOGFILE
	git clone https://github.com/OpenZWave/open-zwave.git $OPENZWAVE >> $LOGFILE
	echo $(date)  " - Open-Zwave pulled from Github" >> $LOGFILE
fi

cd $OPENZWAVE

git pull >> $LOGFILE
echo $(date) "Start compiling Open-Zwave" >> $LOGFILE
make	>> $LOGFILE
echo $(date) "Finished compiling Open-Zwave" >> $LOGFILE
cd ..


############# Done compiling openzwave

############# Start compiling domoticz



# Check if domoticz already pulled else pull from Git
if [ ! -d "$DESTDIR" ]; then
	echo $(date)  " - pulling domoticz from Github" >> $LOGFILE
	cd $DEVDIR
	git clone https://github.com/domoticz/domoticz.git domoticz >> $LOGFILE
	echo $(date)  " - domoticz pulled from Github" >> $LOGFILE
fi

cd $DESTDIR

git pull >> $LOGFILE

echo $(date) "Start building Cfiles" >> $LOGFILE
cmake -DCMAKE_BUILD_TYPE=Release CMakelists.txt -DUSE_OPENSSL_STATIC="NO" -DCMAKE_LIBRARY_PATH=$OPENZWAVE >> $LOGFILE
echo $(date) "Done building Cfiles" >> $LOGFILE

echo $(date) "Start compiling Domoticz" >> $LOGFILE
make	>> $LOGFILE
echo $(date) "Finished compiling Domoticz" >> $LOGFILE

############# Done compiling domoticz


if [ $? = 0 ]
then
        echo $(date)  " - Build-script ran succesfully" >>$LOGFILE
fi

I'm still expanding the script as I want it to be able to 'auto-update' an existing domoticz-server so it has to stop running domoticz-service, move running directory to backup, copy necessary files from build to running directory, copy database and own scripts from backup to running directory and start up again
Also the buildlog should be optional, so there will have to be a check if logfile-variable is used or not.

[meal]

Hello,

I managed to compile domoticz 100% however when I start domoticz cannot login.

In my error log i have the following last messages
/usr/bin/ld: warning: libssl.so.1.0.2, needed by /usr/lib/gcc/arm-linux-gnueabihf/6/../../../arm-linux-gnueabihf/libcurl.so, may conflict with libssl.so.1.1
/usr/bin/ld: warning: libcrypto.so.1.0.2, needed by /usr/lib/gcc/arm-linux-gnueabihf/6/../../../arm-linux-gnueabihf/libcurl.so, may conflict with libcrypto.so.1.1

Would you be kind to tell me which version of libssl.so you are using ?

Best regards
meal

[waaren]

Hello,

I managed to compile domoticz 100% however when I start domoticz cannot login.

In my error log i have the following last messages
/usr/bin/ld: warning: libssl.so.1.0.2, needed by /usr/lib/gcc/arm-linux-gnueabihf/6/../../../arm-linux-gnueabihf/libcurl.so, may conflict with libssl.so.1.1
/usr/bin/ld: warning: libcrypto.so.1.0.2, needed by /usr/lib/gcc/arm-linux-gnueabihf/6/../../../arm-linux-gnueabihf/libcurl.so, may conflict with libcrypto.so.1.1

Would you be kind to tell me which version of libssl.so you are using ?

Best regards
meal

lsof of active domoticz (28725 is PID)

Code: Select all

sudo lsof -P -T -p 28725 | grep "ssl\|curl\|crypto"
lsof: WARNING: can't stat() fuse.gvfsd-fuse file system /run/user/109/gvfs
      Output information may be incomplete.
domoticz 28725 root  mem       REG       0,19  1848612       1437 /usr/lib/arm-linux-gnueabihf/libcrypto.so.1.1
domoticz 28725 root  mem       REG       0,19   332080       1426 /usr/lib/arm-linux-gnueabihf/libssl.so.1.1
domoticz 28725 root  mem       REG       0,19    88072       7382 /usr/lib/python3.5/lib-dynload/_ssl.cpython-35m-arm-linux-gnueabihf.so
domoticz 28725 root  mem       REG       0,19   186260       1835 /usr/lib/arm-linux-gnueabihf/libk5crypto.so.3.1
domoticz 28725 root  mem       REG       0,19   431764       2876 /usr/lib/arm-linux-gnueabihf/libcurl-gnutls.so.4.4.0

ldd of executable

Code: Select all

sudo ldd domoticz | grep "ssl\|curl\|crypto"
libcurl-gnutls.so.4 => /usr/lib/arm-linux-gnueabihf/libcurl-gnutls.so.4 (0x76d8b000)
libk5crypto.so.3 => /usr/lib/arm-linux-gnueabihf/libk5crypto.so.3 (0x767ac000)

[meal]

Hello

The issue is now solved.

I was not able to login because Domoticz when copied from the DEV directory to its target directory via filezilla was not executable.

I updated the access rights and after a sudo reboot Domoticz is now running.

I confirm Darksky and email notifications are now working

Many thanks to the contributors who helped to fix the problem.

BR