Sending data between Domoticz servers

[ILoveIOT]

Hi,

Is there a clean way to send data between different versions off Domoticz, I'am not sure if I can use "more options > send data > http" options, for example to get a temp value from a Domoticz server to the other Domoticz server. Or maybe there are better ways, for master/slave server they must have the same version.

I have a Domoticz server version (V4.9700) with wheater underground (key not working on new version, get a key error), and i wanna display some value's into the new Domoticz (V4.10717) server, also got a broadcom (s3) energy socket, and that is not working with the new version

--------

Output broadcom (s3) energy plug on version V4.10717, on version V4.9700 its working fine

2020-03-18 10:22:52.823 Status: (EnergyPlug) Started.
2020-03-18 10:24:02.946 Error: EnergyPlug hardware (12) thread seems to have ended unexpectedly

[waaren]

I have a Domoticz server version (V4.9700) and i wanna display some value's into the new Domoticz (V4.10717) server

My approach here would be to use a dzVents script on the 'new' domoticz to poll the devices on the 'old' one and update the shadow devices.
Happy to help creating such a script solution. Let me know if that would work for you.

[Toulon7559]

Has the disadvantage that the Destination is dependent on services of the Sender, but data-sync may also be realised by 'push-instruction' like this (simple) one in Python-script for upload of a power&energy-value to Virtual Device in Destination (= Domoticz2)

Code: Select all

Upload to Domoticz2
#domoticz2 settings
domoticz2_host      	= '192.168.1.5'
domoticz2_port      	= '8080'
domoticz2_url       	= 'json.htm'
domoticz2_idx           = '420'          #idx of device in Domoticz2

#uploading values to domoticz2
url = ("http://" + domoticz2_host + ":" + domoticz2_port + "/" + domoticz2_url + "?type=command&param=udevice&idx=" + domoticz2_idx + "&nvalue=0&svalue=" + str(ActualPower) + ";" + str(etotalstr))
urllib.urlopen(url)

Bear in mind that you need to import into the Python-script dependencies like urllib, json, etc.

[ILoveIOT]

Hi thank you for the answers, got the info now, I will go for the script approch, guess I will make it standalone outside Domoticz, with cron, I guess in bash, so I have all the freedom, get value with curl, and send it with curl.

I hoped on the nice http post/get solution, but not today

Thanks !!, best regards

[manjh]

consider the MQTT approach. You'll need to set it up on both sides: MQTT and Node-Red. Then add MQTT as HW in Domoticz.
Once this is done, it is relatively easy to create flows in Node-Red to bring info from one place to the next.

Agreed, there is a one-time learning curve for Node-Red. MQTT only needs to be installed, not configured.
But the upside: once you've set it up, learned how to use Node-Red, and have the transfer mechanism running, you'll be amazed about the possibilities!

[ILoveIOT]

Also a good idea yes,.....thank you,..not as easy as the script setup, but not a bad idea to be MQTT ready

[FireWizard]

HI,

@manjh wrote:

consider the MQTT approach. You'll need to set it up on both sides: MQTT and Node-Red. Then add MQTT as HW in Domoticz.
Once this is done, it is relatively easy to create flows in Node-Red to bring info from one place to the next.

Keep in mind that is is not dead simple to set up this MQTT approach.
Recently we had a discussion in this thread: https://www.domoticz.com/forum/viewtopi ... ve#p239376

As a result of this I tested a setup with MQTT,
To be able to communicate between, lets say Domoticz1 and Domoticz2, you have to setup 2 MQTT servers. Domoticz1 communicates with the first MQTT server and Domoticz2 communicates with the second MQTT server. And Node Red communicates with both and is in between.

Reason for that is that the topics in Domoticz are "hard coded".
This means that Domoticz1 and Domoticz2 both publish their sensors, switches, etc, to domoticz/out and subscribe both to domoticz/in.
As both will very likely use the same IDX's, this will interfere with each other and causes conflicts.

But if you set-up 2 MQTT servers it will work.

On the other hand @waaren offers support to develop a dzVents script, who can do the job as well.
Perhaps easier.

Regards

[Toulon7559]

Waaren's suggestion has the advantage that the Domoticz server acting as source does not need any modification:
the other/destination Domoticz-server(s) ask the required information from the source.

The Python-script in this thread for HP3001 is more or less an example of the approach which waaren suggests:
1) make json-call to the device(s) of the remote (= source) Domoticz-server
2) extract the info from the response
3) upload the info to the virtual device(s) of the local Domoticz server

As stated, the Python-script in that thread is simple & with as many repeats as devices:
a dzvents' script from waaren's hand most probably is much leaner, shorter & quicker.

[ronaldbro]

I created a simple dzVents script to push sensor data to another domoticz. See https://www.domoticz.com/forum/viewtopi ... 72&t=31618

[manjh]

I created a simple dzVents script to push sensor data to another domoticz. See https://www.domoticz.com/forum/viewtopi ... 72&t=31618

Creative solution, but if master and slave are in separate locations it means I would have to port-forward 8080 on the master subnet router. Thhis opens up the master Domoticz to all updates that a hacker could insert.
With MQTT the updates are filtered on both sides. This means a really smart hacker could put his hands on the limited number of devices that I decided to synch across. He would have to break into my MQTT queue...

Nevertheless, this solution is excellent when used within the same subnet!

[ronaldbro]

I understand multi site makes it more complex. But I would always set up a VPN bridge between the two machines, also for the MQTT solution. But that's me...

[ILoveIOT]

Hehe, thanks guys for the tips, was litlle confused, what to do

Created a simple bash script, so i got all the freedom of the world, funny thing is, I was creating this for wheater undergrond, but it stopped sinds 01-04-2020 Now its mainly used for the broadcom energy plug.

Anyway,..maybe another need it to.

EDIT : wheater undergrond stopped working on the 4.9700, but on the new 2020.1 version with new longer key, its working again

Filename : /root/scripts/domoticz.sync.device.servers.sh

Code: Select all

#!/bin/bash
# 
INFO="Sync between (IDX) Domoticz servers (different versions) by ILoveIOT @ 2020"
#
#
# use /root/scripts/domoticz.sync.device.servers.sh >> /dev/null 2>&1 in a cron job, every 1-5 min
# use /root/scripts/domoticz.sync.device.servers.sh on console so you see some output
#
# It can sync temperature + humidity, or just only temp, or some value like a watt meter, create a dummy device on target server voor sync
#
#
#
#

# Setting the names or ip adresses of the Domoticz servers
SOURCEDOMOTICZIP="192.168.8.189"
TARGETDOMOTICZIP="192.168.8.1"

# Setting the port of the Domoticz servers
SOURCEDOMOTICZPORT="8080"
TARGETDOMOTICZPORT="8080"

# Domoticz IDX(s) to check, use : for separate, use source IDX and target IDX like 1-2 or 200-300 for example 1-2:200-300:67-234
# source idx is 1 and target idx is 2, source idx is 200 and target idx is 300, source idx is 67 and target idx is 234
#IDXTOCHECK=`echo -e "1-2:200-300:67-234"`
IDXTOCHECK=`echo -e "237-259:41-41:44-44:136-136:196-196:244-244:197-197:40-40:189-189:35-35:88-88:87-87:199-199:46-46:36-36:42-42"`



# Begin Script
#####################################################################################################################################
#####################################################################################################################################

clear
echo "$INFO" 
echo ""
echo "SOURCEDOMOTICZIP     = $SOURCEDOMOTICZIP with port $SOURCEDOMOTICZPORT"
echo "TARGETDOMOTICZIP     = $TARGETDOMOTICZIP with port $TARGETDOMOTICZPORT"
echo ""

# Do the loop for "IDXTOCHECK"
c=1
temps=`echo "$IDXTOCHECK" | /usr/bin/cut -f $c -d ":"`
while [ -n "$temps" ]; do

	IDXSOURCE=`echo "$temps" | /usr/bin/cut -d '-' -f1`
	IDXTARGET=`echo "$temps" | /usr/bin/cut -d '-' -f2 | /usr/bin/cut -d ':' -f1`

	# Get the domoticz value from a device, en cut the result
	VALUEIDXSOURCEDEVICE1=`curl -s 'http://'$SOURCEDOMOTICZIP':'$SOURCEDOMOTICZPORT'/json.htm?type=devices&rid='$IDXSOURCE'' | /bin/grep Data | /usr/bin/awk '{ print $3 }' | /bin/sed 's/,/ /g' | /usr/bin/tr -d '"'`
	VALUEIDXSOURCEDEVICE2=`curl -s 'http://'$SOURCEDOMOTICZIP':'$SOURCEDOMOTICZPORT'/json.htm?type=devices&rid='$IDXSOURCE'' | /bin/grep Data | /usr/bin/awk '{ print $5 }' | /bin/sed 's/,/ /g' | /usr/bin/tr -d '"'`

	# Send the value to the target domoticz server
	curl -i -s "http://$TARGETDOMOTICZIP:$TARGETDOMOTICZPORT/json.htm?type=command&param=udevice&idx=$IDXTARGET&nvalue=0&svalue=$VALUEIDXSOURCEDEVICE1;$VALUEIDXSOURCEDEVICE2;0" >> /dev/null 2>&1

	# Just for debugging, you can comment this out if it works, or add >> /dev/null 2>&1
	echo ""
	echo "VALUEIDXSOURCEDEVICE1 = $VALUEIDXSOURCEDEVICE1"
	echo "VALUEIDXSOURCEDEVICE2 = $VALUEIDXSOURCEDEVICE2"
	echo "IDXSOURCE             = $IDXSOURCE"
	echo "IDXTARGET             = $IDXTARGET"
	echo "IDXTOCHECK            = $temps"
	echo ""

# End the loop IDXTOCHECK
let c=$c+1
temps=`echo "$IDXTOCHECK" | cut -f $c -d ":"`
done

exit

#EOF

[sailmich]

I have a domoticz remote server with EnOcean running. Problem is that a switch on this server is going on by clicking at first and after 10sec off automatically. On my master it stays on I tried the bin/bash with small modifications but this doesn't work with switches. I just get the value from remote server but can't get it to my master. I believe I should compare booth values and if not the same It should change on the master.
I appreciate any help!

[ILoveIOT]

Not sure what you want ? you wanna sync a switch ? this script is used for value's

http://$TARGETDOMOTICZIP:$TARGETDOMOTICZPORT/json.htm?type=command&param=udevice&idx=$IDXTARGET&nvalue=0&svalue=$VALUEIDXSOURCEDEVICE1;$VALUEIDXSOURCEDEVICE2;0

I guess you need

http://$TARGETDOMOTICZIP:$TARGETDOMOTICZPORT/json.htm?type=command&param=switchlight&idx=$IDXTARGET&switchcmd=$VALUEIDXSOURCEDEVICE1

Not sure if this gonna work, but you can see the difference

[sailmich]

Yes sync a switch!

Code: Select all

  # Get the domoticz value from a device, en cut the result
        VALUEIDXSOURCEDEVICE1=`curl -s 'http://'$SOURCEDOMOTICZIP':'$SOURCEDOMOTICZPORT'/json.htm?type=devices&rid='$IDXSOURCE'' | /bin/grep Data | /usr/bin/awk '{ print $3 }' | /bin/sed 's/,/ /g' | /usr/bin/tr -d '"'`

        # Send the value to the target domoticz server
        VALUEIDXTARGETDEVICE= curl -i -s 'http://'$TARGETDOMOTICZIP':'$TARGETDOMOTICZPORT'/json.htm?type=command&param=switchlight&idx='$IDXTARGET'&switchcmd='$VALUEIDXSOURCEDEVICE1'' >> /dev/null 2>&1

        # Just for debugging, you can comment this out if it works, or add >> /dev/null 2>&1
        echo ""
        echo "VALUEIDXSOURCEDEVICE1 = $VALUEIDXSOURCEDEVICE1"
        echo "VALUEIDXTARGETDEVICE = $VALUEIDXTARGETDEVICE"
        echo "IDXSOURCE             = $IDXSOURCE"
        echo "IDXTARGET             = $IDXTARGET"
        echo "IDXTOCHECK            = $temps"
        echo ""

Debug deliver

Code: Select all

VALUEIDXSOURCEDEVICE1 = Off 
VALUEIDXTARGETDEVICE = 
IDXSOURCE             = 8
IDXTARGET             = 249
IDXTOCHECK            = 8-249

[ILoveIOT]

youre missing at least the `,....and first you need to sync the value to the target server, now you getting it from source, and check value from the target, without putting/syncing it, just play with the `,'," from the script I posted.

# Send the value to the target domoticz server
VALUEIDXTARGETDEVICE= curl -i -s 'http://'$TARGETDOMOTICZIP':'$TARGETDOMOTICZPORT'/json.htm?type=command&param=switchlight&idx='$IDXTARGET'&switchcmd='$VALUEIDXSOURCEDEVICE1'' >> /dev/null 2>&1

change to

# Send the value to the target domoticz server
curl -i -s "http://$TARGETDOMOTICZIP:$TARGETDOMOTICZPORT/json.htm?type=command&param=switchlight&idx='$IDXTARGET'&switchcmd='$VALUEIDXSOURCEDEVICE1'" >> /dev/null 2>&1

# Get the value from the target domoticz server
VALUEIDXTARGETDEVICE=`curl -s 'http://'$TARGETDOMOTICZIP':'$TARGETDOMOTICZPORT'/json.htm?type=devices&rid='$IDXTARGET'' | /bin/grep Data | /usr/bin/awk '{ print $3 }' | /bin/sed 's/,/ /g' | /usr/bin/tr -d '"'`

[sailmich]

Thank you very much! I will try it this evening.

[ILoveIOT]

Yw,..Post you're working script like me for the others, so we have a value and switch sync script

[sailmich]

Script is up and running, but now I got an off signal at the switch every minute. Should have thing about that before Is there a way just sent off if the switch at remote is off and it was on before?

[ILoveIOT]

Hehe true,...at least its syncing

Just check the value from source, then from target, if it is different, resend it to target.

You know how to do that ?

if [ "$a" != "$b" ] ;then
do something
fi