Domoticz

dervogt wrote:I have migrated my root filesystem to an external USB drive with 500GB once my setup was up and running. This avoids the issues with symlinks or other small tricks you can do to "save" your SDCard. As the BOOT filesystem practically is read only and once the pi has booted into the OS, the SDCard practically is out of the game.

I have not gone the extra step of switching over to also boot from the USB drive, I personally am happy with the SDCard boot and then having my external HDD takeover the rest.

Egregius wrote: Your harddrive probably consumes more than your pi

Egregius wrote:Booting from NAS is rather pointless, than you could also run domoticz directly on the nas. Running on my nas is not a good option here as it consumes 65W, calculate the cost of that... Besides that I also tried RPi with NFS on nas but that gave a lot more problems.

Logread wrote:I looked into this a while ago, but not long enough (1 year history so far) to prove this makes a real difference... I wrote a wiki page on what I did:
http://domoticz.com/wiki/Moving_Log_and ... _RAM_Drive

Ideally for zwave users, some of the zwave config files could also be moved (e.g. domoticz\Config\zwcfg_0x????????.xml is written rather frequently) but I did not yet venture into this (not sure a symlink would do the trick, but worth trying... otherwise may need to tweak the source code of openzwave or domoticz and this is probably not worth the effort).

I do something similar:

/etc/fstab: You may easily set a larger memory size, memory will only be occupied depending on the file sizes.

Code: Select all

tmpfs   /temp        tmpfs   defaults,noatime,nosuid,mode=777,size=300m      0       0
tmpfs   /var/log        tmpfs   defaults,noatime,nosuid,mode=777,size=300m      0       0

sudo crontab -e

Code: Select all

@reboot /home/pi/atstartup.sh >/dev/null 2>&1
0 * * * * /home/pi/cronhourly.sh >/dev/null 2>&1
1 0 * * * /home/pi/crondaily.sh >/dev/null 2>&1
2 0 * * 1 /home/pi/cronweekly.sh >/dev/null 2>&1

/home/pi/atstartup.sh: make sure some folders are writable and backups are restored to tmpfs drive

Code: Select all

#!/bin/bash
echo "performance" | tee /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor
chmod 777 /var/log
touch /var/log/domoticz.log
chmod 666 /var/log/domoticz.log
cp /home/pi/zwcfg_0xe9238f6e.xml /var/log/zwcfg_0xe9238f6e.xml
cp /home/pi/domoticz.db /var/log/domoticz.db
cp /home/pi/options.xml /home/pi/domoticz/Config/options.xml
ln -sf /var/log/domoticz.db /home/pi/domoticz
chmod 666 /var/log/zwcfg_0xe9238f6e.xml
echo 0 | tee /sys/class/leds/led0/brightness
echo 0 | tee /sys/class/leds/led1/brightness
/opt/vc/bin/tvservice -o
sudo service domoticz.sh start

/home/pi/crondaily.sh: make a daily backup of the tmpfs files

Code: Select all

#!/bin/bash
cp /var/log/zwcfg_0xe9238f6e.xml /home/pi/zwcfg_0xe9238f6e.xml
cp /var/log/domoticz.db /home/pi/domoticz.db

/home/pi/cronweekly.sh: cleanup the logfiles

Code: Select all

#!/bin/bash
sudo rm /var/log/*.gz
sudo rm /var/log/*.1
sudo rm /var/log/SBFSPOT/*
sudo truncate -s 0 /var/log/*
sudo truncate -s 0 /var/log/lighttpd/*

/home/pi/options.xml: keep some zwave settings and set /var/log as log folder

Code: Select all

<?xml version="1.0" encoding="utf-8"?>
<!-- To be effective, this file should be placed in the user data folder specified in the Options::Create method -->
<Options xmlns='http://code.google.com/p/open-zwave/'>
  <Option name="logging" value="true" />
  <Option name="UserPath" value="/var/log/" />
  <Option name="Associate" value="true" />
  <Option name="NotifyTransactions" value="false" />
  <Option name="DriverMaxAttempts" value="5" />
  <Option name="SaveConfiguration" value="true" />
  <Option name="RetryTimeout" value="5000" /><!-- Default 40000 --><!-- OK: 15000 -->
  <Option name="PerformReturnRoutes" value="true" /><!-- added -->
  <Option name="AssumeAwake" value="false" /><!-- added -->
  <Option name="SaveLogLevel" value="3" /><!-- added -->
  <Option name="QueueLogLevel" value="3" /><!-- added -->
  <Option name="DumpTriggerLevel" value="3" /><!-- added -->
  <Option name="NetworkKey" value="0x09, 0x05, 0x03, 0x04, 0x01, 0x06, 0x07, 0x01, 0x09, 0x0A, 0x0B, 0x0C, 0x0D, 0x0E, 0x0F, 0x10" />
  <Option name="RefreshAllUserCodes" value="false" />
  <Option name="ThreadTerminateTimeout" value="5000" /><!-- 5000 -->
</Options>

And finally in /etc/init.d/domoticz.sh: make a backup of the database and zwave config while stopping domoticz.

Code: Select all

do_stop()
{
        # Return
        #   0 if daemon has been stopped
        #   1 if daemon was already stopped
        #   2 if daemon could not be stopped
        #   other if a failure occurred
        start-stop-daemon --stop --quiet --retry=TERM/30/KILL/5 --pidfile $PIDFILE --name $NAME
        RETVAL="$?"
        [ "$RETVAL" = 2 ] && return 2
        # Wait for children to finish too if this is a daemon that forks
        # and if the daemon is only ever run from this initscript.
        # If the above conditions are not satisfied then add some other code
        # that waits for the process to drop all resources that could be
        # needed by services started subsequently.  A last resort is to
        # sleep for some time.
        start-stop-daemon --stop --quiet --oknodo --retry=0/30/KILL/5 --exec $DAEMON
        [ "$?" = 2 ] && return 2
        # Many daemons don't delete their pidfiles when they exit.
        rm -f $PIDFILE
	cp /var/log/domoticz.db /home/pi/domoticz.db
	cp /var/log/zwcfg_0xe9238f6e.xml /home/pi/zwcfg_0xe9238f6e.xml
        return "$RETVAL"
}

I'm pretty sure that with this you avoid most of the writes to the sd card.
There's one big downside of it: if you cut the power you'll lose data of that day. In my case I don't care about that. The temperatures I care about are pushed to a MySQL database on a VPS. As long as you can send the shutdown command there's nothing to worry about.

ben53252642 wrote:

EddyG wrote:24/7 is just normal for a NAS.
The better type of NAS can take care of backups/snapshots etc. So you are 100% save.

In my environment I have a Synology NAS with RAID 6 which can do LUNs, didn't know about the option to boot from a LUN.

Even found a guide on how to do it!
http://www.berryterminal.com/doku.php/s ... sing_iscsi

Combined with watchdog and monit setup, Indeed this would appear to be a very good option particularly with the ability to schedule automatic snapshots.

nigels0 wrote:but your list is missing cronhourly.sh (it's mentioned in the crontab entries)

#!/bin/bash
df -H | grep -vE '^Filesystem|cdrom' | awk '{ print $5 " " $1 }' | while read output;
do
  echo $output
  usep=$(echo $output | awk '{ print $1}' | cut -d'%' -f1  )
  partition=$(echo $output | awk '{ print $2 }' )
  if [ $usep -ge 80 ]; then
    msg="Running out of space \"$partition ($usep%)\" on $(hostname)"
    curl -s --data-urlencode "text=$msg" --data "silent=false" http://127.0.0.1/secure/telegram.php
  fi
	if [ $usep -ge 50 ] && [ $partition = "tmpfs" ]; then
		sudo rm /var/log/*.gz
		sudo rm /var/log/*.1
		sudo truncate -s 0 /var/log/*
		sudo truncate -s 0 /var/log/lighttpd/*
	fi
done
sudo ntpdate be.pool.ntp.org

I just got a new Pi3 and used this set up. I'm testing it right now with a minimum filled Domoticz (main Domoticz is still running on a Pi2 with SD card). So far it looks good. It's fast, stable and you don't have to worry about the loss of data. I'm going to test it another few days and will move the database and the Zwave stick and 433Mhz over to the new setup. Will keep you up to date

Number8 wrote:Using wired Ethernet, Debian Lite has been installed without problem. Afterwards I tried to install using wifi and it was unsuccessful. It keeps trying to download the distribution forever. The wifi dongle is recognized by Berryterminal and connected to Internet. Once installation has been done using wired Ethernet, it is not possible to switch to wifi afterwards. I then discovered that the distribution cannot be upgraded, you have to stick to the one that has been installed. All these things considered I stopped the experiment.

Mine will stay wired no matter what, just curious what you ment by "cannot be upgraded". Are you referring to the basic 'sudo apt-get update' and 'sudo apt-get upgrade' ?

dressie wrote:So it's stable, but eventually the system will become outdated. Time for some pros and cons .

Egregius wrote: ↑Wednesday 05 July 2017 6:29 /home/pi/crondaily.sh: make a daily backup of the tmpfs files

Code: Select all

#!/bin/bash
cp /var/log/zwcfg_0xe9238f6e.xml /home/pi/zwcfg_0xe9238f6e.xml
cp /var/log/domoticz.db /home/pi/domoticz.db

sqlite3 /var/log/domoticz.db ".backup /home/pi/domoticz.db"

Egregius wrote: ↑Sunday 05 November 2017 16:09Moved to a NUC with proxmox now

Egregius wrote: ↑Sunday 05 November 2017 16:09No need to worry about backups anymore.

ben53252642 wrote: ↑Tuesday 04 July 2017 15:58 Hi Folks,

As a long time Domoticz user and someone who has setup others with Domoticz running on Raspberry Pi's I have observed a general issue with this setup:

Eventually the SD Card ALWAYS gets corrupted, it's only a matter of time. My father for instance, he had two SD cards become corrupted, eventually he lost faith in Domoticz and now has a Vera Edge running his Z-Wave system.

I would like to see the development of a 100% read only Raspberry Pi SD card image (write can be enabled for updates and then disabled again) and the option of writing the Domoticz database to an external storage (USB stick for example) or external MariaDB server.

This setup also has the advantage of being immune to sudden power loss should the user decide to power cycle the device by removing the power cable.

I know how to make such a OS image, wanting feedback on the idea, particularly in relation to options with the Domoticz DB.

Start from SDcard.
Goto RPi3 using ssh command as usual and give this command:

echo program_usb_boot_mode=1 | sudo tee -a /boot/config.txt

Reboot from SDcard and after ssh login check if the change is activated:

$ vcgencmd otp_dump | grep 17:
17:3020000a

When 17:3020000a appears then the change is done
Put USB Stick in RPi3 with Jessy/Strech and remove the SDcard and boot from USB Stick

lrybak wrote: ↑Sunday 05 November 2017 17:33

Egregius wrote: ↑Sunday 05 November 2017 16:09Moved to a NUC with proxmox now

How're you deal with gpio/i2c and stuff like that?

Egregius wrote: ↑Sunday 05 November 2017 16:09No need to worry about backups anymore.

Don't be surprised, HDD/SSD can crash as well