Best practices for Domoticz logfile?

[ThinkPad]

My Domoticz is connected to internet, so i can access it with my smartphone ('Dromotica' Android app). To make the system more secure i am setting up fail2ban, so intruders will get banned automatically.
For fail2ban to work, it needs access to the Domoticz log. By default, the logging of Domoticz seems to be disabled. I enabled it (editing DAEMON_ARGS in /etc/init.d/domoticz.sh), but i noticed it grows quite fast. After 4 hours it was already 1.5MB.

I read something about 'logrotate', which flushes the configured logfiles when the size exceeds <x> MB, or on a daily/weekly/monthly base. But i'm not sure how to set this up, would like to hear from others how they cope with this.

I am also a bit worried about the constant writing to the log. My Domoticz runs in a (Ubuntu) virtual machine on my ESXi server, the virtual machines reside on a SSD, which can wear out.
Maybe it is better to write the log to /tmp/ (that's RAM, right?)

How do you have such things configured?

Btw, i am still working on the fail2ban wiki-page, if you spot any additions/mistakes, please correct them on the wiki/let me know.

[Egregius]

I have this in /etc/fstab to have all logfiles in memory.

Code: Select all

tmpfs   /var/log    tmpfs    defaults,noatime,nosuid,mode=0775,size=100m    0 0

OK, tmpfs get's cleared after reboot, so my fail2ban log is stored in /var/logs. That way the fail2ban log is kept so recidive can be activated.

To preserve memory usage of the tmpfs I truncate the logfiles with a bash script, activated by cron at midnight:

Code: Select all

#!/bin/sh
truncate -s 0 /var/log/apache-access.log
truncate -s 0 /var/log/apache-error.log
truncate -s 0 /var/log/domoticz.log

[ThinkPad]

/var/log/ already exists on my system. So i think i need to create a different directory? Something like /var/tmp/ like they mention here: http://domoticz.com/wiki/Setting_up_a_R ... spberry_Pi
Basically, i can just follow that tutorial i think? (But i will create a bigger RAM-drive, the 1MB they mention there is a bit small hehe).

Edit: Hmm, reading this: http://askubuntu.com/a/99520 you just mount the /var/log folder in a different location, not creating it again or so?

P.S. What is the benefit of 'truncate' over deleting the logfile? You set it to truncate to zero, so it just empties the file? Why not just delete the file? (not argueing your choice, just interested).

[stlaha2007]

@ThinkPad: All depends on how "save" you wanna be...
I'm using standard Raspian Image as bottomlayer for Domoticz.

Tweaked it by disabling unused services etcetera.
One thing indeed to look after is logging.
Yes, by default extensive logging { /var/log} with Raspian, And Domoticz when enabled into eg. /var/log will hurt sdcard really bad.
What i have done is remounted /var/log onto nfs-share on my NAS by editing /etc/fstab.
Also domoticz buildin backup feature is using another nfs-share on that NAS.

Other options are log into tmpfs (RAM Filesystems) like /run/shm/.
Using this for downloaded images from ipcams, image-manipulation, OCR into textfiles. Which are dumped into /dev/null when the PI reboots.

Not currently behind a full console now, but when interrested, i can give some details in this post or PM later.

grtz Stephan

Sent from my K00C using Tapatalk

[ThinkPad]

I don't need safety, i didn't miss the Domoticz log when it wasn't enabled yet. It's just that i don't want to pump my disk full with a quickly-growing logfile

I will have a look on some Ubuntu (as that is what my Domoticz VM is running) tutorials to setup a ramdrive. And then just point the Domotiz log to there. But if i limit my ramdrive to, let's say 100MB, and almost never reboot (why do that when it runs fine?), the ramdrive will be full at some moment.... so i still need a cronjob (running every 5 mins or so) that does 'something' (truncating/deleting) with the logfile i guess...

Saving to my NAS is something i don't want, i just spent a few days in getting my NAS to let the HDD's go into spindown nicely. There was some basic stuff keeping it awake. Saves about 10W of usage that was being used continously before.

[Egregius]

truncate has the advantage of keeping file owner and permissions.
I think domoticz didn't create a new one after a delete and therefore I searched another solution.

[ThinkPad]

Thanks, it is becoming clear now. Will try tonight if i can get the logfile in a RAM-drive and let cron run a truncate command every hour or so (i have quite much sensors, so log fills up quickly).

[stlaha2007]

@Thinkpad: I though Ubuntu has some ramdrives / tmpfs already setup. Take a peek in /etc/fstab

@Egregius: Domoticz will create the logfile after restarting service, Believe it created it also without restarting, can't reember, did rename late last saterday. Because of the huge logfile.

Sent from my K00C using Tapatalk

[Egregius]

I didn't want to restart the service for that, so I truncate. Works already months like that

[gizmocuz]

you can use the standard debian system to rotate the log file which will send a SIGHUP signal to domoticz

http://unix.stackexchange.com/questions ... n-on-linux

or copy the file, and issue a SIGHUP

[ThinkPad]

@Thinkpad: I though Ubuntu has some ramdrives / tmpfs already setup. Take a peek in /etc/fstab
[...]

Nope:

Code: Select all

domoticz@domoticz-vm:~$ cat /etc/fstab | grep 'tmpfs'
domoticz@domoticz-vm:~$

I will create one.

The logrotate is a bit unclear for me, i don't understand how to set it up. It also seems to just 'split up' the logfile in a new part when the 'old' one becomes full. I just want to delete older data, not saving that also and beginning a new file.

For now i will stick with a logfile in ramdrive and truncating it through a script running in cron.

[stlaha2007]

you can use the standard debian system to rotate the log file which will send a SIGHUP signal to domoticz

http://unix.stackexchange.com/questions ... n-on-linux

or copy the file, and issue a SIGHUP

Yep. Needed to dive into docs what it actually does. Sends a restart to the daemon. And in addition recreates / starts over with a new logfile.

Thnx Gizmocuz

Sent from my D6503 using Tapatalk

[stlaha2007]

@Thinkpad: I though Ubuntu has some ramdrives / tmpfs already setup. Take a peek in /etc/fstab
[...]

Nope:

Code: Select all

domoticz@domoticz-vm:~$ cat /etc/fstab | grep 'tmpfs'
domoticz@domoticz-vm:~$

I will create one.

The logrotate is a bit unclear for me, i don't understand how to set it up. It also seems to just 'split up' the logfile in a new part when the 'old' one becomes full. I just want to delete older data, not saving that also and beginning a new file.

For now i will stick with a logfile in ramdrive and truncating it through a script running in cron.

Al depends on what you want and setup. Basics of Logrotate is copy (and depending on config) zip it. Send a SIGHUP to process to 'restart' thus create new logfile.

As root issue a crontab -l and see some calls for logrotate, and look into preconfigured ones in /etc/logrotate/ or /etc/logrotate.d/.

Also look in /var/log i believe you will find some files like syslog syslog.1 syslog.2 etc.
Using mainly SuSE and less Debian kinds so some paths can be different.

Copying lots of prebuild/preconfigured ones to accomplice the tasks needed.

Grtz
Stephan

Sent from my D6503 using Tapatalk

[ceedebee]

I am using logrotate on Domoticz because of the use of fail2ban which needs this log files.
In my setup the log is rotated and a new file is created but this file is empty and not filled by Domoticz.
How does a working logrotate configuration file for Domoticz look like?

[stlaha2007]

I am using logrotate on Domoticz because of the use of fail2ban which needs this log files.
In my setup the log is rotated and a new file is created but this file is empty and not filled by Domoticz.
How does a working logrotate configuration file for Domoticz look like?

Sounds like permission.
Check which user runs domoticz. Then check which user/group created the logfile.

For me this is how domoticz and the logfile are configured...
ps -ef | grep 'domoticz' gives you something like:
root 23696 1 1 Dec18 ? 01:36:19 /home/pi/domoticz/domoticz -daemon -www 8080 -sslwww 443 -log /var/log/domoticz.txt

In my case it's root running the daemon.

With the following (see -log)
ls -l /var/log/domoticz.txt gives something like:
-rw-r----- 1 root root 60982591 Dec 22 2015 /var/log/domoticz.txt

There are user root and group root.


Looking into a logrote conf-file like apache2 there is a keyword for user and group. Change these to the user/group as which domoticz is running.

Perhaps you're running as user pi and group users. So the file needs to be created as pi/users. And by default /var/log is only read/write to root/root.

Will look into a good logrotate for domoticz as im only using monit to check and manually rotate log myself.

Sent from my D6503 using Tapatalk

[ThinkPad]

[...]
Will look into a good logrotate for domoticz as im only using monit to check and manually rotate log myself.

That would be great, i don't understand how to set this up, if you could provide a small howto that would be very helpful.

Oh and go to your Tapatalk settings and disable this, as it is unneeded :

Sent from my D6503 using Tapatalk

[ceedebee]

@stlaha2007: my domoticz is running as root and so is the logfile (domoticz.log).
There is also an domoticz.log.1 file and I just found out that domoticz is logging into this file. So this morning (logrotate is configured to run daily) a new file domoticz.log is created but domoticz is logging in the .1 file.

[Egregius]

I think you need to sent a sighup signal after the logratote. Don't know, don't use logrotate. All my logs are on tmpfs so I use reboot to clear them.

[stlaha2007]

Just created an logrotate for domoticz....
Run it 2 times with: {edit typo}

logrotate -v /etc/logrotate.d/domoticz
logrotate -fv /etc/logrotate.d/domoticz

First one told me it has not run before, so second run to "Force" a newly created config.
The following config just copies it en truncates the original (" copytruncate ") and restarts domoticz afterwards.
the copytruncate can be replaced with "compress" which then needs the companions create "mode" "user" "group", as in "create 644 pi users"

Here's the logrotate for domoticz created:

Code: Select all

filepath and filename:
/etc/logrotate.d/domoticz

file contents:
/var/log/domoticz.txt {
        rotate 7
        daily
        minsize 1M
        missingok
        notifempty
        copytruncate
        delaycompress
        postrotate
                invoke-rc.d domoticz.sh restart > /dev/null
        endscript
}

[kaaswe]

This is my working logrotate. /var/log is mounted as tmpfs in fstab.

Code: Select all

# vi /etc/logrotate.d/domoticz
/var/log/domoticz.log {
    # size 5M
    daily
    rotate 1
    compress
    dateext
    # maxage 1
    postrotate
      /usr/bin/killall -HUP /home/pi/domoticz/domoticz
    endscript
    su root root
}