High CPU after enabling DZVents on build 14098

[BarryT]

Hi,

as the subject says, my cpu usage is going crazy after enabling dzvents.
normal usage is 4/5%, but after enabling dzvents (for 1 script) the cpu usage goes 20+!
is this normal behaviour?

Using dzvents only for a monitoring of domoticz:

Code: Select all

return 
{
    on = 
    {
        system = 
        {
            'start',
        },
    },

    logging = 
    {
        level = domoticz.LOG_DEBUG,
        marker = 'startup tasks' 
    },

    execute = function(dz)
        dz.notify("Domotica is gestart of opnieuw gestart na een crash!") 
    end
}1

Domoticz 2021.1 (build 14098)

[waltervl]

Just updated my test system to latest beta and see no increase of CPU. My test system is running various DzVents scripts. To be sure I also added yours, still no increase

[Zatura24]

Hi,

First of all, I know this post is old and I'm pretty new to Domoticz stuff so excuse me if I get things wrong.

I am experiencing the same behavior in version 2022.1 running on a RPI 4. Simply adding any dummy dzVents script an my CPU usage jumps from ~5% to ~20-30%. After reading some forum posts I got the feeling that dzVents got a lot of events and every events it checks the script. So I disabled the most intensive hardware, in this case a P1 smart meter that gets data every second, and the usage dropped even lower than 5%.

So I guess this is expected behavior then since every second an event gets created.

I would love to know if there is an alternative to still be able to run a script based on a device value, i.e. with python or plain lua.

[waltervl]

Is suppose you had an incorrect trigger in your dzvents script. Check the dzvents documentation on the triggers. https://www.domoticz.com/wiki/DzVents:_ ... .80.A6_.7D

You should have your script triggered on change of a device (not all) or a specific time (every 5 minutes, on 15:10 hrs etc) or other event that happens.

[Zatura24]

This doesn't fix the problem sadly. If I include a script like the following it still jumps to that higher usage. OP posted a script with a on system event.

Code: Select all

return {
	on = {
		timer = {
			'every minute'
		}
	},
	logging = {
		level = domoticz.LOG_INFO,
		marker = 'template',
	},
	execute = function(domoticz, timer)
		domoticz.log('Timer event was triggered by ' .. timer.trigger, domoticz.LOG_INFO)
	end
}

The documentation states the following:

If any of the events or triggers match with the current event coming from Domoticz, then the execute part of the script is executed by dzVents.

. So even though there is no match, it still builds a table with all information as states in other documentation

And on top of that, script performance has increased a lot if you have many scripts because Domoticz will fetch all device information only once for all your device scripts and timer scripts.

https://www.domoticz.com/wiki/Events

So maybe I'm still missing something an can I just exclude a device from generating events

[waltervl]

I do not have a P1 meter so do not know what happens in your meter. Indeed updating a lot of devices every second could create some extra load. What is the refresh setting of your P1 meter? Default 0 or did you set it to 1?

[Zatura24]

I didn't see that option, good catch. The P1 meter is set to 0. I will try increasing it later and see if anything changes.

[Zatura24]

So I increased the timeout delay and as expected the spikes to 20-30% happened less frequently, but still to the same amount. So I've rewritten the dzVents script to a plain Lua script and disabled dzVents all together and now it spikes to a max 12%. So as I see it dzVents is only really useful for a lot of scripts otherwise the overhead is to much (when there are events every second or more)

[waltervl]

I think something is wrong with your setup as DzVents normally would not will not give that much of load.
I have 200+ devices and never have a spike of that much CPU due to Domoticz/DzVents.

Perhaps try to update your install with ./update (for stable installation) or ./updatebeta to be sure everything is OK. Do not revert back from Beta to Stable as there are database changes so your Beta database is not compatible with the stable executable.

[Zatura24]

I've completely reinstalled Domoticz but to no avail. And I don't really feel like upgrading to the beta build. So I'll just leave it how I have it now with plain lua scripts. Thanks for all the help

[elzorrovega]

Over the two years I have been using Domoticz there been a thing or two I learned about the application and the RaspberryPi (RPI). Firstly, there are limits as to what you can do with the combination. RPI has certainly an excellent price/computing_power ratio but like anything else there are limits.

Our LAN has two RPI:

Raspberry Pi 3 Model B Plus Rev 1.3, Domoticz Server
Linux version 5.15.32-v8+ (dom@buildbot) (aarch64-linux-gnu-gcc-8 (Ubuntu/Linaro 8.4.0-3ubuntu1) 8.4.0, GNU ld (GNU Binutils for Ubuntu) 2.34) #1538 SMP PREEMPT Thu Mar 31 19:40:39 BST 2022

Domoticz
Version: 2022.1
Build Hash: c9526851b
Compile Date: 2022-01-31 09:34:32
dzVents Version: 3.1.8
Python Version: 3.9.2 (default, Feb 28 2021, 17:03:44) [GCC 10.2.1 20210110]

Raspberry Pi 4 Model B Rev 1.4 , “Sandbox”
Linux version 5.15.56-v8+ (dom@buildbot) (aarch64-linux-gnu-gcc-8 (Ubuntu/Linaro 8.4.0-3ubuntu1) 8.4.0, GNU ld (GNU Binutils for Ubuntu) 2.34) #1575 SMP PREEMPT Fri Jul 22 20:31:26 BST 2022

I second Waltervi’s point of view that DzVents by itself should not increase CPU Load under normal usage. RPI 3b+ only runs Domoticz server applications. Both Chromium and Firefox Web Browser are resource “hogs” and I strongly advise not to use them in same machine as Domoticz.

RPI 4B is my sandbox where I use Chromium as client to Domoticz; Netflix, Spotify, updates, etc. Your telephone or Pad browser can also do the trick to talk to your Domoticz.

My Domoticz has 30 used devices and six DzVents scripts of which three are device triggered and three are time triggered. The Alarm script is triggered by devices and verifies the state of the security variable to trigger the alarm , thus, it’s executed quite often even when someone is home.

Attached Bash script is run to get a system snapshot. I also attach the output.
Domoticz is using 12.8 % of RAM and ~ 1.0 % load.

If you run this shell script what kind of load does your RPI have?

pstruc.sh - Do not forget to run sudo chmod 777 to make it executable

Code: Select all

#!/bin/bash
#This script will sort process structure with descending order using MEM usage as key (fourth column)
#If no parameter is given then to standard output otherwise to file name. 
if [ "$#" != "1" ]; then
#	top | mawk '{if (NR < 6) print $0} END{printf("\r")}' 
	top -b -n 1 | head -n 5 
	ps aux --sort -%mem
#	ps -auxf | sort -r -k 4
else
	date > "$1"
	top -b -n 1 | head -n 5 >> "$1"
	ps aux --sort -%mem >> "$1"
#	ps -auxf | sort -r -k 4 >> "$1"
fi
exit

RPI Process Structure

Spoiler: show

Wed 17 Aug 17:17:51 CEST 2022
top - 17:17:51 up 25 days, 23:05, 3 users, load average: 0.01, 0.04, 0.00
Tasks: 180 total, 1 running, 179 sleeping, 0 stopped, 0 zombie
%Cpu(s): 4.3 us, 5.3 sy, 0.0 ni, 90.4 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
MiB Mem : 885.8 total, 265.1 free, 481.6 used, 139.2 buff/cache
MiB Swap: 0.0 total, 0.0 free, 0.0 used. 326.0 avail Mem
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
root 564 1.0 12.8 2241380 116460 ? Ssl Jul22 381:19 /home/pi/domoticz/domoticz -daemon -www 8080 -sslwww 443
root 1479 0.0 7.3 363884 66636 ? Ssl Jul22 7:30 /usr/libexec/packagekitd
pi 1078 0.0 7.1 1548916 65200 ? Sl Jul22 2:11 pcmanfm --desktop --profile LXDE-pi
root 539 0.1 6.8 1211560 62440 tty7 Ssl+ Jul22 43:19 /usr/lib/xorg/Xorg :0 -seat seat0 -auth /var/run/lightdm/root/:0 -nolisten tcp vt7 -novtswitch
pi 1077 0.2 5.6 2199104 51516 ? Sl Jul22 106:11 lxpanel --profile LXDE-pi
root 528 0.0 2.9 264812 26380 ? Sl Jul22 13:36 /usr/bin/vncserver-x11-core -service
pi 1098 0.0 2.8 58736 26196 ? S Jul22 0:01 /usr/bin/python3 /usr/share/system-config-printer/applet.py
root 136 0.0 2.7 81948 25372 ? Ss Jul22 0:43 /lib/systemd/systemd-journald
pi 545195 0.0 2.3 380836 21656 ? Sl Jul31 0:01 lxterminal
pi 1073 0.0 2.3 183768 21344 ? Sl Jul22 0:00 lxpolkit
pi 1094 0.0 1.8 329112 17124 ? Sl Jul22 10:15 /usr/lib/aarch64-linux-gnu/libexec/kdeconnectd
root 574 0.0 1.5 18812 13684 ? S Jul22 0:41 /usr/bin/vncagent service 0
pi 702 0.0 1.2 534532 10992 ? S<sl Jul22 0:00 /usr/bin/pulseaudio --daemonize=no --log-target=journal
pi 1210 0.0 1.1 33140 10684 ? S Jul22 0:06 /usr/bin/vncserverui -statusicon 0
pi 1122 0.0 1.1 33860 10492 ? S Jul22 0:19 /usr/bin/vncserverui service 0
root 1495265 0.0 1.1 174108 9992 ? Ssl 00:00 0:00 /usr/sbin/cups-browsed
pi 1071 0.0 1.0 66464 9624 ? S Jul22 0:03 openbox --config-file /home/pi/.config/openbox/lxde-pi-rc.xml
colord 573883 0.0 0.9 240948 8892 ? Ssl Aug01 0:00 /usr/libexec/colord
root 1538437 0.0 0.8 16076 7916 ? Ss 17:08 0:00 sshd: pi [priv]
root 1 0.0 0.8 167948 7860 ? Ss Jul22 0:57 /sbin/init splash
pi 1337 0.0 0.8 495176 7796 ? Ssl Jul22 0:00 /usr/libexec/gvfs-udisks2-volume-monitor
root 399 0.0 0.8 393472 7580 ? Ssl Jul22 0:06 /usr/libexec/udisks2/udisksd
root 1495264 0.0 0.8 26892 7540 ? Ss 00:00 0:00 /usr/sbin/cupsd -l
root 381 0.0 0.7 235892 7252 ? Ssl Jul22 0:01 /usr/libexec/polkitd --no-debug
pi 710 0.0 0.7 248196 7180 ? Ssl Jul22 0:01 /usr/bin/lxsession -s LXDE-pi -e LXDE -w openbox-lxde-pi
systemd+ 1536652 0.0 0.6 88104 5992 ? Ssl 16:24 0:00 /lib/systemd/systemd-timesyncd
root 525 0.0 0.6 244268 5976 ? Ssl Jul22 0:00 /usr/sbin/ModemManager
pi 673 0.0 0.6 16308 5708 ? Ss Jul22 0:00 /lib/systemd/systemd --user
pi 1369 0.0 0.6 317680 5708 ? Sl Jul22 0:00 /usr/libexec/gvfsd-trash --spawner :1.7 /org/gtk/gvfs/exec_spaw/0
pi 1055 0.0 0.6 235176 5512 ? Ssl Jul22 0:00 /usr/libexec/gvfsd
root 170 0.0 0.5 22044 5400 ? Ss Jul22 0:11 /lib/systemd/systemd-udevd
root 668 0.0 0.5 163672 5392 ? Sl Jul22 0:00 lightdm --session-child 14 17
pi 1355 0.0 0.5 233052 5068 ? Ssl Jul22 0:00 /usr/libexec/gvfs-gphoto2-volume-monitor
root 395 0.0 0.5 14612 5040 ? Ss Jul22 0:12 /lib/systemd/systemd-logind
root 521 0.0 0.5 232740 5028 ? Ssl Jul22 0:00 /usr/sbin/lightdm
pi 1349 0.0 0.5 308296 5020 ? Ssl Jul22 2:56 /usr/libexec/gvfs-afc-volume-monitor
root 1020 0.0 0.5 23620 4856 ? Ss Jul22 0:00 /usr/libexec/bluetooth/bluetoothd
pi 1364 0.0 0.5 231212 4804 ? Ssl Jul22 0:00 /usr/libexec/gvfs-goa-volume-monitor
pi 1360 0.0 0.5 231052 4784 ? Ssl Jul22 0:00 /usr/libexec/gvfs-mtp-volume-monitor
pi 535798 0.0 0.5 156464 4748 ? Sl Jul31 0:00 /usr/libexec/dconf-service
pi 1538443 0.0 0.5 16076 4656 ? S 17:08 0:00 sshd: pi@pts/1
pi 1343 0.0 0.5 157016 4624 ? Sl Jul22 0:00 /usr/lib/menu-cache/menu-cached /run/user/1000/menu-cached-:0
pi 1413 0.0 0.5 157752 4620 ? Ssl Jul22 0:00 /usr/libexec/gvfsd-metadata
pi 1538444 0.0 0.5 5628 4576 pts/1 Ss 17:08 0:00 -bash
avahi 368 0.1 0.4 8368 4056 ? Ss Jul22 71:48 avahi-daemon: running [raspberrypi.local]
pi 674 0.0 0.4 168816 3996 ? S Jul22 0:00 (sd-pam)
root 544 0.0 0.4 13656 3924 ? Ss Jul22 0:00 sshd: /usr/sbin/sshd -D [listener] 0 of 10-100 startups
pi 1062 0.0 0.4 380564 3752 ? Sl Jul22 0:00 /usr/libexec/gvfsd-fuse /run/user/1000/gvfs -f
pi 545201 0.0 0.4 5504 3688 pts/0 Ss+ Jul31 0:00 bash
pi 1089 0.0 0.4 231564 3680 ? Sl Jul22 0:00 /usr/libexec/geoclue-2.0/demos/agent
root 388 0.0 0.4 220888 3656 ? Ssl Jul22 0:10 /usr/sbin/rsyslogd -n -iNONE
message+ 370 0.0 0.3 8672 3512 ? Ss Jul22 0:14 /usr/bin/dbus-daemon --system --address=systemd: --nofork --nopidfile --systemd-activation --syslog-only
pi 701 0.0 0.3 91808 3264 ? S<sl Jul22 0:00 /usr/bin/pipewire
pi 740 0.0 0.3 85312 3264 ? S<l Jul22 0:45 /usr/bin/pipewire-media-session
root 537 0.0 0.3 14088 3200 ? Ss Jul22 6:20 wpa_supplicant -B -c/etc/wpa_supplicant/wpa_supplicant.conf -iwlan0
pi 1538833 0.0 0.3 7208 3036 pts/1 R+ 17:17 0:00 ps aux --sort -%mem
pi 1538826 0.0 0.3 4212 3008 pts/1 S+ 17:17 0:00 /bin/bash /home/pi/Scripts/pstruc.sh psbline5
pi 716 0.0 0.3 5504 2916 tty1 S+ Jul22 0:00 -bash
pi 726 0.0 0.3 8040 2848 ? Ss Jul22 0:01 /usr/bin/dbus-daemon --session --address=systemd: --nofork --nopidfile --systemd-activation --syslog-only
root 400 0.0 0.3 13580 2844 ? Ss Jul22 0:21 /sbin/wpa_supplicant -u -s -O /run/wpa_supplicant
root 541 0.0 0.2 9160 2460 tty1 Ss Jul22 0:00 /bin/login -f
rtkit 707 0.0 0.2 153464 2012 ? SNsl Jul22 1:02 /usr/libexec/rtkit-daemon
root 464 0.0 0.2 3188 1972 ? Ss Jul22 7:32 /usr/sbin/dhcpcd -b -q
root 369 0.0 0.2 4112 1944 ? Ss Jul22 0:07 /usr/sbin/cron -f
nobody 396 0.0 0.2 5488 1860 ? Ss Jul22 0:40 /usr/sbin/thd --triggers /etc/triggerhappy/triggers.d/ --socket /run/thd.socket --user nobody --deviceglob /dev/input/event*
root 493 0.0 0.1 92296 1468 ? SLsl Jul22 0:25 /usr/sbin/rngd -r /dev/hwrng
root 518 0.0 0.1 4728 1380 ? Ss Jul22 0:00 /usr/bin/vncserver-x11-serviced -fg
pi 846 0.0 0.1 5452 1348 ? Ss Jul22 0:20 /usr/bin/ssh-agent x-session-manager
avahi 382 0.0 0.1 6892 1088 ? S Jul22 0:00 avahi-daemon: chroot helper
pi 1104 0.0 0.0 5228 672 ? S Jul22 0:00 xcompmgr -aR
pi 1090 0.0 0.0 5452 460 ? Ss Jul22 0:00 /usr/bin/ssh-agent -s
root 950 0.0 0.0 2132 132 ? S Jul22 0:00 /usr/bin/hciattach /dev/serial1 bcm43xx 3000000 flow - b8:27:eb:d4:88:98
root 2 0.0 0.0 0 0 ? S Jul22 0:10 [kthreadd]
root 3 0.0 0.0 0 0 ? I< Jul22 0:00 [rcu_gp]
root 4 0.0 0.0 0 0 ? I< Jul22 0:00 [rcu_par_gp]
root 8 0.0 0.0 0 0 ? I< Jul22 0:00 [mm_percpu_wq]
root 9 0.0 0.0 0 0 ? S Jul22 0:00 [rcu_tasks_kthre]
root 10 0.0 0.0 0 0 ? S Jul22 0:00 [rcu_tasks_rude_]
root 11 0.0 0.0 0 0 ? S Jul22 0:00 [rcu_tasks_trace]
root 12 0.0 0.0 0 0 ? S Jul22 0:48 [ksoftirqd/0]
root 13 0.0 0.0 0 0 ? I Jul22 9:20 [rcu_preempt]
root 14 0.0 0.0 0 0 ? S Jul22 0:00 [migration/0]
root 15 0.0 0.0 0 0 ? S Jul22 0:00 [cpuhp/0]
root 16 0.0 0.0 0 0 ? S Jul22 0:00 [cpuhp/1]
root 17 0.0 0.0 0 0 ? S Jul22 0:00 [migration/1]
root 18 0.0 0.0 0 0 ? S Jul22 0:32 [ksoftirqd/1]
root 21 0.0 0.0 0 0 ? S Jul22 0:00 [cpuhp/2]
root 22 0.0 0.0 0 0 ? S Jul22 0:00 [migration/2]
root 23 0.0 0.0 0 0 ? S Jul22 2:01 [ksoftirqd/2]
root 26 0.0 0.0 0 0 ? S Jul22 0:00 [cpuhp/3]
root 27 0.0 0.0 0 0 ? S Jul22 0:00 [migration/3]
root 28 0.0 0.0 0 0 ? S Jul22 0:23 [ksoftirqd/3]
root 31 0.0 0.0 0 0 ? S Jul22 0:00 [kdevtmpfs]
root 32 0.0 0.0 0 0 ? I< Jul22 0:00 [netns]
root 33 0.0 0.0 0 0 ? I< Jul22 0:00 [inet_frag_wq]
root 38 0.0 0.0 0 0 ? S Jul22 0:00 [kauditd]
root 40 0.0 0.0 0 0 ? S Jul22 0:04 [khungtaskd]
root 41 0.0 0.0 0 0 ? S Jul22 0:00 [oom_reaper]
root 42 0.0 0.0 0 0 ? I< Jul22 0:00 [writeback]
root 43 0.0 0.0 0 0 ? S Jul22 4:15 [kcompactd0]
root 63 0.0 0.0 0 0 ? I< Jul22 0:00 [kblockd]
root 64 0.0 0.0 0 0 ? I< Jul22 0:00 [blkcg_punt_bio]
root 65 0.0 0.0 0 0 ? S Jul22 0:00 [watchdogd]
root 68 0.0 0.0 0 0 ? I< Jul22 0:00 [rpciod]
root 70 0.0 0.0 0 0 ? I< Jul22 0:00 [xprtiod]
root 72 0.0 0.0 0 0 ? S Jul22 0:06 [kswapd0]
root 73 0.0 0.0 0 0 ? I< Jul22 0:00 [nfsiod]
root 74 0.0 0.0 0 0 ? I< Jul22 0:00 [kthrotld]
root 75 0.0 0.0 0 0 ? I< Jul22 0:00 [iscsi_eh]
root 76 0.0 0.0 0 0 ? I< Jul22 0:00 [iscsi_conn_clea]
root 77 0.0 0.0 0 0 ? I< Jul22 0:00 [nvme-wq]
root 78 0.0 0.0 0 0 ? I< Jul22 0:00 [nvme-reset-wq]
root 79 0.0 0.0 0 0 ? I< Jul22 0:00 [nvme-delete-wq]
root 80 0.0 0.0 0 0 ? I< Jul22 0:00 [dwc_otg]
root 81 0.0 0.0 0 0 ? I< Jul22 0:00 [DWC Notificatio]
root 82 0.0 0.0 0 0 ? I< Jul22 0:00 [uas]
root 83 0.0 0.0 0 0 ? S< Jul22 0:00 [vchiq-slot/0]
root 84 0.0 0.0 0 0 ? S< Jul22 0:00 [vchiq-recy/0]
root 85 0.0 0.0 0 0 ? S< Jul22 0:00 [vchiq-sync/0]
root 86 0.0 0.0 0 0 ? I< Jul22 0:00 [zswap-shrink]
root 88 0.0 0.0 0 0 ? I< Jul22 0:00 [mmc_complete]
root 92 0.0 0.0 0 0 ? S Jul22 0:39 [jbd2/mmcblk0p2-]
root 93 0.0 0.0 0 0 ? I< Jul22 0:00 [ext4-rsv-conver]
root 96 0.0 0.0 0 0 ? I< Jul22 0:00 [mld]
root 97 0.0 0.0 0 0 ? I< Jul22 0:00 [ipv6_addrconf]
root 116 0.0 0.0 0 0 ? S Jul22 0:00 [scsi_eh_0]
root 117 0.0 0.0 0 0 ? I< Jul22 0:00 [scsi_tmf_0]
root 118 0.0 0.0 0 0 ? S Jul22 2:47 [usb-storage]
root 120 0.0 0.0 0 0 ? S Jul22 0:00 [irq/184-usb-001]
root 191 0.0 0.0 0 0 ? S Jul22 0:00 [vchiq-keep/0]
root 193 0.0 0.0 0 0 ? S< Jul22 0:00 [SMIO]
root 200 0.0 0.0 0 0 ? I< Jul22 0:00 [mmal-vchiq]
root 201 0.0 0.0 0 0 ? I< Jul22 0:00 [mmal-vchiq]
root 204 0.0 0.0 0 0 ? I< Jul22 0:00 [mmal-vchiq]
root 206 0.0 0.0 0 0 ? I< Jul22 0:00 [mmal-vchiq]
root 208 0.0 0.0 0 0 ? I< Jul22 0:00 [mmal-vchiq]
root 210 0.0 0.0 0 0 ? I< Jul22 0:00 [mmal-vchiq]
root 211 0.0 0.0 0 0 ? I< Jul22 0:00 [mmal-vchiq]
root 235 0.0 0.0 0 0 ? I< Jul22 0:00 [cfg80211]
root 242 0.0 0.0 0 0 ? I< Jul22 0:00 [brcmf_wq/mmc1:0]
root 244 0.0 0.0 0 0 ? S Jul22 4:33 [brcmf_wdog/mmc1]
root 354 0.0 0.0 0 0 ? S Jul22 0:00 [cec-vc4]
root 355 0.0 0.0 0 0 ? S Jul22 0:00 [irq/82-vc4 hdmi]
root 359 0.0 0.0 0 0 ? S Jul22 0:00 [card0-crtc0]
root 360 0.0 0.0 0 0 ? S Jul22 0:00 [card0-crtc1]
root 361 0.0 0.0 0 0 ? S Jul22 0:00 [card0-crtc2]
root 362 0.0 0.0 0 0 ? S Jul22 0:00 [card0-crtc3]
root 1059 0.0 0.0 0 0 ? S< Jul22 0:00 [krfcommd]
root 1401 0.0 0.0 0 0 ? S Jul22 0:00 [jbd2/sda2-8]
root 1402 0.0 0.0 0 0 ? I< Jul22 0:00 [ext4-rsv-conver]
root 1301517 0.0 0.0 0 0 ? I< Aug13 0:13 [kworker/1:1H-mmc_complete]
root 1391723 0.0 0.0 0 0 ? I< Aug15 0:07 [kworker/0:0H-mmc_complete]
root 1452600 0.0 0.0 0 0 ? I< Aug16 0:04 [kworker/2:2H-mmc_complete]
root 1485198 0.0 0.0 0 0 ? I< Aug16 0:00 [kworker/u9:0]
root 1486549 0.0 0.0 0 0 ? I< Aug16 0:41 [kworker/u9:2-brcmf_wq/mmc11]
root 1537402 0.0 0.0 0 0 ? I< 16:42 0:00 [kworker/3:2H-kblockd]
root 1537453 0.0 0.0 0 0 ? I 16:43 0:00 [kworker/2:1-events]
root 1537589 0.0 0.0 0 0 ? I 16:47 0:00 [kworker/1:0-events]
root 1537796 0.0 0.0 0 0 ? I 16:52 0:00 [kworker/u8:0-events_unbound]
root 1537988 0.0 0.0 0 0 ? I 16:57 0:00 [kworker/0:1-events]
root 1538069 0.0 0.0 0 0 ? I< 16:59 0:00 [kworker/3:0H]
root 1538256 0.0 0.0 0 0 ? I< 17:04 0:00 [kworker/1:2H]
root 1538285 0.0 0.0 0 0 ? I< 17:05 0:00 [kworker/0:2H]
root 1538335 0.0 0.0 0 0 ? I 17:06 0:00 [kworker/u8:1-events_unbound]
root 1538336 0.0 0.0 0 0 ? I 17:06 0:00 [kworker/3:3-mm_percpu_wq]
root 1538414 0.0 0.0 0 0 ? I 17:08 0:00 [kworker/0:0-events]
root 1538453 0.0 0.0 0 0 ? I< 17:08 0:00 [kworker/2:0H]
root 1538549 0.0 0.0 0 0 ? I 17:11 0:00 [kworker/3:1-events]
root 1538572 0.0 0.0 0 0 ? I 17:11 0:00 [kworker/1:1-events]
root 1538583 0.0 0.0 0 0 ? I 17:12 0:00 [kworker/2:3-events]
root 1538634 0.0 0.0 0 0 ? I 17:13 0:00 [kworker/u8:2-events_unbound]
root 1538635 0.0 0.0 0 0 ? I< 17:13 0:00 [kworker/1:0H]
root 1538637 0.0 0.0 0 0 ? I 17:13 0:00 [kworker/0:2-events]
root 1538692 0.0 0.0 0 0 ? I< 17:15 0:00 [kworker/0:1H]
root 1538742 0.0 0.0 0 0 ? I 17:16 0:00 [kworker/3:0-events]
root 1538779 0.0 0.0 0 0 ? I 17:17 0:00 [kworker/1:2-events]
root 1538790 0.0 0.0 0 0 ? I< 17:17 0:00 [kworker/2:1H]
root 1538803 0.0 0.0 0 0 ? I 17:17 0:00 [kworker/2:0-events_freezable]
root 1538832 0.0 0.0 0 0 ? I< 17:17 0:00 [kworker/3:1H]

[Zatura24]

Hi, sorry for the late response

I've ran the pstruc.sh script and as can be seen, Domoticz is using 20%. I did notice that the script gets a snapshot of the usage since power on so I enabled dzVents, rebooted and let it run for a bit. I've also added the dummy script that I uses.

Spoiler: show

Code: Select all

top - 23:56:37 up  5:47,  1 user,  load average: 0.00, 0.03, 0.10
Tasks: 117 total,   1 running, 116 sleeping,   0 stopped,   0 zombie
%Cpu(s):  2.8 us,  1.4 sy,  0.0 ni, 95.8 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
MiB Mem :   3839.2 total,   3433.5 free,    119.4 used,    286.4 buff/cache
MiB Swap:    100.0 total,    100.0 free,      0.0 used.   3595.2 avail Mem
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
pi         562  0.0  1.7 179832 67316 ?        Ssl  18:09   0:17 node /usr/bin/easy-server --verbose --serial=460911730 --access-key=W5SvUDUyZCyBHGrU --password=kacheltje --host=127.0.0.1 --port 3000
root       661 18.5  1.3 288388 53648 ?        Ssl  18:09  64:17 /home/pi/domoticz/domoticz -daemon -www 8080 -sslwww 443
root       672  0.0  0.4  48652 16532 ?        Ss   18:09   0:00 /usr/sbin/smbd --foreground --no-process-group
root       559  0.0  0.2  33832 10112 ?        Ss   18:09   0:04 /usr/sbin/nmbd --foreground --no-process-group
root       383  0.0  0.2  21200  8116 ?        Ss   18:09   0:02 /lib/systemd/systemd-journald
root         1  0.0  0.1  15268  7840 ?        Ss   18:09   0:04 /sbin/init
pi        7014  0.0  0.1  14696  7320 ?        Ss   19:30   0:00 /lib/systemd/systemd --user
root      6881  0.0  0.1  12192  6252 ?        Ss   19:30   0:00 sshd: pi [priv]
root       392  0.0  0.1  13008  5756 ?        Ss   18:09   0:00 /lib/systemd/systemd-logind
root       688  0.0  0.1  48652  5752 ?        S    18:09   0:00 /usr/sbin/smbd --foreground --no-process-group
systemd+   388  0.0  0.1  22380  5460 ?        Ssl  18:09   0:00 /lib/systemd/systemd-timesyncd
root       685  0.0  0.1  45412  5312 ?        S    18:09   0:00 /usr/sbin/smbd --foreground --no-process-group
root       604  0.0  0.1  10680  5132 ?        Ss   18:09   0:00 /usr/sbin/sshd -D
root       686  0.0  0.1  45404  4124 ?        S    18:09   0:00 /usr/sbin/smbd --foreground --no-process-group
root       670  0.0  0.0  31964  3888 ?        S    18:09   0:00 /usr/sbin/nmbd --foreground --no-process-group
pi        7030  0.0  0.0   8600  3740 pts/0    Ss   19:30   0:00 -bash
root       154  0.0  0.0  17996  3736 ?        Ss   18:09   0:00 /lib/systemd/systemd-udevd
pi        7029  0.0  0.0  12192  3552 ?        S    19:30   0:00 sshd: pi@pts/0
message+   410  0.0  0.0   6560  2948 ?        Ss   18:09   0:00 /usr/bin/dbus-daemon --system --address=systemd: --nofork --nopidfile --systemd-activation --syslog-only
root       393  0.0  0.0  25500  2800 ?        Ssl  18:09   0:00 /usr/sbin/rsyslogd -n -iNONE
pi         584  0.0  0.0   9940  2624 pts/0    R+   23:56   0:00 ps aux --sort -%mem
avahi      396  0.0  0.0   5764  2620 ?        Ss   18:09   0:00 avahi-daemon: running [raspberrypi.local]
pi         581  1.0  0.0   7672  2540 pts/0    S+   23:56   0:00 /bin/bash ./pstruc.sh
root       422  0.0  0.0   7944  2304 ?        Ss   18:09   0:00 /usr/sbin/cron -f
root       564  0.0  0.0   6612  1732 ?        Ss+  18:09   0:00 /sbin/agetty -o -p -- \u --keep-baud 115200,38400,9600 ttyAMA0 vt220
pi        7015  0.0  0.0  16808  1724 ?        S    19:30   0:00 (sd-pam)
root       554  0.0  0.0   2828  1636 ?        Ss   18:09   0:00 /sbin/dhcpcd -q -w
root       565  0.0  0.0   4304  1280 tty1     Ss+  18:09   0:00 /sbin/agetty -o -p -- \u --noclear tty1 linux
root       425  0.0  0.0   3688   732 ?        SNs  18:09   0:00 /usr/sbin/alsactl -E HOME=/run/alsa -s -n 19 -c rdaemon
avahi      429  0.0  0.0   5764   260 ?        S    18:09   0:00 avahi-daemon: chroot helper
root       403  0.0  0.0  27652    80 ?        SLsl 18:09   0:00 /usr/sbin/rngd -r /dev/hwrng
root         2  0.0  0.0      0     0 ?        S    18:09   0:00 [kthreadd]
root         3  0.0  0.0      0     0 ?        I<   18:09   0:00 [rcu_gp]
root         4  0.0  0.0      0     0 ?        I<   18:09   0:00 [rcu_par_gp]
root         8  0.0  0.0      0     0 ?        I<   18:09   0:00 [mm_percpu_wq]
root         9  0.0  0.0      0     0 ?        S    18:09   0:00 [rcu_tasks_rude_]
root        10  0.0  0.0      0     0 ?        S    18:09   0:00 [rcu_tasks_trace]
root        11  0.0  0.0      0     0 ?        S    18:09   0:01 [ksoftirqd/0]
root        12  0.0  0.0      0     0 ?        I    18:09   0:11 [rcu_sched]
root        13  0.0  0.0      0     0 ?        S    18:09   0:00 [migration/0]
root        14  0.0  0.0      0     0 ?        S    18:09   0:00 [cpuhp/0]
root        15  0.0  0.0      0     0 ?        S    18:09   0:00 [cpuhp/1]
root        16  0.0  0.0      0     0 ?        S    18:09   0:00 [migration/1]
root        17  0.0  0.0      0     0 ?        S    18:09   0:01 [ksoftirqd/1]
root        20  0.0  0.0      0     0 ?        S    18:09   0:00 [cpuhp/2]
root        21  0.0  0.0      0     0 ?        S    18:09   0:00 [migration/2]
root        22  0.0  0.0      0     0 ?        S    18:09   0:01 [ksoftirqd/2]
root        25  0.0  0.0      0     0 ?        S    18:09   0:00 [cpuhp/3]
root        26  0.0  0.0      0     0 ?        S    18:09   0:00 [migration/3]
root        27  0.0  0.0      0     0 ?        S    18:09   0:01 [ksoftirqd/3]
root        30  0.0  0.0      0     0 ?        S    18:09   0:00 [kdevtmpfs]
root        31  0.0  0.0      0     0 ?        I<   18:09   0:00 [netns]
root        34  0.0  0.0      0     0 ?        S    18:09   0:00 [kauditd]
root        36  0.0  0.0      0     0 ?        S    18:09   0:00 [khungtaskd]
root        37  0.0  0.0      0     0 ?        S    18:09   0:00 [oom_reaper]
root        38  0.0  0.0      0     0 ?        I<   18:09   0:00 [writeback]
root        39  0.0  0.0      0     0 ?        S    18:09   0:01 [kcompactd0]
root        59  0.0  0.0      0     0 ?        I<   18:09   0:00 [kblockd]
root        60  0.0  0.0      0     0 ?        I<   18:09   0:00 [blkcg_punt_bio]
root        61  0.0  0.0      0     0 ?        S    18:09   0:00 [watchdogd]
root        62  0.0  0.0      0     0 ?        I<   18:09   0:04 [kworker/0:1H-mmc_complete]
root        63  0.0  0.0      0     0 ?        I<   18:09   0:00 [rpciod]
root        64  0.0  0.0      0     0 ?        I<   18:09   0:00 [kworker/u9:0]
root        65  0.0  0.0      0     0 ?        I<   18:09   0:00 [xprtiod]
root        66  0.0  0.0      0     0 ?        S    18:09   0:00 [kswapd0]
root        69  0.0  0.0      0     0 ?        I<   18:09   0:00 [nfsiod]
root        70  0.0  0.0      0     0 ?        I<   18:09   0:00 [kthrotld]
root        71  0.0  0.0      0     0 ?        I<   18:09   0:00 [iscsi_eh]
root        72  0.0  0.0      0     0 ?        I<   18:09   0:00 [iscsi_destroy]
root        73  0.0  0.0      0     0 ?        I<   18:09   0:00 [nvme-wq]
root        74  0.0  0.0      0     0 ?        I<   18:09   0:00 [nvme-reset-wq]
root        75  0.0  0.0      0     0 ?        I<   18:09   0:00 [nvme-delete-wq]
root        78  0.0  0.0      0     0 ?        I<   18:09   0:00 [DWC Notificatio]
root        79  0.0  0.0      0     0 ?        I<   18:09   0:00 [uas]
root        80  0.0  0.0      0     0 ?        S<   18:09   0:00 [vchiq-slot/0]
root        81  0.0  0.0      0     0 ?        S<   18:09   0:00 [vchiq-recy/0]
root        82  0.0  0.0      0     0 ?        S<   18:09   0:00 [vchiq-sync/0]
root        83  0.0  0.0      0     0 ?        I<   18:09   0:00 [zswap-shrink]
root        87  0.0  0.0      0     0 ?        I<   18:09   0:00 [sdhci]
root        88  0.0  0.0      0     0 ?        S    18:09   0:00 [irq/47-mmc0]
root        90  0.0  0.0      0     0 ?        I<   18:09   0:00 [mmc_complete]
root        92  0.0  0.0      0     0 ?        S    18:09   0:01 [jbd2/mmcblk0p2-]
root        93  0.0  0.0      0     0 ?        I<   18:09   0:00 [ext4-rsv-conver]
root        94  0.0  0.0      0     0 ?        I<   18:09   0:00 [kworker/2:1H-kblockd]
root        95  0.0  0.0      0     0 ?        I<   18:09   0:00 [ipv6_addrconf]
root       101  0.0  0.0      0     0 ?        I<   18:09   0:00 [kworker/1:1H-kblockd]
root       113  0.0  0.0      0     0 ?        I<   18:09   0:00 [kworker/3:2H-kblockd]
root       190  0.0  0.0      0     0 ?        S    18:09   0:00 [vchiq-keep/0]
root       191  0.0  0.0      0     0 ?        S<   18:09   0:00 [SMIO]
root       204  0.0  0.0      0     0 ?        I<   18:09   0:00 [mmal-vchiq]
root       210  0.0  0.0      0     0 ?        I<   18:09   0:00 [mmal-vchiq]
root       213  0.0  0.0      0     0 ?        I<   18:09   0:00 [mmal-vchiq]
root       215  0.0  0.0      0     0 ?        I<   18:09   0:00 [mmal-vchiq]
root       216  0.0  0.0      0     0 ?        I<   18:09   0:00 [mmal-vchiq]
root       358  0.0  0.0      0     0 ?        I    23:55   0:00 [kworker/3:0-events_power_efficient]
root       480  0.0  0.0      0     0 ?        I<   18:09   0:00 [cfg80211]
root       590  0.0  0.0      0     0 ?        I<   18:09   0:00 [wg-crypt-wg0]
root     11865  0.0  0.0      0     0 ?        I    22:42   0:00 [kworker/u8:0-events_unbound]
root     18278  0.0  0.0      0     0 ?        I    23:27   0:00 [kworker/1:2-mm_percpu_wq]
root     19031  0.0  0.0      0     0 ?        I    23:28   0:00 [kworker/u8:2-events_unbound]
root     22179  0.0  0.0      0     0 ?        I    23:32   0:00 [kworker/2:2-events_power_efficient]
root     23103  0.0  0.0      0     0 ?        I<   23:33   0:00 [kworker/2:0H]
root     25075  0.1  0.0      0     0 ?        I    23:35   0:01 [kworker/3:1-events]
root     29346  0.0  0.0      0     0 ?        I    23:44   0:00 [kworker/0:1-mm_percpu_wq]
root     29732  0.0  0.0      0     0 ?        I    23:45   0:00 [kworker/1:1-mm_percpu_wq]
root     30103  0.0  0.0      0     0 ?        I<   23:46   0:00 [kworker/0:0H]
root     30357  0.0  0.0      0     0 ?        I    23:47   0:00 [kworker/2:0-mm_percpu_wq]
root     31381  0.0  0.0      0     0 ?        I    23:49   0:00 [kworker/0:2-events_freezable_power_]
root     31512  0.0  0.0      0     0 ?        I    23:49   0:00 [kworker/3:2-events_power_efficient]
root     31688  0.0  0.0      0     0 ?        I<   23:50   0:00 [kworker/3:0H]
root     31689  0.0  0.0      0     0 ?        I<   23:50   0:00 [kworker/1:2H]
root     31959  0.0  0.0      0     0 ?        I<   23:51   0:00 [kworker/0:2H]
root     32151  0.0  0.0      0     0 ?        I    23:52   0:00 [kworker/2:1-mm_percpu_wq]
root     32211  0.0  0.0      0     0 ?        I<   23:53   0:00 [kworker/2:2H]
root     32326  0.0  0.0      0     0 ?        I    23:53   0:00 [kworker/1:0-wg-crypt-wg0]
root     32744  0.0  0.0      0     0 ?        I    23:55   0:00 [kworker/0:0-events_freezable_power_]

Code: Select all

return {
	on = {
		system = {
			'start',
		},
	},
	logging = {
		level = domoticz.LOG_INFO,
	},
	execute = function(domoticz, triggeredItem)
		domoticz.log('Domoticz has started')
	end
}

The test script is saved in the Domoticz database, which should not be an issue I assume. I'm also monitoring the full system usage and that jumped from around 1.5% to 6% when turning dzVents on. I'm mainly concerned for temperatures which nicely stay around 65c instead of jumping to 75c with dzVents turned on (yes I know these are safe temperatures).

When setting the logging to debug from dzVents I did however notice the following.

Spoiler: show

Code: Select all

2022-08-19 23:57:44.917 Status: dzVents: Debug: Dumping domoticz data to /home/pi/domoticz/scripts/dzVents/domoticzData.lua
2022-08-19 23:57:44.941 Status: dzVents: Debug: Processing device-adapter for Stroom: P1 smart meter energy device adapter
2022-08-19 23:57:44.941 Status: dzVents: Debug: dzVents version: 3.1.8
2022-08-19 23:57:44.941 Status: dzVents: Debug: Event triggers:
2022-08-19 23:57:44.941 Status: dzVents: Debug: - Device: Stroom
2022-08-19 23:57:45.000 Status: dzVents: Debug: Dumping domoticz data to /home/pi/domoticz/scripts/dzVents/domoticzData.lua
2022-08-19 23:57:45.048 Status: dzVents: Debug: Processing device-adapter for Verbruik L1: Electric usage device adapter
2022-08-19 23:57:45.050 Status: dzVents: Debug: Processing device-adapter for Delivery L1: Electric usage device adapter
2022-08-19 23:57:45.050 Status: dzVents: Debug: dzVents version: 3.1.8
2022-08-19 23:57:45.050 Status: dzVents: Debug: Event triggers:
2022-08-19 23:57:45.050 Status: dzVents: Debug: - Device: Verbruik L1
2022-08-19 23:57:45.050 Status: dzVents: Debug: - Device: Delivery L1
2022-08-19 23:57:45.949 Status: dzVents: Debug: Dumping domoticz data to /home/pi/domoticz/scripts/dzVents/domoticzData.lua
2022-08-19 23:57:45.986 Status: dzVents: Debug: Processing device-adapter for Stroom: P1 smart meter energy device adapter
2022-08-19 23:57:45.987 Status: dzVents: Debug: dzVents version: 3.1.8
2022-08-19 23:57:45.987 Status: dzVents: Debug: Event triggers:
2022-08-19 23:57:45.987 Status: dzVents: Debug: - Device: Stroom
2022-08-19 23:57:46.062 Status: dzVents: Debug: Dumping domoticz data to /home/pi/domoticz/scripts/dzVents/domoticzData.lua
2022-08-19 23:57:46.085 Status: dzVents: Debug: Processing device-adapter for Verbruik L1: Electric usage device adapter
2022-08-19 23:57:46.085 Status: dzVents: Debug: Processing device-adapter for Delivery L1: Electric usage device adapter
2022-08-19 23:57:46.085 Status: dzVents: Debug: dzVents version: 3.1.8
2022-08-19 23:57:46.085 Status: dzVents: Debug: Event triggers:
2022-08-19 23:57:46.086 Status: dzVents: Debug: - Device: Verbruik L1
2022-08-19 23:57:46.086 Status: dzVents: Debug: - Device: Delivery L1 

But I assume this is simple because that device is updating every second, and domoticzData is used to build the devices table in dzVents scripts. That device is listening on a usb, so that might also influence things?

In Domoticz I have 7 different hardware, from which 2 are dummy. In total there are 61 devices. Short sensors is set to 5 days. I have a single notification service turned on. If there is an easy way to share further configurations, it might help?

Not sure if any of this helps

[waltervl]

From documentation

When debug logging is enabled, every time dzVents kicks into action (Domoticz throws an event) it will log it, and it will create the files domoticzData.lua and module.log in /path/to/domoticz/scripts/dzVents. domoticzData.lua is a dump of all the data received by dzVents from Domoticz.

So If debugging is on it will do a lot more than when debugging is off.

[elzorrovega]

Hello Zatura,

Wow your CPU is rather high!

As Waltervi suggested, go to Setup; Settings; Others to minimise logs. Follow- up results.

Curiosity, do you really need to collect data every second as shown in your logs. If you need to cumulative values from your device, can you not get a total at a lesser frequency than every second.

I like to ensure that my RPI/Domoticz combination is optimum performance and I was intially having issues with the memory. Please follow link to older post which should help you to setup RPI diagnostic devices which you can follow values like CPU load, % MEM, etc. directly in Domoticz.

viewtopic.php?p=250388#p250388

HTOP command shows that you are using swap memory. Personally, I disabled it for it can lead to premature failure of your SD card. I use following BASH script which I run hourly in crontab to reset cache and ensure that swap memory is off.

Code: Select all

#!/bin/bash
# 14/11/2020
# El Zorro
# Script meant to Clear Cache Memory
# If you are using firefox as web browser,open page about:memory and press Minimize memory usage
# echo "restart lxpanel"
# lxpanelctl restart old command
# Updated following Erem's recommendation from 12/12/2020
# sudo logger "clcache.sh started"
# sleep 2s
DISPLAY=":0.0" lxpanelctl restart 2>&1
sleep 5s
# echo "Clear Cache Memory in Use"
# 1 Freeing Up the Page Cache
sudo sh -c "echo 1 > /proc/sys/vm/drop_caches"
sleep 2s
# 2. Freeing Up the Dentries and Inodes
sudo sh -c  "echo 2 > /proc/sys/vm/drop_caches"
sleep 2s
# 3. Freeing Up the Page Cache, Dentries and Inodes
sudo sh -c "echo 3 > /proc/sys/vm/drop_caches"
sleep 2s
# 4. Flushing the File System Buffers
sync
sleep 2s
sudo swapoff -a
sleep 2s
# sudo logger "clcache.sh executed"