Domoticz aborts after running detached script

[Oaknut]

HW: Raspberry Pi Model 3B 1.2 running Raspberry Pi OS 10 (Buster)
Attached HW: LogiLink Virtual 7.1 USB sound card , GPIO-connected generic 433.92 MHz transmitter
Domoticz Version: 2021.1 (previously 2.492 / 4.10717 / problems occurred after updating)
---
TL/DR: Domoticz runs fine after a reboot and using dummy switches (to trigger shell scripts) works fine. When using one of these dummy switches to trigger a shell script that runs detached (in this case streaming online radio (mp3 audio) using mplayer), Domoticz is no longer reachable after a couple of minutes. The music will keep on playing (without stuttering), but the only way to get Domoticz running again is by stopping the radio with:

Code: Select all

sudo killall mplayer 

--> I did not have this problem when I was running the previous update, and don't feel like going back to an older version, since I feel this should be a very, very basic issue.
--> Any idea where to look?

---

Extra info: For years now, Domoticz has been running at my parents' place as an interface for switching on their lights and to control an internet radio. I visited them recently and updated Domoticz and the OS. It seemed all was well until they started using the "radio" - which in itself is just a dummy switch with events behind it which trigger scripts (see below). Switching on the lights using dummy virtual switches (triggering simple shell scripts to send 433.92 MHz signals) does not cause any problems with Domoticz. It only happens when I run the mplayer scripts. Unfortunately, my 6-year younger self didn't really care for documenting the steps taken in setting stuff up, making it a little tricky to find out which files I need to look into.

---
Example event script, toggled by a virtual button, which causes the problem:

Code: Select all

commandArray = {}

if (devicechanged['Radio'] == 'Off') then
    os.execute ("killall mplayer")
end

if (devicechanged['Radio'] == 'FIP Rock') then
    os.execute ("killall mplayer")
    os.execute ("mplayer -noconsolecontrols https://icecast.radiofrance.fr/fiprock-hifi.aac?id=radiofrance &> /dev/null")
end 

---

Post reboot, running 4 instances of sudo systemctl status domoticz

Fresh reboot, triggering radio script with dummy switch

Code: Select all

pi@paulpi3B:~ $ sudo systemctl status domoticz
● domoticz.service
   Loaded: loaded (/etc/systemd/system/domoticz.service; enabled; vendor preset: enabled)
   Active: activating (start) since Sat 2021-10-30 16:02:51 CEST; 4min 45s ago
Cntrl PID: 834 (domoticz)
    Tasks: 17 (limit: 2088)
   CGroup: /system.slice/domoticz.service
           ├─834 /home/pi/domoticz/domoticz start #removed .sh from all of these
           └─880 mplayer -noconsolecontrols https://icecast.radiofrance.fr/fiprock-hifi.aac?id=radiofrance

Oct 30 16:07:31 paulpi3B domoticz[834]: 2021-10-30 16:07:31.453  Radio: Light/Switch (Radio)
Oct 30 16:07:31 paulpi3B domoticz[834]: mplayer: no process found
Oct 30 16:07:33 paulpi3B domoticz[834]: 2021-10-30 16:07:33.343  Status: User: Admin (IP: 192.168.1.81) initiated a switch command (1/Radio/Set Level)
Oct 30 16:07:33 paulpi3B domoticz[834]: 2021-10-30 16:07:33.362  Radio: Light/Switch (Radio)
Oct 30 16:07:33 paulpi3B domoticz[834]: mplayer: no process found
Oct 30 16:07:34 paulpi3B domoticz[834]: Cannot find HOME directory.
Oct 30 16:07:34 paulpi3B domoticz[834]: MPlayer 1.3.0 (Debian), built with gcc-8 (C) 2000-2016 MPlayer Team
Oct 30 16:07:34 paulpi3B domoticz[834]: Terminal type `unknown' is not defined.
Oct 30 16:07:34 paulpi3B domoticz[834]: Playing https://icecast.radiofrance.fr/fiprock-hifi.aac?id=radiofrance.
Oct 30 16:07:34 paulpi3B domoticz[834]: libavformat version 58.20.100 (external)

m3u8 Audio stream is still loading, can still switch lights on and off using the other dummy switches (Domoticz still responds)

Code: Select all

pi@paulpi3B:~ $ sudo systemctl status domoticz
● domoticz.service
   Loaded: loaded (/etc/systemd/system/domoticz.service; enabled; vendor preset: enabled)
   Active: activating (start) since Sat 2021-10-30 16:02:51 CEST; 4min 59s ago
Cntrl PID: 834 (domoticz)
    Tasks: 17 (limit: 2088)
   CGroup: /system.slice/domoticz.service
           ├─834 /home/pi/domoticz/domoticz start #removed .sh from all of these
           └─880 mplayer -noconsolecontrols https://icecast.radiofrance.fr/fiprock-hifi.aac?id=radiofrance

Oct 30 16:07:42 paulpi3B domoticz[834]: Cannot seek backward in linear streams!
Oct 30 16:07:43 paulpi3B domoticz[834]: 2021-10-30 16:07:43.068  Status: User: Admin (IP: 192.168.1.81) initiated a switch command (4/Ugly lamp/On)
Oct 30 16:07:43 paulpi3B domoticz[834]: 2021-10-30 16:07:43.073  Error: SQLHelper: Error script not found '/home/pi/WiringPi/examples/lights/./kaku'
Oct 30 16:07:43 paulpi3B domoticz[834]: 2021-10-30 16:07:43.076  Radio: Lighting 1 (Ugly lamp)
Oct 30 16:07:43 paulpi3B domoticz[834]: Cannot seek backward in linear streams!
Oct 30 16:07:43 paulpi3B domoticz[834]: Cannot seek backward in linear streams!
Oct 30 16:07:44 paulpi3B domoticz[834]: 2021-10-30 16:07:44.330  Status: User: Admin (IP: 192.168.1.81) initiated a switch command (4/Ugly lamp/Off)
Oct 30 16:07:44 paulpi3B domoticz[834]: 2021-10-30 16:07:44.336  Radio: Lighting 1 (Ugly lamp)
Oct 30 16:07:46 paulpi3B domoticz[834]: Cannot seek backward in linear streams!
Oct 30 16:07:46 paulpi3B domoticz[834]: Cannot seek backward in linear streams!

1 minute later, Domoticz is unreachable and doesn't respond, music starts playing at some point (m3u8 causes a bit of a delay, direct mp3 streams open immediately but also cause Domoticz to crash)

Code: Select all

pi@paulpi3B:~ $ sudo systemctl status domoticz
● domoticz.service
   Loaded: loaded (/etc/systemd/system/domoticz.service; enabled; vendor preset: enabled)
   Active: activating (start) since Sat 2021-10-30 16:08:20 CEST; 3s ago
Cntrl PID: 971 (domoticz)
    Tasks: 10 (limit: 2088)
   CGroup: /system.slice/domoticz.service
           ├─880 mplayer -noconsolecontrols https://icecast.radiofrance.fr/fiprock-hifi.aac?id=radiofrance
           ├─971 /home/pi/domoticz/domoticz start #removed .sh from all of these
           ├─976 /home/pi/domoticz/domoticz start #removed .sh from all of these
           ├─977 /home/pi/domoticz/domoticz start #removed .sh from all of these
           ├─978 /home/pi/domoticz/domoticz start #removed .sh from all of these
           └─979 gdb --batch -n -ex info threads -ex thread apply all bt -ex echo  Main thread: -ex bt -ex detach /home/pi/domoticz/domoticz 971

Oct 30 16:08:20 paulpi3B domoticz[971]: 2021-10-30 16:08:20.927  Error: [web:443] missing SSL certificate chain file ./server_cert.pem!
Oct 30 16:08:20 paulpi3B domoticz[971]: 2021-10-30 16:08:20.927  Error: [web:443] missing SSL certificate file ./server_cert.pem!
Oct 30 16:08:20 paulpi3B domoticz[971]: 2021-10-30 16:08:20.927  Error: [web:443] missing SSL private key file ./server_cert.pem!
Oct 30 16:08:20 paulpi3B domoticz[971]: 2021-10-30 16:08:20.927  Error: [web:443] missing SSL DH parameters file ./server_cert.pem!
Oct 30 16:08:20 paulpi3B domoticz[971]: 2021-10-30 16:08:20.928  Error: WebServer(SSL) startup failed on address 0.0.0.0 with port: 443: bind: Address already in use
Oct 30 16:08:20 paulpi3B domoticz[971]: 2021-10-30 16:08:20.928  Error: WebServer(SSL) check privileges for opening ports below 1024
Oct 30 16:08:20 paulpi3B domoticz[971]: 2021-10-30 16:08:20.928  Error: No servers are configured. Hence mydomoticz will not be started either (if configured)
Oct 30 16:08:20 paulpi3B domoticz[971]: terminate called without an active exception
Oct 30 16:08:20 paulpi3B domoticz[971]: 2021-10-30 16:08:20.929  Error: Domoticz(pid:971, tid:971('domoticz')) received fatal signal 6 (Aborted)
Oct 30 16:08:20 paulpi3B domoticz[971]: 2021-10-30 16:08:20.929  Error: siginfo address=0x3cb, address=0xffffffff

After running sudo killall mplayer, Domoticz is responsive immediately again:

Code: Select all

pi@paulpi3B:~ $ sudo systemctl status domoticz
● domoticz.service
   Loaded: loaded (/etc/systemd/system/domoticz.service; enabled; vendor preset: enabled)
   Active: activating (start) since Sat 2021-10-30 16:09:21 CEST; 32s ago
Cntrl PID: 2160 (domoticz)
    Tasks: 16 (limit: 2088)
   CGroup: /system.slice/domoticz.service
           └─2160 /home/pi/domoticz/domoticz start #removed .sh from all of these

Oct 30 16:09:21 paulpi3B domoticz[2160]: 2021-10-30 16:09:21.423  Error: WebServer(SSL) check privileges for opening ports below 1024
Oct 30 16:09:21 paulpi3B domoticz[2160]: 2021-10-30 16:09:21.425  Starting shared server on: :::6144
Oct 30 16:09:21 paulpi3B domoticz[2160]: 2021-10-30 16:09:21.425  Status: TCPServer: shared server started...
Oct 30 16:09:21 paulpi3B domoticz[2160]: 2021-10-30 16:09:21.426  Status: RxQueue: queue worker started...
Oct 30 16:09:23 paulpi3B domoticz[2160]: 2021-10-30 16:09:23.427  Status: NotificationSystem: thread started...
Oct 30 16:09:23 paulpi3B domoticz[2160]: 2021-10-30 16:09:23.427  Status: EventSystem: reset all events...
Oct 30 16:09:23 paulpi3B domoticz[2160]: 2021-10-30 16:09:23.428  Status: EventSystem: reset all device statuses...
Oct 30 16:09:23 paulpi3B domoticz[2160]: 2021-10-30 16:09:23.541  Status: Python EventSystem: Initalizing event module.
Oct 30 16:09:23 paulpi3B domoticz[2160]: 2021-10-30 16:09:23.542  Status: EventSystem: Started
Oct 30 16:09:23 paulpi3B domoticz[2160]: 2021-10-30 16:09:23.542  Status: EventSystem: Queue thread started...

[Oaknut]

Update:

I've already tried changing the radio event script, for instance adding

Code: Select all

 </dev/null >/dev/null 2>&1 & 

this doesn't change anything however. Running the statements in the radio event scripts from an SSH terminal doesn't make Domoticz crash (obviously).

[EddyG]

Did you try with 'sudo' before the killall command?

[Oaknut]

Did you try with 'sudo' before the killall command?

Hi Eddy, thanks for your reply. I've added the 'sudo' prefix to all of the 'killall mplayer' commands in the os.execute commands in the event script. Domoticz still freezes until 'sudo killall mplayer' is run via ssh in the terminal (using the pi user).

Update: Also tried adding 'runuser -l pi -c' to the os.execute commands (see example below), but this doesn't make a difference either.

Code: Select all

os.execute ("runuser -l pi -c 'sudo killall mplayer' ")
    os.execute ("runuser -l pi -c 'mplayer -noconsolecontrols https://icecast.radiofrance.fr/fiprock-hifi.aac?id=radiofrance &' ")

[EddyG]

Then try this one: viewtopic.php?p=211890#p211890 and log the output to Domoticz log. You might find the reason.

[Oaknut]

Then try this one: viewtopic.php?p=211890#p211890 and log the output to Domoticz log. You might find the reason.

Thanks for your reply. The hint in the link regarding the single-threaded nature of Domoticz events was quite useful. I created a simple script triggered by a virtual button which opens an mp3 stream in Mplayer. While I do try to run it in a detached fashion, for some reason it still makes Domoticz crash after a little bit. The logs make it seem as if Domoticz encounters an odd error (probably the thread being overloaded somehow), which starts multiple instances of Domoticz (or so it seems), which then causes it all to crash. Again, simply executing "sudo killall mplayer" causes the music to stop and Domoticz to be instantly functional again.

Starting the stream from Domoticz, no issues so far..:

Code: Select all

pi@paulpi3B:~/domoticz $ sudo systemctl status domoticz
● domoticz.service
   Loaded: loaded (/etc/systemd/system/domoticz.service; enabled; vendor preset: enabled)
   Active: activating (start) since Tue 2021-11-02 09:32:54 CET; 4min 34s ago
Cntrl PID: 19168 (domoticz)
    Tasks: 17 (limit: 2088)
   CGroup: /system.slice/domoticz.service
           ├─19168 /home/pi/domoticz/domoticz start #removed .sh from all of these
           ├─19195 mplayer -noconsolecontrols http://icecast.vrtcdn.be/stubru-high.mp3
           └─19198 mplayer -noconsolecontrols http://icecast.vrtcdn.be/stubru-high.mp3

Nov 02 09:32:57 paulpi3B domoticz[19168]: 2021-11-02 09:32:57.230  Radio: Lighting 1 (Radio Test (StuBru))
Nov 02 09:32:57 paulpi3B domoticz[19168]: 2021-11-02 09:32:57.451  Status: Executing script: /home/pi/domoticz/lights/./stubru
Nov 02 09:32:57 paulpi3B domoticz[19168]: mplayer: no process found
Nov 02 09:32:57 paulpi3B domoticz[19168]: 2021-11-02 09:32:57.474  Error: Error executing script command (/home/pi/domoticz/lights/./stubru). returned: 256
Nov 02 09:32:58 paulpi3B domoticz[19168]: 2021-11-02 09:32:58.783  Status: User: Admin (IP: 192.168.1.81) initiated a switch command (6/Radio Test (StuBru)/On)
Nov 02 09:32:58 paulpi3B domoticz[19168]: 2021-11-02 09:32:58.790  Radio: Lighting 1 (Radio Test (StuBru))
Nov 02 09:32:59 paulpi3B domoticz[19168]: 2021-11-02 09:32:58.999  Status: Executing script: /home/pi/domoticz/lights/./stubru

Domoticz is unresponsive, stream still plays (choppy at times):

Code: Select all

pi@paulpi3B:~/domoticz $ sudo systemctl status domoticz
● domoticz.service
   Loaded: loaded (/etc/systemd/system/domoticz.service; enabled; vendor preset: enabled)
   Active: activating (start) since Tue 2021-11-02 09:38:09 CET; 2s ago
Cntrl PID: 19312 (domoticz)
    Tasks: 11 (limit: 2088)
   CGroup: /system.slice/domoticz.service
           ├─19195 mplayer -noconsolecontrols http://icecast.vrtcdn.be/stubru-high.mp3
           ├─19198 mplayer -noconsolecontrols http://icecast.vrtcdn.be/stubru-high.mp3
           ├─19312 /home/pi/domoticz/domoticz start #removed .sh from all of these
           ├─19317 /home/pi/domoticz/domoticz start #removed .sh from all of these
           ├─19318 /home/pi/domoticz/domoticz start #removed .sh from all of these
           ├─19319 /home/pi/domoticz/domoticz start #removed .sh from all of these
           └─19320 gdb --batch -n -ex info threads -ex thread apply all bt -ex echo  Main thread: -ex bt -ex detach /home/pi/domoticz/domoticz 19312

Nov 02 09:38:09 paulpi3B domoticz[19312]: 2021-11-02 09:38:09.682  Error: [web:443] missing SSL certificate chain file ./server_cert.pem!
Nov 02 09:38:09 paulpi3B domoticz[19312]: 2021-11-02 09:38:09.682  Error: [web:443] missing SSL certificate file ./server_cert.pem!
Nov 02 09:38:09 paulpi3B domoticz[19312]: 2021-11-02 09:38:09.683  Error: [web:443] missing SSL private key file ./server_cert.pem!
Nov 02 09:38:09 paulpi3B domoticz[19312]: 2021-11-02 09:38:09.683  Error: [web:443] missing SSL DH parameters file ./server_cert.pem!
Nov 02 09:38:09 paulpi3B domoticz[19312]: 2021-11-02 09:38:09.683  Error: WebServer(SSL) startup failed on address 0.0.0.0 with port: 443: bind: Address already in use
Nov 02 09:38:09 paulpi3B domoticz[19312]: 2021-11-02 09:38:09.683  Error: WebServer(SSL) check privileges for opening ports below 1024
Nov 02 09:38:09 paulpi3B domoticz[19312]: 2021-11-02 09:38:09.683  Error: No servers are configured. Hence mydomoticz will not be started either (if configured)
Nov 02 09:38:09 paulpi3B domoticz[19312]: terminate called without an active exception
Nov 02 09:38:09 paulpi3B domoticz[19312]: 2021-11-02 09:38:09.683  Error: Domoticz(pid:19312, tid:19312('domoticz')) received fatal signal 6 (Aborted)

[EddyG]

The solution might be calling a bash or sh script that performs the action you want.
Make sure that that scripts immediately returns to Domoticz.