Domoticz

Open source Home Automation System
https://forum.domoticz.com/

Python plugin: Presence detection from wireless router

https://forum.domoticz.com/viewtopic.php?t=20467

Page 51 of 56

Re: Python plugin: Presence detection from wireless router

Posted: Friday 10 November 2023 13:26
by EscApe
1. Did you manage to install the 64-bit version of raspbian on your rpi3b? I don't think there is a official release for that(?) 64-bit requires rpi4 and up.
2. Domoticz is not yet compatible with Debian bookworm, so it's better to install the 'legacy' raspbian (bullseye)
3. You don't need to install pssh from source when you already installed it from the repo by sudo pip install parallel-ssh (so skip your step 4). This might have broken pssh (not saying it has, but it might)

Re: Python plugin: Presence detection from wireless router

Posted: Friday 10 November 2023 14:28
by Fredom
EscApe wrote: ↑Friday 10 November 2023 13:26 1. Did you manage to install the 64-bit version of raspbian on your rpi3b? I don't think there is a official release for that(?) 64-bit requires rpi4 and up.
2. Domoticz is not yet compatible with Debian bookworm, so it's better to install the 'legacy' raspbian (bullseye)
3. You don't need to install pssh from source when you already installed it from the repo by sudo pip install parallel-ssh (so skip your step 4). This might have broken pssh (not saying it has, but it might)
Hi EscApe,

Bookworm 64 bit runs well on the raspberry, chrome is a bit more unstable but it does work and is slow.
How can I undo step 4?
See photo 64 bit

Re: Python plugin: Presence detection from wireless router

Posted: Friday 10 November 2023 16:36
by EscApe
I stand corrected.. Didn't know a 64-bit image was available now.
I don't know how you can undo a manual install from source or if rerunning "sudo pip install parallel-ssh" would simply overwrite it.
There are several mentions on the Domoticz Forum and tweakers.net of Domoticz not being compatible with bookworm, because of the ssl version in bookworm. That is probably unrelated to the pssh problem, but I would still stick with bullseye for now.

There is a difference between the python environment in Domoticz and the python environment when running python from the commandline as the same (root) user. I don't have the skills or knowledge to troubleshoot that, so besides of some experimenting I can't do much more to solve this issue.

To see the difference you can login to the same rpi, start python3 en run the following code (make sure to edit the password):

Code: Select all

from pssh.clients.ssh import SSHClient
client = SSHClient('127.0.0.1', port=22, user='pi', password='<yourpassword>', timeout=5)
output = client.run_command('ls -al')
ssh_output='\n'.join(map(str,output.stdout))
print(ssh_output)
The code ran fine on my rpi3 running 32-bit raspberry os, but the same code would fail inside a (otherwise empty) plugin on the same rpi.

Re: Python plugin: Presence detection from wireless router

Posted: Friday 10 November 2023 17:04
by Fredom
EscApe wrote: ↑Friday 10 November 2023 16:36 I stand corrected.. Didn't know a 64-bit image was available now.
I don't know how you can undo a manual install from source or if rerunning "sudo pip install parallel-ssh" would simply overwrite it.
There are several mentions on the Domoticz Forum and tweakers.net of Domoticz not being compatible with bookworm, because of the ssl version in bookworm. That is probably unrelated to the pssh problem, but I would still stick with bullseye for now.

There is a difference between the python environment in Domoticz and the python environment when running python from the commandline as the same (root) user. I don't have the skills or knowledge to troubleshoot that, so besides of some experimenting I can't do much more to solve this issue.

To see the difference you can login to the same rpi, start python3 en run the following code (make sure to edit the password):

Code: Select all

from pssh.clients.ssh import SSHClient
client = SSHClient('127.0.0.1', port=22, user='pi', password='<yourpassword>', timeout=5)
output = client.run_command('ls -al')
ssh_output='\n'.join(map(str,output.stdout))
print(ssh_output)
The code ran fine on my rpi3 running 32-bit raspberry os, but the same code would fail inside a (otherwise empty) plugin on the same rpi.
Hi,
I have tried your code in a number of different ways but I always get the same error, a syntax error. I think it is better to go back to Bulsseye then but with the IP instead of the MAC addresses in iDetect

Re: Python plugin: Presence detection from wireless router

Posted: Thursday 16 November 2023 9:40
by rensbr
Yesterday I updated Domoticz to the newest version 2023.2 and not the plugin does not want to start, i'm getting the following erros:

I also have updated the plugin itself.

Code: Select all

2023-11-16 09:34:33.077 Error: Presence detection: 192.168.0.100 ====> SSH Could not connect (using password). Exception: Error reading SSH protocol banner
2023-11-16 09:34:43.108 Error: Presence detection: 192.168.0.104 ====> SSH Could not connect (using password). Exception: Error reading SSH protocol banner
2023-11-16 09:34:53.120 Error: Presence detection: 192.168.0.103 ====> SSH Could not connect (using password). Exception: Error reading SSH protocol banner
2023-11-16 09:35:03.171 Error: Presence detection: 192.168.0.102 ====> SSH Could not connect (using password). Exception: Error reading SSH protocol banner
2023-11-16 09:35:53.089 Error: Presence detection: 192.168.0.100 ====> SSH Could not connect (using password). Exception: Error reading SSH protocol banner
2023-11-16 09:36:03.118 Error: Presence detection: 192.168.0.104 ====> SSH Could not connect (using password). Exception: Error reading SSH protocol banner
2023-11-16 09:36:13.129 Error: Presence detection: 192.168.0.103 ====> SSH Could not connect (using password). Exception: Error reading SSH protocol banner
2023-11-16 09:36:23.180 Error: Presence detection: 192.168.0.102 ====> SSH Could not connect (using password). Exception: Error reading SSH protocol banner
2023-11-16 09:37:38.110 Error: Presence detection: 192.168.0.100 ====> SSH Could not connect (using password). Exception: No existing session
2023-11-16 09:37:48.136 Error: Presence detection: 192.168.0.104 ====> SSH Could not connect (using password). Exception: No existing session
2023-11-16 09:37:58.147 Error: Presence detection: 192.168.0.103 ====> SSH Could not connect (using password). Exception: No existing session
2023-11-16 09:38:08.207 Error: Presence detection: 192.168.0.102 ====> SSH Could not connect (using password). Exception: No existing session
2023-11-16 09:37:53.135 Presence detection: 192.168.0.103 Not (yet) fully initialized for polling
2023-11-16 09:37:53.186 Presence detection: 192.168.0.102 Not (yet) fully initialized for polling
What do I do wrong here?

I'm running Domoticz inside Docker and installed 'paramiko' within the docker with the dockerfile:

Code: Select all

FROM domoticz/domoticz

EXPOSE 8080
EXPOSE 444

WORKDIR /opt/domoticz

RUN pip install --upgrade pip
RUN pip3 install --upgrade pip

RUN pip install pylgtv paramiko
RUN pip install paramiko

RUN pip3 install wheel
RUN pip3 install pyfujitsu

ENTRYPOINT ["docker-entrypoint.sh"]

CMD ["/opt/domoticz/domoticz"]
When I disable and enable the plugin, I'm getting the following errors:

Code: Select all

2023-11-16 09:42:02.678 Error: Presence detection: Call to function 'onStart' failed, exception details:
2023-11-16 09:42:02.683 Error: Presence detection: Traceback (most recent call last):
2023-11-16 09:42:02.684 Error: Presence detection: File "/opt/domoticz/userdata/plugins/iDetect2/plugin.py", line 431, in onStart
2023-11-16 09:42:02.684 Error: Presence detection: _plugin.onStart()
2023-11-16 09:42:02.684 Error: Presence detection: File "/opt/domoticz/userdata/plugins/iDetect2/plugin.py", line 212, in onStart
2023-11-16 09:42:02.684 Error: Presence detection: from trackers import poll_methods
2023-11-16 09:42:02.684 Error: Presence detection: File "/opt/domoticz/userdata/plugins/iDetect2/trackers/__init__.py", line 6, in <module>
2023-11-16 09:42:02.684 Error: Presence detection: from trackers.ssh_tracker import ssh_tracker
2023-11-16 09:42:02.684 Error: Presence detection: File "/opt/domoticz/userdata/plugins/iDetect2/trackers/ssh_tracker.py", line 8, in <module>
2023-11-16 09:42:02.684 Error: Presence detection: import paramiko
2023-11-16 09:42:02.684 Error: Presence detection: File "/usr/local/lib/python3.9/dist-packages/paramiko/__init__.py", line 22, in <module>
2023-11-16 09:42:02.684 Error: Presence detection: from paramiko.transport import (
2023-11-16 09:42:02.684 Error: Presence detection: File "/usr/local/lib/python3.9/dist-packages/paramiko/transport.py", line 33, in <module>
2023-11-16 09:42:02.684 Error: Presence detection: from cryptography.hazmat.primitives.ciphers import algorithms, Cipher, modes
2023-11-16 09:42:02.684 Error: Presence detection: File "/usr/local/lib/python3.9/dist-packages/cryptography/hazmat/primitives/ciphers/__init__.py", line 11, in <module>
2023-11-16 09:42:02.684 Error: Presence detection: from cryptography.hazmat.primitives.ciphers.base import (
2023-11-16 09:42:02.684 Error: Presence detection: File "/usr/local/lib/python3.9/dist-packages/cryptography/hazmat/primitives/ciphers/base.py", line 10, in <module>
2023-11-16 09:42:02.684 Error: Presence detection: from cryptography.exceptions import (
2023-11-16 09:42:02.684 Error: Presence detection: File "/usr/local/lib/python3.9/dist-packages/cryptography/exceptions.py", line 9, in <module>
2023-11-16 09:42:02.684 Error: Presence detection: from cryptography.hazmat.bindings._rust import exceptions as rust_exceptions
2023-11-16 09:42:02.684 Error: Presence detection: ImportError: PyO3 modules may only be initialized once per interpreter process
What I have tried:
- Rebooting the whole system
- Installing pip install parallel-ssh inside the docker
- Run pip install cryptography==40.0.2 --upgrade inside the docker

Edit:

I have it working again. I have changed the dockerfile to this below and then rebooted the whole system

Code: Select all

FROM domoticz/domoticz

EXPOSE 8080
EXPOSE 444

WORKDIR /opt/domoticz

RUN pip install --upgrade pip
RUN pip3 install --upgrade pip

RUN pip install pylgtv paramiko
RUN pip install paramiko
RUN pip install parallel-ssh

RUN pip3 install wheel
RUN pip3 install pyfujitsu

ENTRYPOINT ["docker-entrypoint.sh"]

CMD ["/opt/domoticz/domoticz"]
Edit2:
I had another plugin which did not work correctly, that was the deconz plugin. But when I disable and enable this plugin, both plugins crash:

Code: Select all

2023-11-16 11:21:18.766 Presence detection: 192.168.0.100 Could not be polled
2023-11-16 11:21:18.766 Error: Presence detection: 192.168.0.100 ====> SSH Could not connect (using password). Exception: No existing session
2023-11-16 11:21:19.831 Presence detection: 192.168.0.104 Could not be polled
2023-11-16 11:21:19.830 Error: Presence detection: 192.168.0.104 ====> SSH Could not connect (using password). Exception: No existing session
2023-11-16 11:21:20.664 Presence detection: 192.168.0.103 Could not be polled
2023-11-16 11:21:20.664 Error: Presence detection: 192.168.0.103 ====> SSH Could not connect (using password). Exception: No existing session
2023-11-16 11:21:21.619 Presence detection: 192.168.0.102 Could not be polled
2023-11-16 11:21:21.619 Error: Presence detection: 192.168.0.102 ====> SSH Could not connect (using password). Exception: No existing session
2023-11-16 11:21:23.092 Conbee II: ### Request config
2023-11-16 11:21:23.093 Error: Conbee II: Your python version is missing the requests library
2023-11-16 11:21:23.093 Error: Conbee II: To install it, type : sudo -H pip3 install requests | sudo -H pip install requests
Edit2600:

I have It all working, apparently the request library was needed in this version, so i have added that to the dockerfile aswell:

Code: Select all

FROM domoticz/domoticz

EXPOSE 8080
EXPOSE 444

WORKDIR /opt/domoticz

RUN pip install --upgrade pip
RUN pip3 install --upgrade pip

#FOR LG plugin
RUN pip install pylgtv paramiko
#FOR Precense detection
RUN pip install paramiko
RUN pip install parallel-ssh
#FOR conbee II plugin
RUN pip install requests
RUN pip3 install requests

#FOR airco
RUN pip3 install wheel
RUN pip3 install pyfujitsu

ENTRYPOINT ["docker-entrypoint.sh"]

CMD ["/opt/domoticz/domoticz"]

Result:

Code: Select all

2023-11-16 11:32:21.039 Status: Presence detection: 192.168.0.103 Tracker initialized
2023-11-16 11:32:21.040 Status: Presence detection: Starting address:192.168.0.102, port:None, user:admin, keyfile:, class:ssh_autodetect and poll interval:10
2023-11-16 11:32:21.379 Status: Presence detection: 192.168.0.102 ====> SSH connection established
2023-11-16 11:32:21.533 Status: Presence detection: 192.168.0.102 Tracker initialized

Re: Python plugin: Presence detection from wireless router

Posted: Thursday 23 November 2023 8:09
by rensbr
Troubles again :(

I have updated my docker version as well to get some other things working, and not I'm getting the following errors:

Code: Select all

2023-11-23 08:07:08.631 Error: Presence detection: Call to function 'onStart' failed, exception details:
2023-11-23 08:07:08.636 Error: Presence detection: Traceback (most recent call last):
2023-11-23 08:07:08.636 Error: Presence detection: File "/opt/domoticz/userdata/plugins/iDetect2/plugin.py", line 431, in onStart
2023-11-23 08:07:08.636 Error: Presence detection: _plugin.onStart()
2023-11-23 08:07:08.636 Error: Presence detection: File "/opt/domoticz/userdata/plugins/iDetect2/plugin.py", line 212, in onStart
2023-11-23 08:07:08.636 Error: Presence detection: from trackers import poll_methods
2023-11-23 08:07:08.636 Error: Presence detection: File "/opt/domoticz/userdata/plugins/iDetect2/trackers/__init__.py", line 6, in <module>
2023-11-23 08:07:08.636 Error: Presence detection: from trackers.ssh_tracker import ssh_tracker
2023-11-23 08:07:08.636 Error: Presence detection: File "/opt/domoticz/userdata/plugins/iDetect2/trackers/ssh_tracker.py", line 8, in <module>
2023-11-23 08:07:08.636 Error: Presence detection: import paramiko
2023-11-23 08:07:08.636 Error: Presence detection: File "/usr/local/lib/python3.9/dist-packages/paramiko/__init__.py", line 22, in <module>
2023-11-23 08:07:08.636 Error: Presence detection: from paramiko.transport import (
2023-11-23 08:07:08.636 Error: Presence detection: File "/usr/local/lib/python3.9/dist-packages/paramiko/transport.py", line 33, in <module>
2023-11-23 08:07:08.637 Error: Presence detection: from cryptography.hazmat.primitives.ciphers import algorithms, Cipher, modes
2023-11-23 08:07:08.637 Error: Presence detection: File "/usr/local/lib/python3.9/dist-packages/cryptography/hazmat/primitives/ciphers/__init__.py", line 11, in <module>
2023-11-23 08:07:08.637 Error: Presence detection: from cryptography.hazmat.primitives.ciphers.base import (
2023-11-23 08:07:08.637 Error: Presence detection: File "/usr/local/lib/python3.9/dist-packages/cryptography/hazmat/primitives/ciphers/base.py", line 10, in <module>
2023-11-23 08:07:08.637 Error: Presence detection: from cryptography.exceptions import (
2023-11-23 08:07:08.637 Error: Presence detection: File "/usr/local/lib/python3.9/dist-packages/cryptography/exceptions.py", line 9, in <module>
2023-11-23 08:07:08.637 Error: Presence detection: from cryptography.hazmat.bindings._rust import exceptions as rust_exceptions
2023-11-23 08:07:08.637 Error: Presence detection: ImportError: PyO3 modules may only be initialized once per interpreter process
Any thoughts? I have rebuild the docker container, restarted the plugin, restarted Domoticz and restarted the entire system. All without success.

Re: Python plugin: Presence detection from wireless router

Posted: Thursday 23 November 2023 8:16
by EscApe
rensbr wrote: ↑Thursday 23 November 2023 8:09 Troubles again :(

I have updated my docker version as well to get some other things working, and not I'm getting the following errors:

Code: Select all

2023-11-23 08:07:08.637 Error: Presence detection: ImportError: PyO3 modules may only be initialized once per interpreter process
This is a known problem in the Domoticz framework and also occurs with other plugins. Restarting a plugin seems to not really reset the python environment to the initial state and some modules choke on that. Restarting Domoticz is the only way to safely restart plugins at the moment. There some mentions about this in the forum and GitHub issue log.

Re: Python plugin: Presence detection from wireless router

Posted: Thursday 23 November 2023 8:20
by rensbr
EscApe wrote: ↑Thursday 23 November 2023 8:16
rensbr wrote: ↑Thursday 23 November 2023 8:09 Troubles again :(

I have updated my docker version as well to get some other things working, and not I'm getting the following errors:

Code: Select all

2023-11-23 08:07:08.637 Error: Presence detection: ImportError: PyO3 modules may only be initialized once per interpreter process
This is a known problem in the Domoticz framework and also occurs with other plugins. Restarting a plugin seems to not really reset the python environment to the initial state and some modules choke on that. Restarting Domoticz is the only way to safely restart plugins at the moment. There some mentions about this in the forum and GitHub issue log.
Thanks for the fast reply. I was busy with editing my post, but I have done the following already: Rebuild the docker container, restarted the plugin, restarted Domoticz and restarted the whole system. All ended up with the same error.

Edit, when I look in the logger from the container I also see the following error:

Code: Select all

Traceback (most recent call last):
  File "/usr/lib/python3.9/threading.py", line 954, in _bootstrap_inner
    self.run()
  File "/usr/local/lib/python3.9/dist-packages/paramiko/transport.py", line 2219, in run
    emsg = "{} ({:d})".format(e.args[1], e.args[0])
IndexError: tuple index out of range
2023-11-23 08:22:08.453  Presence detection: 192.168.0.100 Not (yet) fully initialized for polling
2023-11-23 08:22:08.487  Presence detection: 192.168.0.104 Not (yet) fully initialized for polling
2023-11-23 08:22:08.506  Error: Presence detection: 192.168.0.103 ====> SSH Could not connect (using password). Exception: No existing session
2023-11-23 08:22:13.520  Presence detection: 192.168.0.102 Not (yet) fully initialized for polling
2023-11-23 08:22:13.520  Status: Presence detection: 192.168.0.102 tracker initialization (still) not completed after waiting at least 30 seconds. Retrying 5 more times
Exception in thread Thread-57:
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/dist-packages/paramiko/transport.py", line 2129, in run
    ptype, m = self.packetizer.read_message()
  File "/usr/local/lib/python3.9/dist-packages/paramiko/packet.py", line 453, in read_message
    header = self.read_all(self.__block_size_in, check_rekey=True)
  File "/usr/local/lib/python3.9/dist-packages/paramiko/packet.py", line 301, in read_all
    x = self.__socket.recv(n)
socket.timeout: timed out
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "/usr/lib/python3.9/threading.py", line 954, in _bootstrap_inner
    self.run()
  File "/usr/local/lib/python3.9/dist-packages/paramiko/transport.py", line 2219, in run
    emsg = "{} ({:d})".format(e.args[1], e.args[0])
IndexError: tuple index out of range
2023-11-23 08:22:28.455  Presence detection: 192.168.0.100 Not (yet) fully initialized for polling
2023-11-23 08:22:28.476  Domoticz Server: General/Percentage (CPU_Usage)
2023-11-23 08:22:28.478  Domoticz Server: General/Percentage (Memory Usage)
2023-11-23 08:22:28.480  Domoticz Server: General/Custom Sensor (Process Usage)
2023-11-23 08:22:28.488  Presence detection: 192.168.0.104 Not (yet) fully initialized for polling
2023-11-23 08:22:28.508  Presence detection: 192.168.0.103 Not (yet) fully initialized for polling
2023-11-23 08:22:28.531  Presence detection: 192.168.0.102 Not (yet) fully initialized for polling
Edit26:

I have changed the following and now it works again:
1. I have executed the following command in the docker container: 'pip install cryptography==40.0.2 --upgrade' as mentioned in viewtopic.php?p=305792#p305792
2. Changed the Tracker syntax from: 192.168.0.100#user=USER&password=PASSWORD&interval=10 to: 192.168.0.100.
3. Rebooted Domoticz container for the 1000 time.

No idea what from the above steps did the trick, but it works again. Thanks for the support!

Edit27:

That still did not quite do the trick. I have some conflicts with the Domoticz-Fujitsu-AC plugin. When that is enabled and I restart the container, I'm getting the same errors as above. When I disable the Presence detection plugin, restart the container. Let everything start-up and then enable the Presence detection plugin it works without any problems. Very strange, but for now this is a work arround.

Re: Python plugin: Presence detection from wireless router

Posted: Friday 24 November 2023 8:06
by EscApe
I'm not sure what you meen by "the same error", because your logs show different errors, but I think you are on the right track. Maybe there is a conflicting (version) requirement between the imported modules. I have had lots of trouble with because of some underlying modules changed.

Looks like you are already Installed parallel-ssh. You could try the beta iDetect plugin that also uses parallel-ssh instead of paramiko.

Code: Select all

sudo git clone -b DomoticzEx-based-beta https://github.com/d-EScape/Domoticz_iDetect.git iDetect
The beta will require you te remove existing (tracked) devices from Domoticz and new devices will be created automatically.

You might also consider using the standard Domoticz docker image and installing the required modules using a "customstart.sh" file in userdata. Mine looks like this:

Code: Select all

#!/bin/bash

if [ -f /opt/domoticz/FIRSTRUN ]; then
echo 'Domoticz container already prepped'
else
echo 'Preparing Domoticz container for FIRSTRUN'
echo 'installing python modules'
pip install cryptography setuptools requests parallel-ssh paramiko motionblinds python_picnic_api paho-mqtt gTTS pychromecast hwmon fritzconnect>
echo 'updating linux packages'
apt-get -qq update
echo 'installing linux packages'
apt-get install -y \
    iputils-ping \
    nano
echo 'creating FIRSTRUN file so script does not run again on container start'
touch /opt/domoticz/FIRSTRUN
cd /opt/domoticz || return
fi
Works like a charm and is easier to maintain.

I just verified and I have no problem running the stable version of iDetect from a fresh docker container using the above approach. Have not yet tested any possibly conflicting plugins.

Re: Python plugin: Presence detection from wireless router

Posted: Monday 27 November 2023 14:09
by rensbr
EscApe wrote: ↑Friday 24 November 2023 8:06 I'm not sure what you meen by "the same error", because your logs show different errors, but I think you are on the right track. Maybe there is a conflicting (version) requirement between the imported modules. I have had lots of trouble with because of some underlying modules changed.
With the same error I mean this one:

Code: Select all

Exception in thread Thread-57:
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/dist-packages/paramiko/transport.py", line 2129, in run
    ptype, m = self.packetizer.read_message()
  File "/usr/local/lib/python3.9/dist-packages/paramiko/packet.py", line 453, in read_message
    header = self.read_all(self.__block_size_in, check_rekey=True)
  File "/usr/local/lib/python3.9/dist-packages/paramiko/packet.py", line 301, in read_all
    x = self.__socket.recv(n)
socket.timeout: timed out
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "/usr/lib/python3.9/threading.py", line 954, in _bootstrap_inner
    self.run()
  File "/usr/local/lib/python3.9/dist-packages/paramiko/transport.py", line 2219, in run
    emsg = "{} ({:d})".format(e.args[1], e.args[0])
IndexError: tuple index out of range
You might also consider using the standard Domoticz docker image and installing the required modules using a "customstart.sh" file in userdata. Mine looks like this:

Code: Select all

#!/bin/bash

if [ -f /opt/domoticz/FIRSTRUN ]; then
echo 'Domoticz container already prepped'
else
echo 'Preparing Domoticz container for FIRSTRUN'
echo 'installing python modules'
pip install cryptography setuptools requests parallel-ssh paramiko motionblinds python_picnic_api paho-mqtt gTTS pychromecast hwmon fritzconnect>
echo 'updating linux packages'
apt-get -qq update
echo 'installing linux packages'
apt-get install -y \
    iputils-ping \
    nano
echo 'creating FIRSTRUN file so script does not run again on container start'
touch /opt/domoticz/FIRSTRUN
cd /opt/domoticz || return
fi
I see that you are using pip, instead of pip3 to install modules, what is the reason you do this? Or what is the difference?
Looks like you are already Installed parallel-ssh. You could try the beta iDetect plugin that also uses parallel-ssh instead of paramiko.

Code: Select all

sudo git clone -b DomoticzEx-based-beta https://github.com/d-EScape/Domoticz_iDetect.git iDetect
The beta will require you te remove existing (tracked) devices from Domoticz and new devices will be created automatically.
I have installed this beta plugin and it looks like it works. When I restart Domoticz I just works without any issues or errors. So, great thanks! I'm going to use the beta plugin from now-on.

Re: Python plugin: Presence detection from wireless router

Posted: Monday 27 November 2023 16:49
by EscApe
Glad that the new approach (beta version) is working for you.

There is no real reason for using pip instead of pip3. I noticed it does exactly the same on systems that only have python3, so I just stopped typing the 3. Call me lazy ;-)

Re: Python plugin: Presence detection from wireless router

Posted: Thursday 30 November 2023 9:48
by rensbr
EscApe wrote: ↑Monday 27 November 2023 16:49 Glad that the new approach (beta version) is working for you.

There is no real reason for using pip instead of pip3. I noticed it does exactly the same on systems that only have python3, so I just stopped typing the 3. Call me lazy ;-)
Good to know. Thanks.

After the last installation of Tuesday I'm running into memory leak problems, is that something you recognize from the beta plugin?
Within out 6 to 8 hours the whole memory runs full, then Domoticz does 'something', I have no idea what that is, since everything is unresponsive when the memory is full. The memory is cleaned up then and it will run full again over time. Here you can see the log of the memory:
Screenshot 2023-11-30 094414.png
Screenshot 2023-11-30 094414.png (47.58 KiB) Viewed 10655 times
I'm not sure if this has anything to do with your plugin, but I have no idea how I can look into the docker container to look for the separate RAM usages.

Re: Python plugin: Presence detection from wireless router

Posted: Thursday 30 November 2023 17:19
by EscApe
Nope, no (noticeable) memory leaks on my system.

Re: Python plugin: Presence detection from wireless router

Posted: Sunday 03 December 2023 10:19
by rensbr
EscApe wrote: ↑Thursday 30 November 2023 17:19 Nope, no (noticeable) memory leaks on my system.
Thanks for all the support and responses sofar.

I have looked into some stuff. I'm sure that the problem is caused by the plug-in. When I disable the plugin the problem is gone. When is change the interval from 10 to 60 seconds it takes more time to fill the memory.

Is there something I can do to look what happens or where the problem may be?

Re: Python plugin: Presence detection from wireless router

Posted: Sunday 03 December 2023 13:19
by waltervl
There is a memory leak reported on the python plugin framework when using the plugin device touch function: https://github.com/domoticz/domoticz/issues/5881

I did not check if this plugin uses this function.

Re: Python plugin: Presence detection from wireless router

Posted: Sunday 03 December 2023 14:41
by EscApe
waltervl wrote: ↑Sunday 03 December 2023 13:19 There is a memory leak reported on the python plugin framework when using the plugin device touch function: https://github.com/domoticz/domoticz/issues/5881

I did not check if this plugin uses this function.
Thanks for the tip! The plugin doesn't use the touch function but I appreciate your help!

Re: Python plugin: Presence detection from wireless router

Posted: Sunday 03 December 2023 14:49
by EscApe
rensbr wrote: ↑Sunday 03 December 2023 10:19
EscApe wrote: ↑Thursday 30 November 2023 17:19 Nope, no (noticeable) memory leaks on my system.
...
Is there something I can do to look what happens or where the problem may be?
I must revise my first answer. After setting up another test environment I DO see a memory leak. My other system wasn't even using ssh as a poll method, so that was ... uh ... well .... uh .... never mind :oops:

I will have to investigate, but I'm not sure when I will have the time. Maybe other forum members can help find the cause (probably in the use of pssh)?
Looks like the beta is not yet ready for daily use with a ssh based tracker.

Re: Python plugin: Presence detection from wireless router

Posted: Sunday 03 December 2023 15:59
by rensbr
EscApe wrote: ↑Sunday 03 December 2023 14:49
rensbr wrote: ↑Sunday 03 December 2023 10:19
EscApe wrote: ↑Thursday 30 November 2023 17:19 Nope, no (noticeable) memory leaks on my system.
...
Is there something I can do to look what happens or where the problem may be?
I must revise my first answer. After setting up another test environment I DO see a memory leak. My other system wasn't even using ssh as a poll method, so that was ... uh ... well .... uh .... never mind :oops:
No worries. Thanks for the support so far. I think it is good that you now having the problem to. Makes reproducing easier.
I will have to investigate, but I'm not sure when I will have the time. Maybe other forum members can help find the cause (probably in the use of pssh)?
Looks like the beta is not yet ready for daily use with a ssh based tracker.
Is there something I can do? I'm a PLC programmer, so I know some code. But i have no knowledge of phyton or something.

Re: Python plugin: Presence detection from wireless router

Posted: Saturday 09 December 2023 0:20
by rensbr
Maybe I found a right direction to look. When I look in the log I see that the on/off status is triggered every minute (the check is set to 60s)

I can imagine this is not entirely how it should work and maybe cause the problem. See below:
Screenshot_20231209-001839.png
Screenshot_20231209-001839.png (256.15 KiB) Viewed 10521 times

Re: Python plugin: Presence detection from wireless router

Posted: Saturday 09 December 2023 23:18
by EscApe
Thanks for your help in diagnosing this issue.
What grace period did you set? If it is also 60 seconds or less then this is normal behavior. The presence will be set to off after the grace period unless presence has been confirmed by a positive poll. So poll interval must be shorter than the grace period.
All times are UTC+02:00
Page 51 of 56

Powered by phpBB® Forum Software © phpBB Limited