Python plugin: Presence detection from wireless router

[Kubra]

Any maintancance (update) jobs on the Fritzbox at 22:00? Maybe some timed blocking rules?

Hi EscApe,
Thanks for the reply. I had been looking for tasks/blocking rules running around that time (it's GMT btw) but couldn't find any. However I now disabled "AVM Services" > "Searching for updates" and "Sending Diagnostic data" (can't even remember when I switched that on). Let's see how that works out.

[hjzwiers]

Is there any planning for the beta to be released?

[EscApe]

Is there any planning for the beta to be released?

I have been testing it for a while because it looked like something was causing a memory leak. However, on my setup the Domoticz memory usage indeed rises the first few days but then stabilizes. I don't know if there is any relation with the plugin itself, the plugin framework, domoticz or anything else. Could just have been a coincidence.
Any other results out there?

The release or beta status is a bit arbitrary in this case. It is not like the beta will be unsupported in a production environment..... there is no guaranteed supported for either of them and we are not running domoticz to control nuclear power plants

I can make it the new 'master' if no-one objects. It is a breaking change for existing setups, since you will have te recreate the devices.

Will look into making it the master this weekend.

[rensbr]

Is there any planning for the beta to be released?

we are not running domoticz to control nuclear power plants

My wife and kids think otherwise, when it does not work, it's 'rubbish' and they don't want to have it in their house. So like a nuclair plant, it has to work all the time, or we get a meltdown.

[EscApe]

Is there any planning for the beta to be released?

we are not running domoticz to control nuclear power plants

My wife and kids think otherwise, when it does not work, it's 'rubbish' and they don't want to have it in their house. So like a nuclair plant, it has to work all the time, or we get a meltdown.

I think you are onto something … this might even be much more dangerous

[hjzwiers]

I changed my router to Asus XT8. I'm still running into issues using type=aimesh_json.

It used to be fool-proof, not so much anymore. Thats why I was wondering about the Beta

[EscApe]

I changed my router to Asus XT8. I'm still running into issues using type=aimesh_json.

It used to be fool-proof, not so much anymore. Thats why I was wondering about the Beta

The beta uses the newer DomoticzEx framework to manage the Domoticz devices. There have also been some structural changes, but the polling methods haven't changed. I have tried to move to another ssh client, but in the end reverted to back to paramiko, because the other client introduced other problems.
So, if you have trouble getting data from you router you will probably still have it using the 'beta' version.

I was unable to diagnose your problem earlier, based on the issues you were experiencing and the logging you provided. You can of course try the beta to see if it makes a difference, but I'm afraid I cannot offer any more help on your peculiar case. It was and probably is too contradictory and complicated for me to diagnose remotely.

[hjzwiers]

What are the optimal settings for my router? The Asus xt8

[hjzwiers]

Error: IDetect: Call to function 'onStart' failed, exception details:
2024-07-02 15:30:30.966 Error: IDetect: Traceback (most recent call last):
2024-07-02 15:30:30.966 Error: IDetect: File "/home/hjz/domoticz/plugins/iDetect/plugin.py", line 431, in onStart
2024-07-02 15:30:30.966 Error: IDetect: _plugin.onStart()
2024-07-02 15:30:30.966 Error: IDetect: File "/home/hjz/domoticz/plugins/iDetect/plugin.py", line 262, in onStart
2024-07-02 15:30:30.966 Error: IDetect: if get_domoticz_status(self.OVERRIDE_UNIT):
2024-07-02 15:30:30.966 Error: IDetect: File "/home/hjz/domoticz/plugins/iDetect/plugin.py", line 181, in get_domoticz_status
2024-07-02 15:30:30.966 Error: IDetect: if Devices[unit].nValue == True:
2024-07-02 15:30:30.966 Error: IDetect: KeyError: 255

This is the result from a fresh re-install using 192.168.2.1#type=aimesh_json as the call to start iDetect

Update: deleted everything in the iDetect folder, reinstalled the Beta without the logging on in the iDetect sheet, all ok again with 192.168.2.1#type=aimesh_json I don't get it ........ This is how the install must go, but why then the installs with the errors?

[EscApe]

Update: deleted everything in the iDetect folder, reinstalled the Beta without the logging on in the iDetect sheet, all ok again with 192.168.2.1#type=aimesh_json I don't get it ........ This is how the install must go, but why then the installs with the errors?

If you switch back and forth between the master and beta branch then you MUST delete all devices that te plugin has created, including the override switch and anyone home. The DomticzEx framework used by the beta addresses the devices differently, so if you do not delete them there will be a mismatch when you change versions. After deleting the devices you can restart and the devices will be (re)created the richt way.

[hjzwiers]

Thanks, I didn't do that.

I deleted all devices iDetect installs, restarted, installed the Beta, restarted and all was installed again. I wonder how it goes!

[hjzwiers]

I did a clean install with the beta version of iDetect.

I runs fine over several days, then all of a sudden

Error: IDetect: 192.168.2.1 ====> SSH failed with exception:
IDetect: 192.168.2.1 ====> SSH Could not connect (using password). Exception: Error reading SSH protocol bannertimed out

From the router I get a message that the password connection is succes

I can reset the router and restart Domoticz:

iDetect: 192.168.2.1 ====> SSH Could not connect (using password). Exception: Error reading SSH protocol bannertimed out

I can log into the router with: sudo ssh [email protected]

I added the password to the tracker: 192.168.2.1#type=aimesh_json&user=HJZ&password=xxxxxxxx same result

Then if I delete the iDetect directory and reinstall the tracker, it works for a few days ....

[EscApe]

I already spent many hours trying to reproduce your issues back in januari. Unfortunately I could not, so I am unable to offer any new insights. Maybe someone with a comparable setup can help?

Btw: i would not consider replacing the plugin directory a “clean install”. The files in the plug-in directory do not change while using it. All status and configuration data is in the Domoticz database. If you want to try a really clean setup you could instal Linux on an empty drive with a fresh Domoticz install without the old database. Nothing from backups or existing configurations. I don’t mean you should immediately reinstall your entire setup, but this would be the correct way to create a suitable test machine.

[Gingerpale]

Can anybody help me with the first step, I get errors when installing paramiko with 'sudo pip3 install requests paramiko'.
I am running Python 3.7.3 and Domoticz 2025.1_build16576 on Rpi4. After multiple tries, I still get this error:

Code: Select all

Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple
Requirement already satisfied: requests in /usr/lib/python3/dist-packages (2.21.0)
Collecting paramiko
  Using cached https://files.pythonhosted.org/packages/15/f8/c7bd0ef12954a81a1d3cea60a13946bd9a49a0036a5927770c461eade7ae/paramiko-3.5.1-py3-none-any.whl
Collecting pynacl>=1.5 (from paramiko)
  Using cached https://www.piwheels.org/simple/pynacl/PyNaCl-1.5.0-cp37-cp37m-linux_armv7l.whl
Collecting bcrypt>=3.2 (from paramiko)
  Using cached https://files.pythonhosted.org/packages/56/8c/dd696962612e4cd83c40a9e6b3db77bfe65a830f4b9af44098708584686c/bcrypt-4.2.1.tar.gz
Exception:
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/pip/_internal/cli/base_command.py", line 143, in main
    status = self.run(options, args)
  File "/usr/lib/python3/dist-packages/pip/_internal/commands/install.py", line 338, in run
    resolver.resolve(requirement_set)
  File "/usr/lib/python3/dist-packages/pip/_internal/resolve.py", line 102, in resolve
    self._resolve_one(requirement_set, req)
  File "/usr/lib/python3/dist-packages/pip/_internal/resolve.py", line 256, in _resolve_one
    abstract_dist = self._get_abstract_dist_for(req_to_install)
  File "/usr/lib/python3/dist-packages/pip/_internal/resolve.py", line 209, in _get_abstract_dist_for
    self.require_hashes
  File "/usr/lib/python3/dist-packages/pip/_internal/operations/prepare.py", line 298, in prepare_linked_requirement
    abstract_dist.prep_for_dist(finder, self.build_isolation)
  File "/usr/lib/python3/dist-packages/pip/_internal/operations/prepare.py", line 100, in prep_for_dist
    self.req.load_pyproject_toml()
  File "/usr/lib/python3/dist-packages/pip/_internal/req/req_install.py", line 428, in load_pyproject_toml
    str(self)
  File "/usr/lib/python3/dist-packages/pip/_internal/pyproject.py", line 43, in load_pyproject_toml
    pp_toml = pytoml.load(f)
  File "/usr/share/python-wheels/pytoml-0.1.2-py2.py3-none-any.whl/pytoml/parser.py", line 303, in load
    filename=fin.name)
  File "/usr/share/python-wheels/pytoml-0.1.2-py2.py3-none-any.whl/pytoml/parser.py", line 370, in loads
    toks.expect('=', 'expected_equals')
  File "/usr/share/python-wheels/pytoml-0.1.2-py2.py3-none-any.whl/pytoml/parser.py", line 250, in expect
    self.error(error_text)
  File "/usr/share/python-wheels/pytoml-0.1.2-py2.py3-none-any.whl/pytoml/parser.py", line 253, in error
    raise TomlError(message, self.pos[0][0], self.pos[0][1], self._filename)
pytoml.core.TomlError: /tmp/pip-install-k7fp3wu9/bcrypt/pyproject.toml(62, 5): expected_equals

[EscApe]

Can anybody help me with the first step, I get errors when installing paramiko with 'sudo pip3 install requests paramiko'.
I am running Python 3.7.3 and Domoticz 2025.1_build16576 on Rpi4. After multiple tries, I still get this error:

Python 3.7 support ended in 2023. I would try a more recent version first.

[Gingerpale]

Almost there!
I installed Debian version 12 (bookworm) on a clean rpi4 with python3.11 pre-installed.
Created a venv and installed Paramiko, etc.
Git cloned iDetect.
I also installed Domoticz and enabled iDetect in Hardware.
Domoticz Hardware Settings for my ASUS RT-AC88U with SSH on port 2020 enabled:
192.168.1.1#port=2020
'username'
'psw'
phone1=12:34:56:78:90:AB,phone2=BA:09:87:65:43:21
yes
15
30
override until next real presence
on

It creates the switches but I still get the following message:

Code: Select all

Error: Presence detection: Tracker type is not available. Check if all required python modules for tracker type "default" are installed.

Any clue?

[EscApe]

It doesn't seem te find the required modules. Since you installed paramiko to a venv my first guess would be that Domoticz is not using the same venv. Otherwise the debug mode might give some more clues.

[Gingerpale]

Hi,
thx for your quick reply.
I've enabled debug in Domoticz Hardware iDetect settings but where can I see more debug output from that?
In Domoticz log I only get this one error line.

[EscApe]

Maybe the Domoticz log level setting is limiting it. Can you start Domoticz without a log-level setting?

The error message indicates that the plugin failed to load a required module for the tracker. You could try installing paramiko globally, just to rule out the venv as a possible cause.

Which branch of iDetect did you install? You could try the DomoticEx-based-beta. I have been running it for a while without problems. I didn't make it the master branch because it brings some braking changes for existing users (you have to remove existing iDetect devices from Domoiticz first).
It has better error handling and logging for this kind of issue.

[Gingerpale]

I finally got it to work by:
1. Removing the current iDetect switches, etc. in Domoticz
2. Installing the branch by: git checkout DomoticzEx-based-beta
3. sudo nano /etc/init.d/domoticz.sh and added a line: export PYTHONPATH=/home/pi/pyvenv/lib/python3.11/site-packages:$PYTHONPATH (where pyvenv is my venv for python)
4. reboot

The only problem I still had is that it only saw the devices directly connected to the main router ASUS RT-AC88U and not the devices connected to the mesh AP's ASUS RT-AC68U. I searched the forum and tried in tracker: '192.168.1.1#port2020&forcegeneric', and (only) after reboot that seemed to work. Is this the correct way to do it?
And one more observation: every time when changing and updating iDetect settings in Domoticz - Hardware it seems to yield an error: 'Error while trying to import tracker module:PyO3 modules do not yet support subinterpreters, see https://github.com/PyO3/pyo3/issues/576'
But there is no error seen after a reboot.
Many thx for your support.