[REQUEST] Ikea Trådfri Gateway (smart lights hub controller) plugin?

Python and python framework

Moderator: leecollings

moroen
Posts: 408
Joined: Sunday 15 January 2017 11:06
Target OS: Linux
Domoticz version: beta
Location: Norway
Contact:

Re: [REQUEST] Ikea Trådfri Gateway (smart lights hub controller) plugin?

Post by moroen »

wvdweerd wrote: Sunday 26 January 2020 11:29
moroen wrote: Thursday 23 January 2020 9:28 Hm, you might be able to find something if you enable writing the log to a file, but when domoticz dies, sometimes it's not even able to write to the log. You'll get the best info if you run domoticz from the command line. Is tradfri the only plugin you're running?
Just to be sure:
How do I run Domoticz from the command line? Is it as simple as: first stop the Domoticz service (sudo systemctl stop domoticz.service), and then start it again (sudo systemctl start domoticz.service) ?
And then I have to let my PC run with the PuTTY command window open until Domoticz dies?

Reagarding other pluguins: I am running:
  • P1 Slimme meter USB
  • SolarEdge
  • Aeon Labs Z-stick

I have tried disabling the SolarEdge plugin, because that was the last one I added.
But still Domoticz dies when running your Ikea Trafdri plugin.
You need to stop the service, and go to the domoticz directory and start domoticz:

Code: Select all

$ sudo systemctl stop domoticz.service
$ cd ~/domoticz/
$ ./domoticz
And then, as you said, keep PuTTY open until domoticz dies..

I have identified a few more places in the code where the plugin thread would just terminate, and bring domoticz down with it. I have fix for this, but it's part of a larger change that's not quite ready yet, I'll release it as soon as I can!

Regards,
M
moroen
Posts: 408
Joined: Sunday 15 January 2017 11:06
Target OS: Linux
Domoticz version: beta
Location: Norway
Contact:

Re: [REQUEST] Ikea Trådfri Gateway (smart lights hub controller) plugin?

Post by moroen »

NOTE: This has been updated to reflect some changes to the latest version on github

There's a new version available, version 0.8.1. Unfortunately this update has a couple of changes that need a bit more intervention when doing the upgrade:

Major changes:
- Pycoap has been moved to the official pypi index and been renamed to py3coap in the process. Unfortunatly there already is a module called pycoap in the official index, and thus the name had to be changed.
- tradfricoap.py as been removed, and the needed commands (config, api and list) have been implemented in plugin.py
- support for settting WS of GUNNARP panel light (untested)

Bug fixes (py3coap):
- Identified and hopefully fixed points in the code than could terminate the plugin thread and bring domoticz down with it. There probably will be errors in the log that the plugin can't handle, but that should allow both domoticz and the plugin to continue. If any errors are seen in the log, please post them here.

Work in progress:
- Implementing reading battery status of remotes and curtains/blinds

Upgrading to the latest version (when using pycoap):

Code: Select all

$ cd domoticz/plugins/IKEA-Tradfri
$ pip3 uninstall pycoap
$ git pull
$ pip3 install py3coap
$ python3 plugin.py list
Upgrading to the latest version (using coapcmd):

Code: Select all

$ cd domoticz/plugins/IKEA-Tradfri
$ git pull
$ python3 plugin.py api coapcmd
$ python3 plugin.py list
Regards,
M
Last edited by moroen on Monday 27 January 2020 15:42, edited 2 times in total.
dressie
Posts: 156
Joined: Monday 25 May 2015 22:44
Target OS: Raspberry Pi / ODroid
Domoticz version: V2020.2
Location: Netherlands
Contact:

Re: [REQUEST] Ikea Trådfri Gateway (smart lights hub controller) plugin?

Post by dressie »

Followed your update proces, this is what I get:

10:05:14.745 Error: (IKEA) Failed to initialize tradfri module.
2020-01-27 10:05:14.745 Error: (IKEA) Unable to find tradfricoap
2020-01-27 10:05:24.707 Error: (IKEA) Plugin not intialized!

Update broke it, so it seems.

UPDATE: During install:
python3: can't open file 'tradfricoap.py': [Errno 2] No such file or directory

Seems like this file isn't coming with it?
Last edited by dressie on Monday 27 January 2020 10:16, edited 1 time in total.
moroen
Posts: 408
Joined: Sunday 15 January 2017 11:06
Target OS: Linux
Domoticz version: beta
Location: Norway
Contact:

Re: [REQUEST] Ikea Trådfri Gateway (smart lights hub controller) plugin?

Post by moroen »

dressie wrote: Monday 27 January 2020 10:06 Followed your update proces, this is what I get:

10:05:14.745 Error: (IKEA) Failed to initialize tradfri module.
2020-01-27 10:05:14.745 Error: (IKEA) Unable to find tradfricoap
2020-01-27 10:05:24.707 Error: (IKEA) Plugin not intialized!

Update broke it, so it seems.
Ok...

I've been told that this change will also be a problem on a NAS, so I'll revert the decision to split it out as a separate module, I'll release a corrected version as soon as possible!

In the meantime, just revert to the previous version, and you should be fine!

Code: Select all

$ cd domoticz/plugins/IKEA-Tradfri
$ git reset --hard ba0b245aec5b6085662f203da3fc985427052732
$ pip3 uninstall py3coap
$ pip3 install -r requirements.txt
Reagards,
M
moroen
Posts: 408
Joined: Sunday 15 January 2017 11:06
Target OS: Linux
Domoticz version: beta
Location: Norway
Contact:

Re: [REQUEST] Ikea Trådfri Gateway (smart lights hub controller) plugin?

Post by moroen »

dressie wrote: Monday 27 January 2020 10:06 UPDATE: During install:
python3: can't open file 'tradfricoap.py': [Errno 2] No such file or directory

Seems like this file isn't coming with it?
Where in the process did you get this?

Regards,
M
dressie
Posts: 156
Joined: Monday 25 May 2015 22:44
Target OS: Raspberry Pi / ODroid
Domoticz version: V2020.2
Location: Netherlands
Contact:

Re: [REQUEST] Ikea Trådfri Gateway (smart lights hub controller) plugin?

Post by dressie »

moroen wrote: Monday 27 January 2020 10:19
dressie wrote: Monday 27 January 2020 10:06 UPDATE: During install:
python3: can't open file 'tradfricoap.py': [Errno 2] No such file or directory

Seems like this file isn't coming with it?
Where in the process did you get this?

Regards,
M
Here:
pi@DomoticzV2:~/domoticz/plugins/IKEA-Tradfri $ sudo python3 tradfricoap.py api coapcmd
python3: can't open file 'tradfricoap.py': [Errno 2] No such file or directory

P.S. I use the coapcmd option after our PM a while ago
moroen
Posts: 408
Joined: Sunday 15 January 2017 11:06
Target OS: Linux
Domoticz version: beta
Location: Norway
Contact:

Re: [REQUEST] Ikea Trådfri Gateway (smart lights hub controller) plugin?

Post by moroen »

dressie wrote: Monday 27 January 2020 11:04
moroen wrote: Monday 27 January 2020 10:19
dressie wrote: Monday 27 January 2020 10:06 UPDATE: During install:
python3: can't open file 'tradfricoap.py': [Errno 2] No such file or directory

Seems like this file isn't coming with it?
Where in the process did you get this?

Regards,
M
Here:
pi@DomoticzV2:~/domoticz/plugins/IKEA-Tradfri $ sudo python3 tradfricoap.py api coapcmd
python3: can't open file 'tradfricoap.py': [Errno 2] No such file or directory
The tradfricoap.py command has been removed, and the config, list and api commands are now a part of plugin.py, and the following should do the trick:

Code: Select all

$ python3 plugin.py api coapcmd
dressie wrote: Monday 27 January 2020 11:04 P.S. I use the coapcmd option after our PM a while ago
Yes, I remember now, you couldn't use pycoap due to somekind of python environment hassle, and that explains why pip3 install tradfricoap didn't work.

Anyway, I'll find another way to create a tradfri command line utility, and separating out tradfricoap isn't worth the hassle for users with non-default python environments, like your problems and on Synology NAS, so I have updated the plugin to version 0.8.1, where tradfricoap is included, it's available on github now. You still need to substitute tradfricoap.py with plugin.py though... :)

Regards,
M
dressie
Posts: 156
Joined: Monday 25 May 2015 22:44
Target OS: Raspberry Pi / ODroid
Domoticz version: V2020.2
Location: Netherlands
Contact:

Re: [REQUEST] Ikea Trådfri Gateway (smart lights hub controller) plugin?

Post by dressie »

Yes, with the 0.8.1 and update like your instruction for the coapcmd it work again! Thanks for the quick fix! 8-)

P.S. on my test Domoticz on Synology it's working too!
wvdweerd
Posts: 22
Joined: Saturday 20 October 2018 18:53
Target OS: Raspberry Pi / ODroid
Domoticz version: 4.10717
Location: The Netherlands
Contact:

Re: [REQUEST] Ikea Trådfri Gateway (smart lights hub controller) plugin?

Post by wvdweerd »

moroen wrote: Monday 27 January 2020 9:26 NOTE: This has been updated to reflect some changes to the latest version on github

There's a new version available, version 0.8.1. Unfortunately this update has a couple of changes that need a bit more intervention when doing the upgrade:

Major changes:
- Pycoap has been moved to the official pypi index and been renamed to py3coap in the process. Unfortunatly there already is a module called pycoap in the official index, and thus the name had to be changed.
- tradfricoap.py as been removed, and the needed commands (config, api and list) have been implemented in plugin.py
- support for settting WS of GUNNARP panel light (untested)

Bug fixes (py3coap):
- Identified and hopefully fixed points in the code than could terminate the plugin thread and bring domoticz down with it. There probably will be errors in the log that the plugin can't handle, but that should allow both domoticz and the plugin to continue. If any errors are seen in the log, please post them here.

Work in progress:
- Implementing reading battery status of remotes and curtains/blinds

Regards,
M
Thanks for the update and very clear update instructions :)
I just upgraded and enabled the plugin again, I'll post my findings soon. Fingers crossed :P

Battery status of remotes would be a very welcome addition!
I often miss the subtle orange blinking light on the remote, so I then get stuck with a dead remote - which I then need to reconnect with the hub and lights again :cry:
User avatar
bartplessers
Posts: 12
Joined: Sunday 28 July 2019 22:21
Target OS: Raspberry Pi / ODroid
Domoticz version:
Contact:

Re: [REQUEST] Ikea Trådfri Gateway (smart lights hub controller) plugin?

Post by bartplessers »

Hi

I have
- IKEA Tradfri gateway
- DOMOTICZ v4.10717 with IKEA Tradfri Plugin - version 0.7.3
- HOME ASSISTANT (hassio in docker) v0.104.3, with IKEA Tradfri integration

I noticed that I can NOT connect both addons to my IKEA hub.
It works... sometimes, but I have the feeling that the Domoticz plugin "locks" the connection so that hassio can not control my devices anymore.

Hass.io is responsive and shows changes in status immediatly.
But if I enable the Domoticz Tradfri plugin in domoticz, things are working well for a short while. After that, hass.io doesn't show any status changes.

In domoticz, I tried the different options for "observe changes" in the plugin settings, but no solution.

Does anybody has the same experience?
Is this a bug?

Kind regards,
Bart
moroen
Posts: 408
Joined: Sunday 15 January 2017 11:06
Target OS: Linux
Domoticz version: beta
Location: Norway
Contact:

Re: [REQUEST] Ikea Trådfri Gateway (smart lights hub controller) plugin?

Post by moroen »

bartplessers wrote: Tuesday 28 January 2020 17:44 Hi

I have
- IKEA Tradfri gateway
- DOMOTICZ v4.10717 with IKEA Tradfri Plugin - version 0.7.3
- HOME ASSISTANT (hassio in docker) v0.104.3, with IKEA Tradfri integration

I noticed that I can NOT connect both addons to my IKEA hub.
It works... sometimes, but I have the feeling that the Domoticz plugin "locks" the connection so that hassio can not control my devices anymore.

Hass.io is responsive and shows changes in status immediatly.
But if I enable the Domoticz Tradfri plugin in domoticz, things are working well for a short while. After that, hass.io doesn't show any status changes.

In domoticz, I tried the different options for "observe changes" in the plugin settings, but no solution.

Does anybody has the same experience?
Is this a bug?

Kind regards,
Bart
What happens if you shut down domoticz without restarting HA? Does HA then start to respond again, or do you need to restart HA in order to get it to work?

Regards,
M
wvdweerd
Posts: 22
Joined: Saturday 20 October 2018 18:53
Target OS: Raspberry Pi / ODroid
Domoticz version: 4.10717
Location: The Netherlands
Contact:

Re: [REQUEST] Ikea Trådfri Gateway (smart lights hub controller) plugin?

Post by wvdweerd »

wvdweerd wrote: Monday 27 January 2020 22:34 I just upgraded and enabled the plugin again, I'll post my findings soon. Fingers crossed :P
So far so good: it is running for more than 24 hours now, and no (none, zero, nada :mrgreen: ) errors in the Domoticz log! :P
And of course Domoticz is also still running fine :)

Regards,
Wouter
User avatar
bartplessers
Posts: 12
Joined: Sunday 28 July 2019 22:21
Target OS: Raspberry Pi / ODroid
Domoticz version:
Contact:

Re: [REQUEST] Ikea Trådfri Gateway (smart lights hub controller) plugin?

Post by bartplessers »

moroen wrote: Tuesday 28 January 2020 21:00

What happens if you shut down domoticz without restarting HA? Does HA then start to respond again, or do you need to restart HA in order to get it to work?

Regards,
M
Hi,

meanwhile I upgraded to IKEA Tradfri Plugin - version 0.8.1

STATUS 1 (initial)
------------------------------------------------------
- Domoticz + IKEA plugin installed and hardware added, but disabled
- Hassio + IKEA integration

==>
Hassio is working, lights are responding to hassio


STATUS 2
------------------------------------------------------
- I enabled IKEA plugin in Domotics
- plugin setting: observe changes = disabled

==>
Domoticz can turn on/of IKEA lights
Hassio can not turn IKEA lights on/off anymore
after some time, lights are working again in hassio (and also in domoticz)



STATUS 3
------------------------------------------------------
- restart hassio

==>
Domoticz can turn IKEA lights on/off
Hassio can turn IKEA lights on/off


STATUS 4
------------------------------------------------------
- plugin setting: observe changes = enabled

==>
Domoticz can turn IKEA lights on/off
Hassio can NOT turn IKEA lights on/off


STATUS 5
------------------------------------------------------
- restart hassio

==>
Domoticz can turn IKEA lights on/off
Hassio can turn IKEA lights on/off JUST ONCE. After that, hassio lights become unresponsive again.



STATUS 6
------------------------------------------------------
- plugin setting: observe changes = disabled

==>
Domoticz can turn IKEA lights on/off. Response is sometimes slow (subjective)
Hassio can turn IKEA lights on/off JUST ONCE. After that, hassio lights become unresponsive again.
After a while, Hassio is working fully again. No restart needed



So I don't have to restart domoticz, just disable the observe changes and wait a while.

Does this help?

Kind regards,
Bart
wvdweerd
Posts: 22
Joined: Saturday 20 October 2018 18:53
Target OS: Raspberry Pi / ODroid
Domoticz version: 4.10717
Location: The Netherlands
Contact:

Re: [REQUEST] Ikea Trådfri Gateway (smart lights hub controller) plugin?

Post by wvdweerd »

wvdweerd wrote: Tuesday 28 January 2020 22:43
wvdweerd wrote: Monday 27 January 2020 22:34 I just upgraded and enabled the plugin again, I'll post my findings soon. Fingers crossed :P
So far so good: it is running for more than 24 hours now, and no (none, zero, nada :mrgreen: ) errors in the Domoticz log! :P
And of course Domoticz is also still running fine :)

Regards,
Wouter
So far, not so good anymore... :cry:
Domoticz is still running, but the Domoticz webserver has died.

And since yesterday afternoon I see the following errors in the log:

Code: Select all

2020-01-29 16:07:50.901  Status: dzVents: Error (2.4.19): /home/pi/domoticz/dzVents/runtime/EventHelpers.lua:311: attempt to index local 'pfile' (a nil value)
2020-01-29 16:07:50.901  Status: dzVents: Error (2.4.19): /home/pi/domoticz/dzVents/runtime/EventHelpers.lua:311: attempt to index local 'pfile' (a nil value)
2020-01-29 16:08:00.418  Error: EventSystem: in /home/pi/domoticz/dzVents/runtime/dzVents.lua: cannot open /home/pi/domoticz/dzVents/runtime/dzVents.lua: Too many open files
2020-01-29 16:08:04.111  Error: (IKEA Tradfri) 'onHeartbeat' failed 'JSONDecodeError'.
2020-01-29 16:08:04.112  Error: (IKEA Tradfri) ----> Line 597 in '/home/pi/domoticz/plugins/IKEA-Tradfri/plugin.py', function onHeartbeat
2020-01-29 16:08:04.112  Error: (IKEA Tradfri) ----> Line 544 in '/home/pi/domoticz/plugins/IKEA-Tradfri/plugin.py', function onHeartbeat
2020-01-29 16:08:04.112  Error: (IKEA Tradfri) ----> Line 283 in '/home/pi/domoticz/plugins/IKEA-Tradfri/plugin.py', function registerDevices
2020-01-29 16:08:04.112  Error: (IKEA Tradfri) ----> Line 204 in '/home/pi/domoticz/plugins/IKEA-Tradfri/plugin.py', function indexRegisteredDevices
2020-01-29 16:08:04.112  Error: (IKEA Tradfri) ----> Line 280 in '/home/pi/domoticz/plugins/IKEA-Tradfri/plugin.py', function updateDevice
2020-01-29 16:08:04.112  Error: (IKEA Tradfri) ----> Line 307 in '/home/pi/domoticz/plugins/IKEA-Tradfri/tradfricoap/device.py', function get_device
2020-01-29 16:08:04.112  Error: (IKEA Tradfri) ----> Line 45 in '/home/pi/domoticz/plugins/IKEA-Tradfri/tradfricoap/device.py', function __init__
2020-01-29 16:08:04.113  Error: (IKEA Tradfri) ----> Line 319 in '/usr/lib/python3.5/json/__init__.py', function loads
2020-01-29 16:08:04.113  Error: (IKEA Tradfri) ----> Line 339 in '/usr/lib/python3.5/json/decoder.py', function decode
2020-01-29 16:08:04.113  Error: (IKEA Tradfri) ----> Line 357 in '/usr/lib/python3.5/json/decoder.py', function raw_decode
After these initial errors the log is filling up with the same errors.

One thing I am curious about:
if I use

Code: Select all

 sudo lsof -p <pid of domoticz>
I see a lot of lines like:

Code: Select all

domoticz 1938 root 1023u     IPv6     897756      0t0        UDP *:35986
Really a lot: 982 lines to be exact.

Is the Tradfri plugin using these UDP connections to connect to the Ikea Hub?
And are these maybe not closed correctly after a connection time-out?

Regards,
Wouter
moroen
Posts: 408
Joined: Sunday 15 January 2017 11:06
Target OS: Linux
Domoticz version: beta
Location: Norway
Contact:

Re: [REQUEST] Ikea Trådfri Gateway (smart lights hub controller) plugin?

Post by moroen »

wvdweerd wrote: Thursday 30 January 2020 20:28
wvdweerd wrote: Tuesday 28 January 2020 22:43
wvdweerd wrote: Monday 27 January 2020 22:34 I just upgraded and enabled the plugin again, I'll post my findings soon. Fingers crossed :P
So far so good: it is running for more than 24 hours now, and no (none, zero, nada :mrgreen: ) errors in the Domoticz log! :P
And of course Domoticz is also still running fine :)

Regards,
Wouter
So far, not so good anymore... :cry:
Domoticz is still running, but the Domoticz webserver has died.

And since yesterday afternoon I see the following errors in the log:

Code: Select all

2020-01-29 16:07:50.901  Status: dzVents: Error (2.4.19): /home/pi/domoticz/dzVents/runtime/EventHelpers.lua:311: attempt to index local 'pfile' (a nil value)
2020-01-29 16:07:50.901  Status: dzVents: Error (2.4.19): /home/pi/domoticz/dzVents/runtime/EventHelpers.lua:311: attempt to index local 'pfile' (a nil value)
2020-01-29 16:08:00.418  Error: EventSystem: in /home/pi/domoticz/dzVents/runtime/dzVents.lua: cannot open /home/pi/domoticz/dzVents/runtime/dzVents.lua: Too many open files
2020-01-29 16:08:04.111  Error: (IKEA Tradfri) 'onHeartbeat' failed 'JSONDecodeError'.
2020-01-29 16:08:04.112  Error: (IKEA Tradfri) ----> Line 597 in '/home/pi/domoticz/plugins/IKEA-Tradfri/plugin.py', function onHeartbeat
2020-01-29 16:08:04.112  Error: (IKEA Tradfri) ----> Line 544 in '/home/pi/domoticz/plugins/IKEA-Tradfri/plugin.py', function onHeartbeat
2020-01-29 16:08:04.112  Error: (IKEA Tradfri) ----> Line 283 in '/home/pi/domoticz/plugins/IKEA-Tradfri/plugin.py', function registerDevices
2020-01-29 16:08:04.112  Error: (IKEA Tradfri) ----> Line 204 in '/home/pi/domoticz/plugins/IKEA-Tradfri/plugin.py', function indexRegisteredDevices
2020-01-29 16:08:04.112  Error: (IKEA Tradfri) ----> Line 280 in '/home/pi/domoticz/plugins/IKEA-Tradfri/plugin.py', function updateDevice
2020-01-29 16:08:04.112  Error: (IKEA Tradfri) ----> Line 307 in '/home/pi/domoticz/plugins/IKEA-Tradfri/tradfricoap/device.py', function get_device
2020-01-29 16:08:04.112  Error: (IKEA Tradfri) ----> Line 45 in '/home/pi/domoticz/plugins/IKEA-Tradfri/tradfricoap/device.py', function __init__
2020-01-29 16:08:04.113  Error: (IKEA Tradfri) ----> Line 319 in '/usr/lib/python3.5/json/__init__.py', function loads
2020-01-29 16:08:04.113  Error: (IKEA Tradfri) ----> Line 339 in '/usr/lib/python3.5/json/decoder.py', function decode
2020-01-29 16:08:04.113  Error: (IKEA Tradfri) ----> Line 357 in '/usr/lib/python3.5/json/decoder.py', function raw_decode
After these initial errors the log is filling up with the same errors.

One thing I am curious about:
if I use

Code: Select all

 sudo lsof -p <pid of domoticz>
I see a lot of lines like:

Code: Select all

domoticz 1938 root 1023u     IPv6     897756      0t0        UDP *:35986
Really a lot: 982 lines to be exact.

Is the Tradfri plugin using these UDP connections to connect to the Ikea Hub?
And are these maybe not closed correctly after a connection time-out?

Regards,
Wouter
The error message shows that the data returned from the gateway is not what's expected, probably some error that still isn't handled properly. I'll check this further. Regarding the number of UDP connections, I think you might be right that connections are not closed correctly after timeouts, and I have a pretty good idea why, I'll take a look as soon as possible!

I still ponder why you have so many timeouts? If you right and that the number of UDP-connections are somehow connected to timeouts, that's a lot of timeouts in a couple of days? The time the plugin waits before timing out is a balance between responsivness of the whole plugin thread and network latency, but it might be worth it to make it a bit longer?

I'll get back to you as soon as possible!

Regards,
M
moroen
Posts: 408
Joined: Sunday 15 January 2017 11:06
Target OS: Linux
Domoticz version: beta
Location: Norway
Contact:

Re: [REQUEST] Ikea Trådfri Gateway (smart lights hub controller) plugin?

Post by moroen »

moroen wrote: Friday 31 January 2020 9:30
wvdweerd wrote: Thursday 30 January 2020 20:28
wvdweerd wrote: Tuesday 28 January 2020 22:43

So far so good: it is running for more than 24 hours now, and no (none, zero, nada :mrgreen: ) errors in the Domoticz log! :P
And of course Domoticz is also still running fine :)

Regards,
Wouter
So far, not so good anymore... :cry:
Domoticz is still running, but the Domoticz webserver has died.

And since yesterday afternoon I see the following errors in the log:

Code: Select all

2020-01-29 16:07:50.901  Status: dzVents: Error (2.4.19): /home/pi/domoticz/dzVents/runtime/EventHelpers.lua:311: attempt to index local 'pfile' (a nil value)
2020-01-29 16:07:50.901  Status: dzVents: Error (2.4.19): /home/pi/domoticz/dzVents/runtime/EventHelpers.lua:311: attempt to index local 'pfile' (a nil value)
2020-01-29 16:08:00.418  Error: EventSystem: in /home/pi/domoticz/dzVents/runtime/dzVents.lua: cannot open /home/pi/domoticz/dzVents/runtime/dzVents.lua: Too many open files
2020-01-29 16:08:04.111  Error: (IKEA Tradfri) 'onHeartbeat' failed 'JSONDecodeError'.
2020-01-29 16:08:04.112  Error: (IKEA Tradfri) ----> Line 597 in '/home/pi/domoticz/plugins/IKEA-Tradfri/plugin.py', function onHeartbeat
2020-01-29 16:08:04.112  Error: (IKEA Tradfri) ----> Line 544 in '/home/pi/domoticz/plugins/IKEA-Tradfri/plugin.py', function onHeartbeat
2020-01-29 16:08:04.112  Error: (IKEA Tradfri) ----> Line 283 in '/home/pi/domoticz/plugins/IKEA-Tradfri/plugin.py', function registerDevices
2020-01-29 16:08:04.112  Error: (IKEA Tradfri) ----> Line 204 in '/home/pi/domoticz/plugins/IKEA-Tradfri/plugin.py', function indexRegisteredDevices
2020-01-29 16:08:04.112  Error: (IKEA Tradfri) ----> Line 280 in '/home/pi/domoticz/plugins/IKEA-Tradfri/plugin.py', function updateDevice
2020-01-29 16:08:04.112  Error: (IKEA Tradfri) ----> Line 307 in '/home/pi/domoticz/plugins/IKEA-Tradfri/tradfricoap/device.py', function get_device
2020-01-29 16:08:04.112  Error: (IKEA Tradfri) ----> Line 45 in '/home/pi/domoticz/plugins/IKEA-Tradfri/tradfricoap/device.py', function __init__
2020-01-29 16:08:04.113  Error: (IKEA Tradfri) ----> Line 319 in '/usr/lib/python3.5/json/__init__.py', function loads
2020-01-29 16:08:04.113  Error: (IKEA Tradfri) ----> Line 339 in '/usr/lib/python3.5/json/decoder.py', function decode
2020-01-29 16:08:04.113  Error: (IKEA Tradfri) ----> Line 357 in '/usr/lib/python3.5/json/decoder.py', function raw_decode
After these initial errors the log is filling up with the same errors.

One thing I am curious about:
if I use

Code: Select all

 sudo lsof -p <pid of domoticz>
I see a lot of lines like:

Code: Select all

domoticz 1938 root 1023u     IPv6     897756      0t0        UDP *:35986
Really a lot: 982 lines to be exact.

Is the Tradfri plugin using these UDP connections to connect to the Ikea Hub?
And are these maybe not closed correctly after a connection time-out?

Regards,
Wouter
The error message shows that the data returned from the gateway is not what's expected, probably some error that still isn't handled properly. I'll check this further. Regarding the number of UDP connections, I think you might be right that connections are not closed correctly after timeouts, and I have a pretty good idea why, I'll take a look as soon as possible!

I still ponder why you have so many timeouts? If you right and that the number of UDP-connections are somehow connected to timeouts, that's a lot of timeouts in a couple of days? The time the plugin waits before timing out is a balance between responsivness of the whole plugin thread and network latency, but it might be worth it to make it a bit longer?

I'll get back to you as soon as possible!

Regards,
M
I haven't been able to do any testing, but the lastest version of pycoap (0.7.3) available via pypi should explicitly close the UDP connection on timeouts. Upgrade pycoap with pip, no need to update the plugin itself.

Regards,
M
wvdweerd
Posts: 22
Joined: Saturday 20 October 2018 18:53
Target OS: Raspberry Pi / ODroid
Domoticz version: 4.10717
Location: The Netherlands
Contact:

Re: [REQUEST] Ikea Trådfri Gateway (smart lights hub controller) plugin?

Post by wvdweerd »

moroen wrote: Friday 31 January 2020 12:20
moroen wrote: Friday 31 January 2020 9:30 The error message shows that the data returned from the gateway is not what's expected, probably some error that still isn't handled properly. I'll check this further. Regarding the number of UDP connections, I think you might be right that connections are not closed correctly after timeouts, and I have a pretty good idea why, I'll take a look as soon as possible!

I still ponder why you have so many timeouts? If you right and that the number of UDP-connections are somehow connected to timeouts, that's a lot of timeouts in a couple of days? The time the plugin waits before timing out is a balance between responsivness of the whole plugin thread and network latency, but it might be worth it to make it a bit longer?

I'll get back to you as soon as possible!

Regards,
M
I haven't been able to do any testing, but the lastest version of pycoap (0.7.3) available via pypi should explicitly close the UDP connection on timeouts. Upgrade pycoap with pip, no need to update the plugin itself.

Regards,
M
Thanks! I will try this now.

Probably the many time-outs are caused by my Wifi modem / internet provider router and connections. Everybody in our street is having these kinds of internet and wifi problems since our cable network provider Ziggo made a mess last November of our connection hubs in our street.

And the last Ikea update introduced some problems getting 2 of my led lights connected again with my Ikea hub, so that may also be a cause of this.
I just connected all lights succesfully again, and a Ziggo engineer has been working today on our street connection hub, so maybe less time-outs from now on?! :?

Regards, Wouter
moroen
Posts: 408
Joined: Sunday 15 January 2017 11:06
Target OS: Linux
Domoticz version: beta
Location: Norway
Contact:

Re: [REQUEST] Ikea Trådfri Gateway (smart lights hub controller) plugin?

Post by moroen »

moroen wrote: Friday 31 January 2020 12:20
I haven't been able to do any testing, but the lastest version of pycoap (0.7.3) available via pypi should explicitly close the UDP connection on timeouts. Upgrade pycoap with pip, no need to update the plugin itself.

Regards,
M
I've been able to conduct some tests, and it seems that timeouts indeed did leave open UDP-connections, one open connection for every timeout. It also seems that the latest update to py3coap (0.7.3) fixes this, and no longer leaves open UDP-connections. I would recommend everybody using py3coap to upgrade to the latest version:

Code: Select all

$ sudo -H pip3 install -U py3coap
Regards,
M
moroen
Posts: 408
Joined: Sunday 15 January 2017 11:06
Target OS: Linux
Domoticz version: beta
Location: Norway
Contact:

Re: [REQUEST] Ikea Trådfri Gateway (smart lights hub controller) plugin?

Post by moroen »

bartplessers wrote: Wednesday 29 January 2020 23:26
So I don't have to restart domoticz, just disable the observe changes and wait a while.

Does this help?

Kind regards,
Bart
I've not been able to recreate your problem with using both HASS and domoticz, but I do suspect that this might be related to timeouts leaving behind open UDP-connections, since you noticed that the plugin seemed to respond slowly before locking out HASS.

Please try updating to the latest version of py3coap, and see if that fixes this!

Regards,
M
User avatar
bartplessers
Posts: 12
Joined: Sunday 28 July 2019 22:21
Target OS: Raspberry Pi / ODroid
Domoticz version:
Contact:

Re: [REQUEST] Ikea Trådfri Gateway (smart lights hub controller) plugin?

Post by bartplessers »

moroen wrote: Friday 31 January 2020 22:08
bartplessers wrote: Wednesday 29 January 2020 23:26
So I don't have to restart domoticz, just disable the observe changes and wait a while.

Does this help?

Kind regards,
Bart
I've not been able to recreate your problem with using both HASS and domoticz, but I do suspect that this might be related to timeouts leaving behind open UDP-connections, since you noticed that the plugin seemed to respond slowly before locking out HASS.

Please try updating to the latest version of py3coap, and see if that fixes this!

Regards,
M


Hi Moroen,

- Before the tests, I upgraded already to IKEA Tradfri Plugin - version 0.8.1. This is the latest at this moment, not?
- is it expected behaviour that you can NOT use "observe changes" with other connections to the IKEA bridge? Than the plugin becomes quit useless for me. I want to be as much as independent from any software. Playing around now with node-red, home-assistant, domoticz, openhab, etc... If a state is changed somewhere, I want to have it reflected in all my tools. It's really a bummer if the plugin locks out other connections
- If you have not been able to recreate my problems: does it mean that you have a working setup with both hass and domoticz? Or you just haven't got the opportunity to test this combination?

I alsof forgot to mention: I have 4 instances of home-assistant running on different servers. The "IKEA integrations" on al these servers talk smoothly to the IKEA bridge, so at least it is possible to have multiple connections to the bridge...

Kind regards,
Bart
Post Reply

Who is online

Users browsing this forum: No registered users and 1 guest