Importing package eggs failing

[moroen]

Trying to import python packages in .egg format fails, even if the packages are installed to the dist-packages-folder using sudo.

I have a custom python package installed as an egg, using setuptools, it's stored as an .egg in the dist-packages directory:

Code: Select all

$ ls /usr/local/lib/python3.7/dist-packages/
pycoap-0.2.0-py3.7-linux-x86_64.egg

Using the system installed python (python 3.7, Ubuntu 19.04) the package imports without problems, trying to import it from a python plugin fails:

Code: Select all

2019-09-24 21:44:36.200  Error: (IKEA-Tradfri) failed to load 'plugin.py', Python Path used was '/home/mak/Projects/Domoticz/domoticz/plugins/IKEA-Tradfri/:/usr/lib/python37.zip:/usr/lib/python3.7:/usr/lib/python3.7/lib-dynload:/usr/local/lib/python3.7/dist-packages:/usr/lib/python3/dist-packages:/usr/lib/python3.7/dist-packages'.
2019-09-24 21:44:36.200  Error: (IKEA) Module Import failed, exception: 'ModuleNotFoundError'
2019-09-24 21:44:36.200  Error: (IKEA) Module Import failed: ' Name: pycoap'

According to the setuptools documentation, the *.egg directories need to be in the sys.path in order to be importable:

The layout of the code and resources is dictated by Python’s normal import layout, relative to the egg’s “base location”.

For the .egg format, the base location is the .egg itself. That is, adding the .egg filename or directory name to sys.path makes its contents importable.

For the .egg-info format, however, the base location is the directory that contains the .egg-info, and thus it is the directory that must be added to sys.path to make the egg importable. (Note that this means that the “normal” installation of a package to a sys.path directory is sufficient to make it an “egg” if it has an .egg-info file or directory installed alongside of it.)

Comparing the sys.path using the system python interpreter and the framework shows that all my installed .eggs are part of the sys.path using the system python interpreter, but from the plugin only the dist-packages folder is present:

Framework:

Code: Select all

2019-09-24 21:52:04.494  (IKEA) ['/home/mak/Projects/Domoticz/domoticz/plugins/IKEA-Tradfri/', '/usr/lib/python37.zip', '/usr/lib/python3.7', '/usr/lib/python3.7/lib-dynload', '/usr/local/lib/python3.7/dist-packages', '/usr/lib/python3/dist-packages', '/usr/lib/python3.7/dist-packages']

CLI:

Code: Select all

['/home/mak/Projects/Domoticz/IKEA-Tradfri-plugin-V2', '/usr/lib/python37.zip', '/usr/lib/python3.7', '/usr/lib/python3.7/lib-dynload', '/home/mak/.local/lib/python3.7/site-packages', '/usr/local/lib/python3.7/dist-packages', '/usr/local/lib/python3.7/dist-packages/IKEA_Tradfri_plugin-0.0.1-py3.7.egg', '/usr/local/lib/python3.7/dist-packages/pytradfri-6.1.0-py3.7.egg', '/usr/local/lib/python3.7/dist-packages/appdirs-1.4.3-py3.7.egg', '/usr/local/lib/python3.7/dist-packages/aiohttp-4.0.0a0-py3.7-linux-x86_64.egg', '/usr/local/lib/python3.7/dist-packages/aiocoap-0.4a1-py3.7.egg', '/usr/local/lib/python3.7/dist-packages/DTLSSocket-0.1.9-py3.7-linux-x86_64.egg', '/usr/local/lib/python3.7/dist-packages/yarl-1.3.0-py3.7-linux-x86_64.egg', '/usr/local/lib/python3.7/dist-packages/multidict-4.5.2-py3.7-linux-x86_64.egg', '/usr/local/lib/python3.7/dist-packages/attrs-19.1.0-py3.7.egg', '/usr/local/lib/python3.7/dist-packages/async_timeout-3.0.1-py3.7.egg', '/home/mak/Projects/Domoticz/IKEA-Tradfri-plugin/.eggs/Cython-0.29.13-py3.7-linux-x86_64.egg', '/usr/local/lib/python3.7/dist-packages/pycoap-0.2.0-py3.7-linux-x86_64.egg', '/usr/lib/python3/dist-packages']

If I convert the .egg-package to a single module, and place the module file in for example /usr/local/lib/python3.7/dist-packages, the file imports without problems. I have some ideas on how to work around this, but I guess this should be handled by the plugin-framework, akin to the regular python interpreter? Or am I missing something obvious here?

Regards,
M

[moroen]

Turns out that importing the standard library site module and running site.main() appends the missing eggs to sys.path, and thus enables importing .eggs without further shenanigans:

Code: Select all

import Domoticz
import site, sys
site.main()
import ...

The site module is automatically imported and executed when running the system python interpreter, I guess it should be automatically imported and run from the framework interpreter as well?

Regards,
M