Page 1 of 2

Debian and Raspbian packages now available

Posted: Tuesday 06 August 2019 13:11
by pocock
I've made packages available for Domoticz on Debian (amd64 and i386) and Raspbian (armhf). They are compiled for stretch, buster and sid. My blog gives full instructions.

https://danielpocock.com/domoticz-packa ... an-fedora/

The same repository includes packages for the domoticz-zigate-plugin, the JennicModuleProgrammer and zigate-flasher

Please share any feedback or observations. Pull requests through Gitlab are welcome.

Re: Debian and Raspbian packages now available

Posted: Tuesday 06 August 2019 13:18
by pipiche
Well done !

Re: Debian and Raspbian packages now available

Posted: Tuesday 06 August 2019 13:37
by Dnpwwo
@pocock,

Wow, this is a bit next level !

We need to cut stable versions a little more often to support this.

Re: Debian and Raspbian packages now available

Posted: Tuesday 06 August 2019 20:08
by pocock
There were a few issues that I've worked around in the packaging but before we spread the packages more widely it would be nice to fix them in the main project repositories

- compile errors
https://gitlab.com/dpocock/domoticz-deb ... ning.patch

- cmake version: many distributions don't have the latest cmake. I changed it back to an older version and it seems to work:
https://gitlab.com/dpocock/domoticz-deb ... sion.patch

- specific to Domoticz-Zigate and any other plugin that wants to write into the plugins/foo directory, the systemd unit file needs to include something like this:
https://gitlab.com/dpocock/domoticz-deb ... ervice#L49

ReadWriteDirectories=/usr/lib/domoticz/plugins/Domoticz-Zigate
ReadWriteDirectories=/etc/domoticz/plugins/Domoticz-Zigate

but that is just a hack, it is better if every plugin uses /var/lib/domoticz/plugins automatically

Domoticz-Zigate also wants to write to the config files in etc, those files probably need to live in var as well if they are managed by the process

Re: Debian and Raspbian packages now available

Posted: Tuesday 06 August 2019 20:22
by JuanUil
Good Work!

Re: Debian and Raspbian packages now available

Posted: Tuesday 06 August 2019 20:39
by pipiche
pocock wrote: Tuesday 06 August 2019 20:08
- specific to Domoticz-Zigate and any other plugin that wants to write into the plugins/foo directory, the systemd unit file needs to include something like this:
https://gitlab.com/dpocock/domoticz-deb ... ervice#L49

ReadWriteDirectories=/usr/lib/domoticz/plugins/Domoticz-Zigate
ReadWriteDirectories=/etc/domoticz/plugins/Domoticz-Zigate

but that is just a hack, it is better if every plugin uses /var/lib/domoticz/plugins automatically

Domoticz-Zigate also wants to write to the config files in etc, those files probably need to live in var as well if they are managed by the process
Happy to work on those matters. Please let me know what you would expect.

Re: Debian and Raspbian packages now available

Posted: Thursday 08 August 2019 14:06
by pocock
Some observations about firmware and packaging:

It would be really nice if we could build the Zigate firmware as a package so that
a) people don't have to download it, they will just have it in /usr/share/zigate-firmware/whatever.bin
b) we have the consistency of building 100% of the stack from sources

The Domoticz-Zigate plugin also includes some firmware for specific devices (Ikea Tradfri):
https://github.com/pipiche38/Domoticz-Z ... TAFirmware

If we don't have source code for those then it may be better to keep them in a separate repository and a separate package that is installed to /usr/share/ikea/tradfri or something like that. Git users can still link it into their tree using Git submodules and package users can have a symlink to it.

If you are happy with this in principle, are there any other firmware modules that need to be distributed? Maybe it is a good idea to discuss them and the best directory locations collectively before doing any work on moving the files.

Re: Debian and Raspbian packages now available

Posted: Thursday 08 August 2019 14:45
by pipiche
pocock wrote: Thursday 08 August 2019 14:06 Some observations about firmware and packaging:

It would be really nice if we could build the Zigate firmware as a package so that
a) people don't have to download it, they will just have it in /usr/share/zigate-firmware/whatever.bin
b) we have the consistency of building 100% of the stack from sources

The Domoticz-Zigate plugin also includes some firmware for specific devices (Ikea Tradfri):
https://github.com/pipiche38/Domoticz-Z ... TAFirmware

If we don't have source code for those then it may be better to keep them in a separate repository and a separate package that is installed to /usr/share/ikea/tradfri or something like that. Git users can still link it into their tree using Git submodules and package users can have a symlink to it.

If you are happy with this in principle, are there any other firmware modules that need to be distributed? Maybe it is a good idea to discuss them and the best directory locations collectively before doing any work on moving the files.
For the Ikea firmware, I might in the feature do the download directly on the target system. So OTAFirmware might need to be writeable. For now, I have to download and do a manual process to make it in a format the plugin can use it.

For the Zigate firmware, you might have a look to that https://github.com/fairecasoimeme/ZiGate/issues/105. I don't know ho we can automate it.

Re: Debian and Raspbian packages now available

Posted: Monday 12 August 2019 11:01
by pocock
pipiche wrote: Thursday 08 August 2019 14:45 For the Ikea firmware, I might in the feature do the download directly on the target system. So OTAFirmware might need to be writeable. For now, I have to download and do a manual process to make it in a format the plugin can use it.
An automated download like that could go to /var/cache/something, e.g. /var/cache/domoticz/plugins/zigate/OTAFirmware


However, I also feel that we need to be cautious about automatic downloads for various reasons:

- from a privacy perspective, some people don't like any software installed from Debian to make automatic connections to external servers

- some people like to control the exact versions of everything in their network or ensure that versions only change during planned maintenance windows and putting the firmware in a package gives them the ability to control when that package is upgraded

- some people don't have any network access (sometimes by choice) and they simply download the packages onto a portable hard disk and then copy them to their isolated network

- from a practical perspective, Zigate is not the only project that will use this firmware, keeping it in a separate repository with a separate update tool may allow people from other projects to collaborate on it and share the workload
pipiche wrote: Thursday 08 August 2019 14:45 For the Zigate firmware, you might have a look to that https://github.com/fairecasoimeme/ZiGate/issues/105. I don't know ho we can automate it.
Has anybody had any personal interaction with NXP about their tools? It looks like they support Eclipse but has anybody asked them if they will support a plugin for Maven or Gradle or some other command line builds? Being able to do continuous integration (e.g. travis-ci or Jenkins builds) is a very popular requirement these days.

Re: Debian and Raspbian packages now available

Posted: Monday 12 August 2019 14:33
by pipiche
pocock wrote: Monday 12 August 2019 11:01
pipiche wrote: Thursday 08 August 2019 14:45 For the Zigate firmware, you might have a look to that https://github.com/fairecasoimeme/ZiGate/issues/105. I don't know ho we can automate it.
Has anybody had any personal interaction with NXP about their tools? It looks like they support Eclipse but has anybody asked them if they will support a plugin for Maven or Gradle or some other command line builds? Being able to do continuous integration (e.g. travis-ci or Jenkins builds) is a very popular requirement these days.
You might ask on the Zigate Github https://github.com/fairecasoimeme/ZiGate

Re: Debian and Raspbian packages now available

Posted: Monday 19 August 2019 14:47
by pipiche
@pocock, Daniel I have a number of users whom are having trouble when installing your package. I believe this is due to installing the package on an existing Domoticz environment.

They reach a point where everything is lost from a Domoticz standpoint (most-likely not using the same folders)

Re: Debian and Raspbian packages now available

Posted: Monday 19 August 2019 14:56
by ben53252642
I'd really like to see an AppImage release that can run on any distro and includes all the dependencies.

https://appimage.org/

Re: Debian and Raspbian packages now available

Posted: Tuesday 20 August 2019 11:47
by djdem1664
Hi everybody,

Back with Domoticz, I just installed Domoticz on an Orange Pi 3 with Debian Buster, but I have a problem installing the Zigate plugin.

I followed the procedure here:
https://danielpocock.com/domoticz-packa ... an-fedora/
But when I run the command: $ sudo apt install domoticz-zigate-plugin
I have this message in return:
domoticz-zigate-plugin: depends: domoticz but it is not installable: unable to correct problems, you have held broken packages.

Do you have any solutions ?

Have a good time.

Re: Debian and Raspbian packages now available

Posted: Tuesday 20 August 2019 14:33
by pocock
pipiche wrote: Monday 19 August 2019 14:47 @pocock, Daniel I have a number of users whom are having trouble when installing your package. I believe this is due to installing the package on an existing Domoticz environment.

They reach a point where everything is lost from a Domoticz standpoint (most-likely not using the same folders)

Can you suggest any additional logic that the packages should include to avoid this or migrate data?

Re: Debian and Raspbian packages now available

Posted: Tuesday 20 August 2019 14:35
by pocock
djdem1664 wrote: Tuesday 20 August 2019 11:47 Hi everybody,

Back with Domoticz, I just installed Domoticz on an Orange Pi 3 with Debian Buster, but I have a problem installing the Zigate plugin.

I followed the procedure here:
https://danielpocock.com/domoticz-packa ... an-fedora/
But when I run the command: $ sudo apt install domoticz-zigate-plugin
I have this message in return:
domoticz-zigate-plugin: depends: domoticz but it is not installable: unable to correct problems, you have held broken packages.

Do you have any solutions ?

Have a good time.
Can you provide all the output from that command?

Did you try

Code: Select all

sudo apt update
It says something is held, can you try

Code: Select all

dpkg --list | egrep -i 'domoticz|zwave'
and share the output?

Re: Debian and Raspbian packages now available

Posted: Tuesday 20 August 2019 16:00
by pipiche
pocock wrote: Tuesday 20 August 2019 14:33
pipiche wrote: Monday 19 August 2019 14:47 @pocock, Daniel I have a number of users whom are having trouble when installing your package. I believe this is due to installing the package on an existing Domoticz environment.

They reach a point where everything is lost from a Domoticz standpoint (most-likely not using the same folders)

Can you suggest any additional logic that the packages should include to avoid this or migrate data?
For me, after installing the Domoticz and Zigate packages, we must have to migrate Databases and Config file. This has to go through a manual steps as it will be hard to detect if there is already a pre-existing installation.

This might be an additional txt file or something like that, which document what the user will have to do to recover its Db and environment

From Domoticz side, I suspect this is about
- domoticz.db
- some OZW files which are usally on the domoticz home directory

From Zigate plugin, this is about
- Data folder
- Conf folder
- Reports folder

Re: Debian and Raspbian packages now available

Posted: Tuesday 20 August 2019 18:55
by djdem1664
pocock wrote: Tuesday 20 August 2019 14:35 Can you provide all the output from that command?

Did you try

Code: Select all

sudo apt update
root@orangepi3:~# sudo apt update
Hit:2 http://security.debian.org buster/updates InRelease
Hit:6 http://apt.debify.org/debify debify-buster-backports InRelease
Hit:1 http://cdn-fastly.deb.debian.org/debian buster InRelease
Hit:3 http://cdn-fastly.deb.debian.org/debian buster-updates InRelease
Hit:5 http://cdn-fastly.deb.debian.org/debian buster-backports InRelease
Hit:4 https://beta.armbian.com buster InRelease
Reading package lists... Done
Building dependency tree
Reading state information... Done
All packages are up to date.
N: Skipping acquire of configured file 'main/binary-arm64/Packages' as repository 'http://apt.debify.org/debify debify-buster-backports InRelease' doesn't support architecture 'arm64'
N: Skipping acquire of configured file 'contrib/binary-arm64/Packages' as repository 'http://apt.debify.org/debify debify-buster-backports InRelease' doesn't support architecture 'arm64'
N: Skipping acquire of configured file 'non-free/binary-arm64/Packages' as repository 'http://apt.debify.org/debify debify-buster-backports InRelease' doesn't support architecture 'arm64'
pocock wrote: Tuesday 20 August 2019 14:35 It says something is held, can you try

Code: Select all

dpkg --list | egrep -i 'domoticz|zwave'
and share the output?
root@orangepi3:~# dpkg --list | egrep -i 'domoticz|zwave'
root@orangepi3:~#

Thanks and have a good time.

Re: Debian and Raspbian packages now available

Posted: Tuesday 20 August 2019 23:56
by pocock
Thanks for the detailed log, this line shows us the problem:
djdem1664 wrote: Tuesday 20 August 2019 18:55
N: Skipping acquire of configured file 'main/binary-arm64/Packages' as repository 'http://apt.debify.org/debify debify-buster-backports InRelease' doesn't support architecture 'arm64'
You have an arm64 architecture and I only built for amd64, i386 and armhf so far.

I'd be happy to build for arm64 as well but it might be a few days before I do another build, is that OK for you?

Is this your device?

http://www.orangepi.org/Orange%20Pi%203/

Re: Debian and Raspbian packages now available

Posted: Wednesday 21 August 2019 0:06
by pocock
pipiche wrote: Tuesday 20 August 2019 16:00
For me, after installing the Domoticz and Zigate packages, we must have to migrate Databases and Config file. This has to go through a manual steps as it will be hard to detect if there is already a pre-existing installation.
We can use a domoticz.preinst script or domoticz-zigate-plugin.preinst script to detect which files and directories already exist. If the preinst scripts detect the wrong version or wrong file locations they can stop the package install process before any changes are made to the system.

Example for domoticz-zigate-plugin.preinst

Code: Select all

if [ -d /usr/lib/domoticz/plugins/Domoticz-Zigate/.git ];
then
  echo "You already installed the plugin from Git, please manually move the existing plugin directory aside"
  echo "and backup your configuration before installing this package"
  exit 1
fi
We can use a domoticz.postinst script or domoticz-zigate-plugin.postinst script to move files and directories before starting the process

https://www.debian.org/doc/manuals/debi ... intscripts

https://wiki.debian.org/MaintainerScripts

If anybody has time to help develop either script and submit a merge request in gitlab it would be very welcome.

On my own systems, I was able to simply delete all the old data and start fresh but I understand other users won't like to do that.

The same scripting can eventually be replicated into scriptlets in the spec file for RPM users:
https://docs.fedoraproject.org/en-US/pa ... criptlets/

The logic for RPM users will probably be very similar but the script syntax and environment is slightly different.

Re: Debian and Raspbian packages now available

Posted: Wednesday 21 August 2019 7:58
by djdem1664
pocock wrote: Tuesday 20 August 2019 23:56
Is this your device?

http://www.orangepi.org/Orange%20Pi%203/
Yes, it's this one.
Thanks a lot for your help!
No problem, I'll wait a few days!

Have a good day.