The great Docker problem*

[njlammerts]

You can also use the docker compose method to install/run Domoticz

https://www.domoticz.com/wiki/Docker

I am trying for more then a year now to get Domoticz running on Docker (in conjunctions with Zigbee2mqtt and Mosquitto) I had several tries and every time the outcome is different. I even followed a link who had the complete installation described, but during install I got all kinds of errors, probably due to the age of this post and the change in software versions etc. like this one https://mannema.nl/howto-docker.html
I find it very annoying there are so much means and ways of installing. We talk bout docker, docker compose and docker-compose each with a different install aproach.

Yes I can get Domoticz to run on Docker, also I can get Mosquitto and Zigbee2MQTT to run, but I was not able to get then running and communicating wich each other in a working way,

I would like to use Docker (with compose or -compose), but also with Mosquitto and Zigbee2MQTT , but every time I try, I get a non running environment, Once Domoticz cannot connect to Zigbee2MQTT , the other time Mosquitto won't start. I had even had the respons to install Zigbee2MQTT and Mosquito "native" and only Domoticz in Docker., but it should be possible to get it all running in Docker and to the advantages of Docker.

Unfortunately the installations keeps changing due to updates and upgrades, but it should be possible to get 1 general way of installing all 3 (domoticz, Mosquitto, Zigbee2MQTT) in a concistance way/method so everything works like you installed it native on RPI

[gizmocuz]

I have to assume you followed this guide

https://www.domoticz.com/wiki/Docker

It's easy to follow

Regarding zigbee2mqtt, our wiki has guides for this as well like:

https://www.domoticz.com/wiki/Zwave-JS-UI

The above is a step by step instruction for a docker based installation

[BartSr]

can you provide some more details to show where you get stuck?
Bart

[Kedi]

Perhaps you can install an other container "smeagolworms4/mqtt-explorer:latest" too.
Then you have visual what zigbee2mqtt sends to Mosquitto, and that might help you configuring zigbee2mqtt and Domoticz MQTT-AD

But the most important part of @Gismocuz link is "First get yourself familiar with Docker."

[njlammerts]

I want to use docker so I can use portainer to "manage" it all. With Docker/portainer it is possible to create a copy of a container before running an update. In some situations an update went wrong and with domoticz there is mostly no way back when the update is not functional or gives problems.

I followed the installation Wiki's and the suggested guide's and also for Mosquitto https://hub.docker.com/_/eclipse-mosquitto, or the guide from Github. The result is never a system which functions together.
I thought I found 1 link to describe the complete installation of "]Mannema" all the thing I need, but when follow this I get installation errors due to updates and upgrades of the different packages.
This frustrates me, because I know there must be a "complete" installation description somewhere.

in respons of Kedi: I am a car driver and know the basic functions on how to change tire's, bulbs, water, oil and filters, I don't know how to repair a engine or tune it for better performance. I "just" want to use the different intens functionality. My daily work does not consists of software, programing or so. I think I came a long way to get working with RPI, Raspbian OS, Domoticz etc.

For now I, as an individual, would like some "help" from the great people who are on this forum and all together know so much they can achieve almost everything together.

Thanks in advance!

[BartSr]

yes, there are several ways to reach your goal. But I would advice to use the way domoticz wiki describes. in other words go for docker compose.

[Kedi]

I want to use docker so I can use portainer to "manage" it all. With Docker/portainer it is possible to create a copy of a container before running an update. In some situations an update went wrong and with domoticz there is mostly no way back when the update is not functional or gives problems.

I favor the KISS (Keep It Simple and Stupid) approach of installing software. So why not stay with 3 simple installs without docker.
But the docker approach can be done, all by the info from the Domoticz-Wiki, it really is strait forward.
But you will have to describe the errors you encounter when you install those 3 programs the Docker way.
B.t.w. you just cannot simply create a copy of a container. You have to take into account several things, like ports etc...

[Thorgal789]

I favor the KISS (Keep It Simple and Stupid) approach of installing software

Lol, nice I will re-use it ^^.
I m like you, I m sorry but re-install domoticz from scratch + other applications + some plugin + a last saved configuration from them don't take so long time, and you will have a recent version, in date, with the more help/support you can find, and the less problem with the OS (applications are made to work first on normal OS, and after someone make an image to use them on docker)
The problem is more the compatibility beetween version, but do the same thing on older version take the same time.

And no, using docker is not something easy if you don't know how it work, and its always a pain to find help, because no one know how it work, follow a tutorial to install a "typical" docker is easy but after to make a custom configuration ....

[HvdW]

KISS is a plain install without using docker. Keep It Simple and Smart or Safe or Stupid
If you want to KISS the Safe way then
- SSH into your RPI
- play it extra safe with a sudo service domoticz stop
- sudo cp -r ~/domoticz ~/20240324-domoticz
- sudo service domoticz start
- Ctrl-D to exit SSH
- update your domoticz in the regular way.

I have HomeAssistant running in Docker just for fun.
The backbone of my home automation is Domoticz.

[njlammerts]

I used plain install of Domoticz on Docker (using the WiKi) and used it with Portainer as a sort of management tool.https://www.portainer.io/blog/installin ... spberry_pi
When a new version of Domoticz comes available, you make a copy/clone of the original Domoticz container and apply the update to the original Domoticz/docker container. When the Domoticz update fails or has unwanted results, you can delete this container, "rename" the copied/cloned Domoticz container, boot it, and it will run. It has nothing to do with ports etc. I tried this several time in a test/simulated situation to make sure the practice follows the theory.

[Kedi]

So you don't use Stacks from Portainer. Very complicated but might work. You still have the problem that the database might get a new version and you can't go back. Is that the reason that it took you about a year and still don't get things running?
I have Domoticz, Mosquitto and Zigbee2Mqtt running on Docker within 15 minutes just by following the guides.
But if you want to persue the Docker way I am glad to help but like I wrote before:
But you will have to describe the errors you encounter when you install those 3 programs the Docker way.

[njlammerts]

No, I dont use Stacks, as this is a set of Containers (Domoticz+Zigbee2MQTT+Mosquitto). I use them in seperate containers.
The reason that it takes more then a year is the fact I can't get Domoticz, Mosquitto, Zigbee2MQTT running in/on docker in a way the can communicate with each other, like they would do on "bare metal"

I can get Domoticz running on docker, but it is not usable for e when It can't communicate with Mosquitto/Zigbee2MQTT because I have a lot of Zigbee2MQTT devices.

It is very hard from to describe the errors I had in the past year. I tried more the 5 times a new setup with portainer as base, followed by Domoticz, Zigbee2MQTT and Mosquitto. I followed the particular WiKi pages and other pages like mannama.nl, but because Manama.nl is several years old, ik get errors during installation due to version differences and updates during this period. Also it is very confusing when to use docker, docker compose, docker-compose.

I would appreciate it very much if there would be 1 unified method (docker/-compose) of installing Domoticz, Zigbee2MQTT, Mosquito in a way they can all communicate together without the problem of version differences.

As I wrote before, I am a car driver not a car mechanic, I can use/copy/paste installation instructions/scripts, but when it comes to writing compose scripts or phyton, I am lost and I have to fall back to others who can and wrote about it in forums etc.

[Kedi]

If you don't use Stacks then stop using Portainer, and even better stop using docker, because it looks like you don't understand the process.
Different version is NOT the problem in docker. Every container is a like a standalone program with its own OS.
But better: Just use the "bare metal".

But if you must pursue the docker way, just install (without portainer) the 3 images with the commands from:
Domoticz: https://hub.docker.com/r/domoticz/domoticz and use that compose file.
Zigbee2mqtt: https://www.zigbee2mqtt.io/guide/instal ... er-compose and use that compose file.
Mosquitto: https://hub.docker.com/_/eclipse-mosquitto and use this setup if you want to use password protection
https://techoverflow.net/2021/11/25/how ... r-compose/
and change the config file part if you don't want to use password protection

Code: Select all

listener 1883
## Authentication ##
allow_anonymous true
#password_file /mosquitto/conf/mosquitto.conf

Change ALL the docker-compose.yml files according to your setup, like volumes, ports, devices, etc....

But first make your self familiar with docker-compose, Google for "how to use docker compose"

[BartSr]

why not install domoticz and mosquitto in the conventional way and then docker-compose for realising zigbee2mqtt.

So after you did 1st step dz and mqtt revert here and we can guide further

[njlammerts]

As I wrote earlier in this post, I want to use portioner as a kind of tool to "manage" the Domoticz docker, major to make a copy/backup before an update.
In the past I had have several (major) problems after updating Domoticz or Zigbee2MQtt. The problem with Domoticz is, there is no way back after an update, other the reinstalling and do a difficult restore/install of database, scripts etc.
With portioner it is possible to make a "copy" of the running Domoticz container and continue with the update. If it goes wrong I can revert the "copy" back to running situation. I tested it with a basic Domoticz setup, and it worked. This is way less intens as shutting down and making an ISO/IMG of the SD card/SSD and booting again.

That's the main reason why I want to use docker and/with portioner.

[andreo]

This may be of some use to you, I installed Portainer and placed the code below in the /opt/domoticz directory with the name docker-compose.yml
The other directories are in /var/mydata/

Code: Select all

version: '3.3'
services:
    rpi-influxdb:
        restart: always
        ports:
            - '8086:8086'
        container_name: myInfluxDB
        volumes:
            - '/var/mydata/influxdb:/data'
        image: influxdb

    cam2mqtt:
        restart: always
        ports:
            - "8081:8080"
        container_name: myCam2Mqtt
        volumes:
            - '/var/mydata/cam2mqtt/config.yml:/config.yml'
        image: acasal/cam2mqtt:latest
        environment:
            - TZ=Europe/Amsterdam
    
    grafana:
        restart: always
        ports:
            - '3010:3000'
        volumes:
            - '/var/lib/grafana:/grafana_data'
        container_name: myGrafana
        image: grafana/grafana
    
    eclipse-mosquitto:
        restart: always
        ports:
            - '1883:1883'
            - '9001:9001'
        container_name: myMosquitto
        volumes:
            - '/var/mydata/mosquitto/config/mosquitto.conf:/mosquitto/config/mosquitto.conf'
            - '/var/mydata/mosquitto/data:/mosquitto/data'
            - '/var/mydata/mosquitto/log:/mosquitto/log'
        image: 'eclipse-mosquitto:latest'
    
    node-red:
        restart: always
        ports:
            - '1880:1880'
            - '502:502'
        volumes:
            - '/var/mydata/nodered/data:/data'
        environment:
            - TZ=Europe/Amsterdam
        container_name: myNodered
        image: 'nodered/node-red:latest'
    
    domoticz:
        image: domoticz/domoticz:beta
        container_name: myDomoticz
        restart: always
            # Pass devices to container
        devices:
            - "/dev/serial/by-id/usb-Arduino__www.arduino.cc__Arduino_Mega_2560_852353331353518060D0-if00:/dev/ttyACM0"
            - "/dev/snd/:/dev/snd/"
        ports:
            - "161:161"
            - "8080:8080"
            - "8088:8088"
            - "8443:443"
            - "5000:5000"
            - "1012:1012"
            - "2000:2000"
            - "2001:2001"
            - "49000:49000"
        volumes:
            - ./userdata:/opt/domoticz/userdata
        environment:
            - TZ=Europe/Amsterdam
            - LOG_PATH=/opt/domoticz/userdata/domoticz.log

    nefiteasy-http-server:
        restart: always
        ports:
            - '3000:3000'
        container_name: myNefitEasy
        environment:
            - NEFIT_SERIAL_NUMBER=xxxxxxxxxxx
            - NEFIT_ACCESS_KEY=BNBsFYnZ4S6xxxxx
            - NEFIT_PASSWORD=xxxx
        image: trafex/nefiteasy-http-server

    duplicati:
        image: duplicati/duplicati:latest
        container_name: myDuplicati
        environment:
            - PUID=0
            - PGID=0
            - TZ=Etc/UTC
            - CLI_ARGS= #optional
        volumes:
            - /var/mydata/duplicati:/config
            - /var/mydata:/source
            - /opt:/source1    
        ports:
            - 8200:8200
        restart: always

The file customstart.sh is located in the /opt/domoticz/userdata directory.

Code: Select all

#!/bin/bash

if [ -f /opt/domoticz/FIRSTRUN ]; then
true
else
echo 'updating packages'
apt-get -qq update
apt install -y ffmpeg
apt-get install -y mpg123
apt -y  install snmpd snmp
apt-get install curl
apt -y install libjson-c-dev
apt-get install nodejs npm node-semver
echo 'installing python libraries'
pip3 install reolink-aio
pip3 install pylunar
pip3 install -U pymodbus pymodbusTCP
pip install webhook_listener
echo 'copying event script templates to external mapped storage /volume1/domoticz/scripts/templates'
cp -r /opt/domoticz/scripts/templates /opt/domoticz/userdata/scripts
echo 'creating FIRSTRUN file so script can check on next run'
# touch /opt/domoticz/FIRSTRUN
cd /opt/domoticz || return
fi

A complete update of all containers is done with the following command

Code: Select all

docker compose pull && docker compose up --force-recreate --build -d && docker image prune -f

[Kedi]

As I wrote earlier in this post, I want to use portioner as a kind of tool to "manage" the Domoticz docker, major to make a copy/backup before an update.

Again: when you make a backup in Portainer it does NOT make a backup of the database and others outside the container.
It is so simple: just run domoticz, zigbee2mqtt (and mosquito) the 'normal' way.
Make your backups with rsnapshot to an USB-stick. It is easy to setup.
I have my rsnapshot setup every 4 hours for a year of backups.
Even simpler: just do a cp of the whole domoticz, zigbee2mqtt dir to an USB-stick just before the update.
@andreo why not use Stacks from Portainer, then you have much more control. And use a Stack for every container, even more control.
B.t.w. Why not use

Code: Select all

if [ ! -f /opt/domoticz/FIRSTRUN ]; then

instead of

Code: Select all

if [ -f /opt/domoticz/FIRSTRUN ]; then
true
else

[njlammerts]

In reply of message from Kedi on 01 April.

I am running latest RPI bookworm 64bit lite (headless)

I followed your suggestion for installing the docker way, starting with domoticz, using the applied link..... and there we go again: Error "yaml: line 5: mapping values are not allowed in this context" when executing like explained.

or " yaml: line 4: did not find expected key" when changing image from stable to latest

[Kedi]

You better check the exact indentation of yml file, don't put extra spaces and/or don't remove spaces.

[njlammerts]

I did an exact copy/paste of the file like described in the link. Find the fault and tell me !!!!!

Scherm­afbeelding 2024-04-29 om 20.36.39.png (398.01 KiB) Viewed 1489 times