Domoticz

Posted: **Monday 20 February 2023 22:15**

Hi Forum,

Spoiler: show: see March 6 post for a summary to see what I did wrong and how I got zwave-js-ui running

Following the advice to move from OZW to zwave-js-ui, I used the steps from the https://www.domoticz.com/wiki/Zwave-JS-UI. As suggested, I opted for using docker-compose. After some challenging evenings, I got to the point where I needed to start zwave-js-ui with the command:

Code: Select all

docker-compose up -d

which yielded this error message:

Code: Select all

ERROR: Version in "./docker-compose.yml" is unsupported. You might be seeing this error because you're using the wrong Compose file version. Either specify a supported version (e.g "2.2" or "3.3") and place your service definitions under the `services` key, or omit the `version` key and place your service definitions at the root of the file to use version 1.
For more on the Compose file format versions, see https://docs.docker.com/compose/compose-file/

Any suggestion appreciated, it is probably simple, but not to me at the moment...

Here is some background information:
I'm currently running Domoticz 2022.1 on RPI3B / Buster. I have OZW installed, MQTT, Modbus, Hue, P1, Nefit Easy, some DzVents scripts, and some other devices and it all works fine.
At some point during the installation, to get docker-compose installed, the wiki suggests:

Code: Select all

sudo pip3 install docker-compose

However, this triggered some errors, so I eventually used:

Code: Select all

sudo apt install docker-compose

On versions used:

Code: Select all

pi@raspberrypi:~ $ docker version
Client: Docker Engine - Community
 Version:           23.0.1
 API version:       1.42
 Go version:        go1.19.5
 Git commit:        a5ee5b1
 Built:             Thu Feb  9 19:46:40 2023
 OS/Arch:           linux/arm
 Context:           default

Server: Docker Engine - Community
 Engine:
  Version:          23.0.1
  API version:      1.42 (minimum version 1.12)
  Go version:       go1.19.5
  Git commit:       bc3805a
  Built:            Thu Feb  9 19:46:40 2023
  OS/Arch:          linux/arm
  Experimental:     false
 containerd:
  Version:          1.6.18
  GitCommit:        2456e983eb9e37e47538f59ea18f2043c9a73640
 runc:
  Version:          1.1.4
  GitCommit:        v1.1.4-0-g5fd4c4d
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

pi@raspberrypi:~ $ docker-compose version
docker-compose version 1.21.0, build unknown
docker-py version: 3.4.1
CPython version: 3.7.3
OpenSSL version: OpenSSL 1.1.1n  15 Mar 2022

The docker-compose.yml file used:

Code: Select all

version: '3.7'
services:
  zwave-js-ui:
    container_name: zwave-js-ui
    image: zwavejs/zwave-js-ui:latest
    restart: always
    tty: true
    stop_signal: SIGINT
    environment:
      - SESSION_SECRET=SomeThingDifferent
      - ZWAVEJS_EXTERNAL_CONFIG=/usr/src/app/store/.config-db
      # Uncomment if you want logs time and dates to match your timezone instead of UTC
      # Available at https://en.wikipedia.org/wiki/List_of_tz_database_time_zones
      - TZ=Europe/Amsterdam
    networks:
      - zwave
    devices:
      # Do not use /dev/ttyUSBX serial devices, as those mappings can change over time.
      # Instead, use the /dev/serial/by-id/X serial device for your Z-Wave stick.
      - '/dev/serial/by-id/usb-0658_0200-if00:/dev/zwave'
    volumes:
      - zwave-config:/usr/src/app/store
    ports:
      - '8091:8091' # port for web interface
      - '3000:3000' # port for Z-Wave JS websocket server
networks:
  zwave:
volumes:
  zwave-config:
    name: zwave-config

Posted: **Tuesday 21 February 2023 1:13**

Dependable of your docker installation it could be that compose was pre installed and then you should use "docker compose" (without the dash in between).

Posted: **Tuesday 21 February 2023 20:40**

Thanks for the suggestion!
The result differs, but not yet what I expected. I need some further googling:

Code: Select all

pi@raspberrypi:~ $ docker compose up -d
[+] Running 0/0
 ⠋ zwave-js-ui Pulling                                                                                        0.0s
error getting credentials - err: exit status 1, out: `GDBus.Error:org.freedesktop.DBus.Error.ServiceUnknown: The name org.freedesktop.secrets was not provided by any .service files`

Posted: **Wednesday 22 February 2023 17:14**

Hello Domoberry,

I noticed that you still have Buster in RPI. Perhaps, it's a good idea to upgrade to Bullseye.

Here is my personal experience with the upgrade https://www.dropbox.com/s/b46opxj6ccvoo ... .docx?dl=0

Below is an apt list of all docker related packaes which are currently running in my RPi

Regards,

ElZorro

Code: Select all

apt list docker*
Listing... Done
docker-buildx-plugin/bullseye,now 0.10.2-1~debian.11~bullseye arm64 [installed]
docker-ce-cli/bullseye,now 5:23.0.1-1~debian.11~bullseye arm64 [installed]
docker-ce-rootless-extras/bullseye,now 5:23.0.1-1~debian.11~bullseye arm64 [installed]
docker-ce/bullseye,now 5:23.0.1-1~debian.11~bullseye arm64 [installed]
docker-clean/stable,stable 2.0.4-3 all
docker-compose-plugin/bullseye,now 2.16.0-1~debian.11~bullseye arm64 [installed]
docker-compose/stable,stable 1.25.0-1 all
docker-doc/stable,stable 20.10.5+dfsg1-1+deb11u2 all
docker-registry/stable 2.7.1+ds2-7+b6 arm64
docker-registry/stable 2.7.1+ds2-7+b6 armhf
docker.io/stable 20.10.5+dfsg1-1+deb11u2 arm64
docker.io/stable 20.10.5+dfsg1-1+deb11u2 armhf
docker2aci/stable 0.17.2+dfsg-2.1+b5 arm64
docker2aci/stable 0.17.2+dfsg-2.1+b5 armhf
docker/stable,stable 1.5-2 all

Posted: **Wednesday 22 February 2023 17:37**

Thanks for the reply, would you recommend RPI update first to solve zwave-js-ui challenges I experiencing?
(my original plan was to do the zwave-js-ui first, than Domoticz 2022.2, then Bullseye, always afraid something might break...)
I have some trouble accessing the Dropbox link.

In my case:

Code: Select all

pi@raspberrypi:~ $ apt list docker*
Listing... Done

Update: recall that Bullseye and Domoticz 2022.1 is not a good option due to some Python issue, rather use a later Domoticz, but... I'm still using OZW, which means I need to stick to 2022.1. Chicken-Egg situation it seems.

Posted: **Wednesday 22 February 2023 20:49**

Hello,

I did not have any python scripts so bullseye and domoticz 2022.1 worked. Version 2023.1 is compatible with python and bullseye.

I would recommend upgrade first.

Here is the link to google drive https://drive.google.com/drive/folders ... QN8KSWk_fX

Hope this works better.

Regards,

Elzorro.

Posted: **Wednesday 22 February 2023 22:28**

Thanks a lot for the document - I will need some time to digest it.

In the meantime I googled on differences between 'docker compose' and 'docker-compose' and which one I would actually need.
I eventually concluded (please correct or confirm!) I would need the 'no dash' version as the 'with dash' version is an older one.
So, did a 'apt remove docker-compose' and a 'pip3 install docker-compose' (confusingly with dash ...) which this time did not trigger errors (some warnings on PATH improvements though - need to digest)

Nevertheless, running 'docker compose up –d' dit not yield expected result (in fact, same as before):

Code: Select all

pi@raspberrypi:~ $ docker compose up -d
[+] Running 0/0
 ⠋ zwave-js-ui Pulling                                                                      0.0s
error getting credentials - err: exit status 1, out: `GDBus.Error:org.freedesktop.DBus.Error.ServiceUnknown: The name org.freedesktop.secrets was not provided by any .service files`

So, unfortunately no progress yet and a document to read (thanks again).

Posted: **Thursday 23 February 2023 20:18**

Hello, I hope that diary was of use.

I have second RPI as a "sandbox" where I test before doing any changes to RPI running Domoticz. This may not be your case but you could try using another SD Card and in my humble suggestion,

Backup you current Domoticz database and store it in an external USB ext4 (Linux) memory stick
Upgrade to Bullseye
Install clean version of Domotics
Install your backup
Install Docker as per script in diary
Install the ZWave JS UI as a service
This should allow to test procedure without changing your running system.

Extra work but at least you will confirm procedure.

Regards,

ElZorro

Posted: **Monday 27 February 2023 14:31**

You are right, original plan was to take this as an excuse to get a second RPI (3b+ or 4) and start from scratch. Unfortunately, they are really not available still. Alternative is to use a second SD card and also follow your notes.

Alternatively, if someone could interpret the error I get:

Code: Select all

pi@raspberrypi:~ $ docker compose up -d
[+] Running 0/0
 ⠋ zwave-js-ui Pulling                                                                      0.0s
error getting credentials - err: exit status 1, out: `GDBus.Error:org.freedesktop.DBus.Error.ServiceUnknown: The name org.freedesktop.secrets was not provided by any .service files`

or point me to information on what credentials from where the system is looking for. That might shine some light for me.

After reading docker basics from https://docs.docker.com/ I have tried pulling the image with

Code: Select all

docker pull zwavejs/zwave-js-ui:latest

which results in the image being downloaded, so that seems to work (I have removed it again thereafter).

Posted: **Monday 27 February 2023 15:42**

Domoberry wrote: ↑Monday 27 February 2023 14:31 Alternatively, if someone could interpret the error I get

Looks this heavy bulky containerization stuff just needs gnome-keyring... So if you use a non gnome based DE, or no DE at all, you IMO run into this issue.

This'll be "funny" if going the bumpy zwaveJS road way needs to have a DE installed, with so many domoticz hosts running headless without any DE installed!

I myself always used minimal raspbian along Domoticz. Only having a minimal X11 support (xvfb package, aka X11 frame buffer) dedicated to use over network connection (perfect companion to "ssh -X ..." X11 forwarding) only with a few minimalist graphical xterms and editors (like rxvt-unicode, or urxvt and nedit) that are not pulling most Gnome or KDE dependancies at install. Just to be able to work more conveniently on PI for a few development/administration, not being stick to console based tools.

Posted: **Tuesday 28 February 2023 12:59**

Hi Lost,
Thanks for your comment, appreciated. As you might have concluded from above posts, the issue is outside my comfort zone, please allow me some clarifying questions as my objective remains to just get zwave-js-ui to work...
- For what purpose is execution of 'docker compose up' (with a yml file for zwave-js-ui) depending on (which) credentials?
- If moving from OZW to zwave-js-ui implies the need to install gnome Development Environment, this sounds to me as indeed heavy. Yet none of the 'how to install zwave-js-ui on your RPI' stuff I have seen mentions this as a requirement. Neither did I see any references stating that a headless RPI setup cannot be used if you want to use docker compose for zwave-js-ui. Is there any logic here?
- some posts I found on the error message suggest to install "pass", would that be related?
- According "cat /boot/issue.txt" I'm using a 'stage 4' setup on my RPI. That should include the desktop, shouldn't that be sufficient?
- using "raspi-config" I have setup my RPI to boot up in just CLI via ssh. Could that be the issue?
(if these questions do not make much sense, I apologize up front)

Posted: **Wednesday 01 March 2023 9:07**

Low and behold...
Thinking that updating the RPI (as done at least twice during this installation process) would never harm, executed 'sudo apt-get update' and 'sudo apt-get upgrade' again. However, this time, as was actually suggested in the upgrade output, also executed 'sudo apt autoremove'.
The 'docker compose up -d' this time around was taking it to the next level:

Code: Select all

pi@raspberrypi:~ $ docker compose up -d
[+] Running 8/8
 ⠿ zwave-js-ui Pulled                                                    494.5
   ⠿ beefe5ad637c Pull complete                                            2.8
   ⠿ 0f0aded1960e Pull complete                                          117.4
   ⠿ 5ad64b3f1478 Pull complete                                          118.8
   ⠿ dd50bd9a6d82 Pull complete                                          119.4
   ⠿ 0771602ac318 Pull complete                                          143.7
   ⠿ e702dc5d51ad Pull complete                                          490.7
   ⠿ 4f4fb700ef54 Pull complete                                          491.4
[+] Running 2/3
 ⠿ Network pi_zwave       Created                                          0.9
 ⠿ Volume "zwave-config"  Created                                          0.1
 ⠿ Container zwave-js-ui  Starting                                        22.4
Error response from daemon: driver failed programming external connectivity on endpoint zwave-js-ui (240fc8be09736379000ea7d5a8870ab4485e2f95b059b8cca99ec3a45dfda734): Error starting userland proxy: listen tcp4 0.0.0.0:3000: bind: address already in use
pi@raspberrypi:~ $

Something to research this evening. Perhaps because MQTT auto discover endpoint not yet installed in Domoticz. [Update: nothing to do with this, see next post]
Will update later.

Posted: **Wednesday 01 March 2023 11:53**

Port 3000 can be used by a lot of applications, for example grafana.

Posted: **Wednesday 01 March 2023 18:24**

Thanks Waltervl,

I'll be lookings at port usage, and undoubtly learn a lot again!
UPDATE 1
So, indeed nothing to do with MQTT auto discovery endpoint not yet installed.
in fact, my "Nefit Easy HTTP server over LAN interface" integration was using local port 3000 to reach the "Nefit server" running on RPI
Next step!
UPDATE 2
With the port corrected in the yml (- '3001:3000' # port for Z-Wave JS websocket server) able to take go the next step:
- start container with 'docker compose up -d' -> looks like zwave-js-ui starts up
- check status 'docker ps' -> status is 'restarting', not 'up'
- check log 'dockers logs zwave-js-ui' -> the error repeats probably at every restart attempt

Code: Select all

pi@raspberrypi:~ $ docker compose up -d
[+] Running 2/2
 ⠿ Network pi_zwave       Created                                          0.5s
 ⠿ Container zwave-js-ui  Started                                          3.5s

pi@raspberrypi:~ $ docker ps
CONTAINER ID   IMAGE                        COMMAND                  CREATED          STATUS                           PORTS     NAMES
8f1148f9830d   zwavejs/zwave-js-ui:latest   "docker-entrypoint.s…"   26 seconds ago   Restarting (133) 2 seconds ago             zwave-js-ui

pi@raspberrypi:~ $ docker logs zwave-js-ui
#
# Fatal error in , line 0
# unreachable code
#
#
#

So, some more google-ing to do!

Posted: **Sunday 05 March 2023 1:17**

brief update:
The error message in the log above is described in the 'zwave2mqtt' post on the Domoticz forum (viewtopic.php?t=36766&start=360). Two links are suggested: a related github bug-report (https://github.com/zwave-js/zwave-js-ui/issues/2725) and a note that was added to the zwave-js-ui documentation on how to address the bug when using Buster (https://zwave-js.github.io/zwave-js-ui/ ... h-with-820).
The solution to this bug is given as either upgrading to Bullseye (as was already recommended at the begin of this thread by Elzorrovega ...) or installing libseccomp2.
I have changed in the docker-compose.yml file the line 'image: zwavejs/zwave-js-ui:latest' to 'image: zwavejs/zwavejsmqtt:8.1.0' as it was also mentioned that the start up issue did not happen with version 8.1.0 of zwavejs2mqtt (the former name of zwave-js-ui); the current version is 8.9.0.
With this change zwavejs2mqtt would startup as I could now access the zwavejs2mqtt control page at http://<ip-of-my-rpi>:8091 (no further functional checks done).
Hence concluding I was hit by the same bug.

Posted: **Monday 06 March 2023 22:10**

I reverted back to the line 'image: zwavejs/zwave-js-ui:latest' in the yml file and executed the suggested commands in case you hit said error and want to stick to Buster described here: https://zwave-js.github.io/zwave-js-ui/ ... h-with-820:

Code: Select all

sudo apt-key adv --keyserver hkps://keyserver.ubuntu.com:443 --recv-keys 04EE7237B7D453EC 648ACFD622F3D138
echo "deb http://httpredir.debian.org/debian buster-backports main contrib non-free" | sudo tee -a "/etc/apt/sources.list.d/debian-backports.list"
sudo apt update
sudo apt install libseccomp2 -t buster-backports

After which 'docker compose up -d' resulted in:

Code: Select all

pi@raspberrypi:~ $ docker compose up -d
[+] Running 2/2
 ⠿ Network pi_zwave       Created     0.3s
 ⠿ Container zwave-js-ui  Started     2.5s

and browsing to http://<ip-of-my-rpi>:8091 provides access to the zwave-js-ui control panel

Next steps are to apply some settings via the control panel, setup MQTT Auto Discover Client Gateway with LAN interface in Domoticz, move the devices from OZW to zwave-js-ui, upgrade Domoticz, upgrade to Bullseye

In summary what I did to correct my mishaps when attempting to get zwave-js-ui running:
* use docker compose (no dash)
* ensure you have the latest updates/upgrades for your RPI
* ensure that the ports you specify in the yml are actually available on your RPI
* if you hit "Fatal error in , line 0 / # unreachable code" error, follow suggestions in https://zwave-js.github.io/zwave-js-ui/ ... h-with-820

Thanks all for your comments!

Posted: **Saturday 11 March 2023 13:47**

To wrap up once I got zwave-js-ui up&running
* give the nodes in zwave-js-ui a practical name (and location)
* you might want to clear the MQTT broker (mosquitto) database to not have retained messages that just create confusion
* install in Domoticz the MQTT auto discovery hardware, also enter the 'discovery prefix' you have in the zwave-js-ui/settings/home assistant
* took me a while before I understood that zwave-js-ui actually has 2 ways to interface to a home automation system: using "WS Server" (websocket, used by HA) and "MQTT Auto discovery" as used for Domoticz.
* note that the devices you see appearing in Domoticz are not always exactly the same as what OZW created, the total number might differ as well
* assuming you have 'disabled' rather then 'deleted' OZW, you can still decide which of the 'old OZW' devices you want to replace with the 'new zwave-js-ui' ones to keep any scripts working and retain the history of the device.
* however, 'disabling' OZW, in my case, rendered its devices invisible in Domoticz. So, based on a tip in the forum, Instead of disabling OZW, I connected it to an incorrect USB port. Then the old devices remain and zwave-js-ui can still work properly
* finally delete OZW and restart your system from the ground up.
So, now zwave-js-ui is working fully with Domoticz. The forum and its posts are an essential resource for me. Thanks.

Posted: **Friday 18 August 2023 8:55**

Domoberry wrote: ↑Saturday 11 March 2023 13:47 To wrap up once I got zwave-js-ui up&running
..
* however, 'disabling' OZW, in my case, rendered its devices invisible in Domoticz. So, based on a tip in the forum, Instead of disabling OZW, I connected it to an incorrect USB port. Then the old devices remain and zwave-js-ui can still work properly
...
So, now zwave-js-ui is working fully with Domoticz. The forum and its posts are an essential resource for me. Thanks.

When disabling a hardware device, the visibility of its devices depends on the setting 'Hide disabled hardware sensors'. It is mentioned in the instruction ( https://www.domoticz.com/wiki/Zwave-JS-UI ), the note in the migration part.

: Schermafbeelding 2023-08-18 om 08.43.55.png (57 KiB) Viewed 3489 times

I've setup my wave-js-ui yesterday, (with help of this post). Zwave is working and ll devices appeared and are replaced. Struggled with three things:

1- I now know that a 'prefix' setting in the 'zwave-js-ui MQTT setting part' is something else than the 'auto discovery prefix' in the 'zwave-js-ui Home Assistant setting part'. AND that these two prefixes may NOT be the same. Solved by changing one of them.
2- the webserver of wave-js-ui cannot be accessed from a browser by domain name. It must be the ip (and port 8091). Must be something to do with ssl? A wget on domain name (and port 8091) gives index.html.
3- Made a costum menu in Domoticz as instructed (with ip and port), but when selecting the menu, nothing appears.

If someone has a tip for 2 and 3, I'll will continue searching for solution. Thanks!

Posted: **Friday 18 August 2023 10:15**

Hansbit wrote: ↑Friday 18 August 2023 8:55
3- Made a costum menu in Domoticz as instructed (with ip and port), but when selecting the menu, nothing appears.

If someone has a tip for 2 and 3, I'll will continue searching for solution. Thanks!

For 3: Do you see a white screen within the domoticz UI when selecting the page from custom menu? There is no error in the screen? Do you use http or https? Dou you use a proxy for remote access?

Perhaps use the new window method like grafana example op wiki https://www.domoticz.com/wiki/Custom_menu

Posted: **Friday 18 August 2023 11:49**

Tried the java-script method and that worked! (After allowing pop-ups in Safari

)

Think that the problem is that wave-js-ui is http and my Domoticz is httpS. (Think that https with an http iframe isn't allowed. ) But solved now. Thanks!

Domoticz

newbie stuck with installing zwave-js-ui

newbie stuck with installing zwave-js-ui

Re: newbie stuck with installing zwave-js-ui

Re: newbie stuck with installing zwave-js-ui

Re: newbie stuck with installing zwave-js-ui

Re: newbie stuck with installing zwave-js-ui

Re: newbie stuck with installing zwave-js-ui

Re: newbie stuck with installing zwave-js-ui

Re: newbie stuck with installing zwave-js-ui

Re: newbie stuck with installing zwave-js-ui

Re: newbie stuck with installing zwave-js-ui

Re: newbie stuck with installing zwave-js-ui

Re: newbie stuck with installing zwave-js-ui

Re: newbie stuck with installing zwave-js-ui

Re: newbie stuck with installing zwave-js-ui

Re: newbie stuck with installing zwave-js-ui

Re: newbie stuck with installing zwave-js-ui

Re: newbie stuck with installing zwave-js-ui

Re: newbie stuck with installing zwave-js-ui

Re: newbie stuck with installing zwave-js-ui

Re: newbie stuck with installing zwave-js-ui