Migration of Zwavejs-ui and ZME UZB from RPI3 to RPI4

[Michel13]

Hello the forum,

I try to migrate domoticz from RPI3B to a RPI4B.

MQTT and Zwavejs-ui are running on docker on the RPI3B hence, I try to keep the same on the RPI4B.

The installation of domoticz, docker and mqtt is ok. I retrieve all of my system on the RPI4 after the restoration of domoticz.db, except for Zwavejs-ui which ruin all my work and I'am running out of ideas to solve this issue.

My Zwave network have some remote switches and door contacts controlled by a Zwave key. It is working fine on the RPI3B, I can add, remove, control from the control panel but when it is migrated to the RPI4B, nothing is working.

I have first do the installation following this wiki : https://wiki.domoticz.com/Zwave-JS-UI
When I install the USB key the control panel show me all the nodes, without any information on the manufacturers, the products and the product code. The system is permanently interwieving the nodes but not going any further. Zwave is not seen in MQTT Explorer.

My next attempt was to copy /opt/zwavejs-ui from RPI3B to RPI4B. Now the control panel shows all the node interviews as completed, but status are failed. Zwave is well seen in MQTT Explorer. I try to re-interview nodes, but as previously, it run endlessly.

What can I do to make the nodes operational on the RPI4B ? Is there a method to migrate Zwavejs-ui from one platform to an other ?

Note : Title modified with tha addition of the ZME UZB to reflect the main issue of this subject

[waltervl]

There is some issue on RPi4 with a specific Zwave usb stick as it is not using USB3 norms correctly. Google for your Zwave stick type and RPi4 for solution.
I believe most users had to use a powered USB2 hub to get it working.
Edit: It is an issue with Aeotec Z-Wave USB Stick Gen5. See for example https://forums.raspberrypi.com/viewtopic.php?t=245031

Also your Zwave-JS-UI log should show errors as it cannot connect correctly.

Edit 2: If it is not the stick use the export/import buttons to move the old Zwave-JS-UI settings and configuration to your new Zwave-JS-UI environment: https://zwave-js.github.io/zwave-js-ui/ ... p-settings

[Michel13]

Thanks Waltervl for your quick reply.

This incompatibility with the Aeotec key is new to me.
In fact, although my key is a Z-Wave.Me, model UZB, for some unknown reason it is recognized as Aeotec.
With the RPI3B

Code: Select all

pi@raspberrypi:~$lsusb
Bus 001 Device 006: ID 0658:0200 Sigma Designs, Inc. Aeotec Z-Stick Gen5 (ZW090) - UZB
Bus 001 Device 003: ID 0424:ec00 Microchip Technology, Inc. (formerly SMSC) SMSC9512/9514 Fast Ethernet Adapter
Bus 001 Device 002: ID 0424:9514 Microchip Technology, Inc. (formerly SMSC) SMC9514 Hub
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 Root Hub

With the RPI4B

Code: Select all

pi@Domoticz:~$lsusb
Bus 002 Device 002: ID 174c:55aa ASMedia Technology Inc. ASM1051E SATA 6Gb/s Bridge, ASM1053E SATA 6Gb/s Bridge, ASM1153 SATA 3Gb/s Bridge, ASM1153E SATA 6Gb/s Bridge
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 Root Hub
Bus 001 Device 003: ID 0658:0200 Sigma Designs, Inc. Aeotec Z-Stick Gen5 (ZW090) - UZB
Bus 001 Device 002: ID 2109:3431 VIA Labs, Inc. Hub
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 Root Hub

The key is connected to a USB2 port of the RPI4B. It is detected by both RPIs.
dmesg also seems to work (the key is recognized)

ls /dev/serial/by-id gives the same output on each RPI :

Code: Select all

pi@Domoticz:~ $ ls /dev/serial/by-id
usb-0658_0200-if00

For information, as ports are different and are subject to change time to time, I have added the 99-usb-serial.rules to /etc/udev/rules-d/ with the following entry:

Code: Select all

SUBSYSTEM=="tty", ATTRS{idVendor}=="0658", ATTRS{idProduct}=="0200", SYMLINK+="ttyUSB-ZStick-5G"

Another point, the UZB does not look like the Aeotec model at all. It is much more compact and probably very different internally. Looking at your first link, it seems that the incompatibility is due to the hardware design https://community.home-assistant.io/t/s ... pi4/218405
Although they are completely different, it is possible that the same error occurs on the UZB, but the probability is very low in my opinion.

I could try with a USB2 Hub (I should buy one), but I come from a RPI3 running on an SD card with a single Z-Wave stick attached to it (so very compact). The RPI4 runs on an external SSD, so I am not keen on adding another unit to land with some kind of ugly spider just because of this problem. And it is not sure that it will solve the problem. I would prefer to replace the Z-wave stick with another model compatible with the RPI4, but is it possible to transfer all existing nodes to the new stick, keeping all the identifiers? Any suggestion?

Regarding your edit 2, I am in exactly this situation at the moment. By copying the /opt/zwavejs-ui folder from RPI3B to RPI4B, all the json and settings are imported into the new environment.

I am currently running a rebuild of the routes. I don't know how long this will take and I am not sure if it will do the trick. I will get back to you with the result.

[waltervl]

You should not have to create udev rules but use the by-id port. These by-id devices do not change even if you put the USB stick in another port.
In your case /dev/serial/by-id/usb-0658_0200-if00

Are you using Zwave-JS-UI in docker? As that also has different configuration rules.
But start checking the Zwave-JS-UI logging as that should show what really is going on....

[theo11]

I just migrated from RPI3B to RPI5

On RPI3B, I had only zwave-js-ui running in a container, the rest Domoticz, mosquito, node-red and homebridge was plain installation.
I also have the zwave.me UZB stick on the RPI3B , I also used a rule for not changing the USB name.

On RPI5, I have all the applications installed via containers (Docker compose with docker-compose.yaml files).
I have a spare UZB stick, which I used to test the zwavejs-ui container and used the - '/dev/serial/by-id/usb-0658_0200-if00:/dev/zwave' in the docker-compose file.
After configuring all the other containers, I have exported and imported the Domoticz data base from RPI3B into RPI5 and from the UI of zwavejs-ui I made a backup from the store from 3RPIB and restored these settings in the new container on RPI5.

Then I shutdown the RPI5 system and installed the UZB stick from the RPI3B system and all the devices were there.
After a few minutes the system was working fine.

[Michel13]

I have created the rule because it is recommended in the wiki for the installation of domoticz on a rasp.
https://wiki.domoticz.com/PersistentUSBDevices
I understand this is maybe obsolete since it is included in the parameters of ZWave-JS-UI. Thus I have removed it.

I use ZWave-JS-UI on docker and the port is well indicated at the right position in the docker-compose.yml

Code: Select all

version: "3.7"
services:
  zwave-js-ui:
    container_name: zwave-js-ui
    image: zwavejs/zwave-js-ui:latest
    restart: always
    tty: true
    stop_signal: SIGINT
    environment:
      - SESSION_SECRET=mySecretSession
      - TZ=Europe/Amsterdam
    devices:
      - '/dev/serial/by-id/usb-0658_0200-if00:/dev/zwave'
    volumes:
      - ./store:/usr/src/app/store
    ports:
      - "8091:8091" # port for web interface
      - "3000:3000" # port for Z-Wave JS websocket server

I have given a quick look to the logs and it is really huge (>15500 lines right now)! For sure, I have not the right expertise to understand all of them but I'll try to do my best.

Concerning the rebuild of the routes, it has finished without any change. I have then try to re-interview a node. It turns for hours now but no progress at all.

Unless the logs give a clue, I don't see how I can make it working without a complete reinstallation which would be a nightmare to me. And even so, I'am not sure to make this Zwave thing working properly.

[waltervl]

As you are working on docker you only need to transfer the store as theo11 also explained. And the export/import settings functionality does that for you so you cannot make mistakes.

So perhaps restart the new Zwave-JS-UI docker from scratch and only move to store through the export/import functions.

[Michel13]

Thank you to both of you for your great help.

Well, I have make a big progress in the sense that I have been able to get all my sensors back to work in ZWave-JS-UI. But unfortunatly not for long....

I have first export from the RPI3B all the json for each node, as well as the backup and the dump. I don't find in the interface the backup of the store as explained by theo11. I suppose it is just a copy of the store folder.
Then I have reinstalled the UZB stick in the RPI4. The import of the jsons gives no result : active nodes remain in fail status.
So I decided to clear all the faulty ones and perform new inclusions. It is a long job but it works !
All the nodes restored with their ID's and well reported in the MQTT broker, althought it was not as fast as with the RPI3B.
However, if it was ok up to MQTT, the interface with Domoticz was not. All the door switches work, not the controlled plugs. In the mean time, the UZB was flashing permanently.
So I have done a reboot in the hope that will restore the operation of the UZB and the links to MQTT : a desaster !
Except for the battery powered door switches, all the controlled plugs are again in fail condition
I have then started a new interview of the nodes and it is running for 1 hour now, and no progress. In fact, I am exactly in the same situation as before.

Looking at the store folder, It seems that the jsons have been updated when I have perform the inclusions, but that do not appear in the interface.

Maybe the problem is that I had not cleared the content of the store folder before the installation of the UZB. Do you think it is advisable to do it ?

theo11, how have you perform the backup of the store folder ? Is it via Zwave-JS-UI (I didn't find it) or by copying the folder itself ?

At last, owing the fact that the situation remains unchanged, maybe the UZB itself is the culpid and does not record the changes made in the nodes. In that case, is it possible to reset et reinstall it from cratch ?

[theo11]

Hi Michel,

I did a fresh install of the zwave-js-ui container, with a spare UZB stick.
Then I made a backup of the store my old container in the zwave-js-ui store menu:

Backup

Scherm­afbeelding 2024-11-28 om 13.00.15.png (229.08 KiB) Viewed 322 times

Then on the new container I restored the store also in in the zwave-js-ui store menu:

Restore

Scherm­afbeelding 2024-11-28 om 13.00.43.png (230.52 KiB) Viewed 322 times

Good luck,

Theo.

[Michel13]

Thanks theo11 for these explanations.
I am not really used to the Zwave-JS-UI interface, so I had not noticed the "+" button at the bottom of this page and what it was hiding.

I followed the procedure and as I feared, it still does not work.
I came to the conclusion that the problem is in the UZB.
I bought it in January 2016 and it has never been updated since. The firmware version of which I do not know, is most certainly obsolete and needs several updates.
For this, I installed Z-Way first on the RPI4, then on the RPI3. Z-Way cannot detect the UZB on any of the RPIs.
I wonder how it still works with Domoticz on the RPI3.
In short, I think it is time to replace this controller.

I saw that the Z-Wave.me UZB1 is still on the market at attractive prices. Is it worth going back to the same model?

Otherwise, I saw the Aeotec Z-Stick 7 but it is much, much more expensive. Is it necessary to take this model when I only have about ten simple nodes (door contacts and controlled sockets)?

I am looking for a compact model like those mentioned above, but if there are other models with a good quality/price ratio that you recommend, I am interested.

[solarboy]

Otherwise, I saw the Aeotec Z-Stick 7 but it is much, much more expensive. Is it necessary to take this model when I only have about ten simple nodes (door contacts and controlled sockets)?

I am looking for a compact model like those mentioned above, but if there are other models with a good quality/price ratio that you recommend, I am interested.

There have been a lot of issues with the newer stick 7 and I think later versions of the 5 stick had the USB problems dealt with. I have 50 nodes on a 5 stick, not had problems since the switch to zwavejsUI.

[Michel13]

There have been a lot of issues with the newer stick 7 and I think later versions of the 5 stick had the USB problems dealt with. I have 50 nodes on a 5 stick, not had problems since the switch to zwavejsUI.

Thanks solarboy for this information.
I didn't have any problems either when I did the transfer to zwavejs-ui. It's the migration from RPI3 to RPI4 that is causing big problems.

I've done a lot of operations since my last message.
I managed to update the UZB from version 5.2 to version 5.23 with Z-way installed on my Rasp. It wasn't easy either and I spent lot of time on it. After that, I imported the store and the interviews worked correctly on the RPI4. But the import reintroduced all the nodes, even those that were deleted but still exist in the Json. Result: I ended up with 50 additional devices in Domoticz !

I then started from an image of my SD taken before my migration attempt and did a factory reset of the UZB to remove all nodes.
I reset all my sensors, completely reinstalled the RPI4, imported the domoticz.db database and installed the UZB.
I was able to include sensors, but just like from the beginning, the interview is not successful for some of them.
The conclusion is that the operation of my UZB with the RPI4 is too erratic or that my UZB itself has a problem. In short, it seems impossible to continue using it.

So I turned to the market. I forget the Z-Stick 7. If it causes problems: I have had enough.
The Aeotec Z-Stick Gen5+ is a very popular model. It seems that the compatibility issues with the RPI4 have been solved with the + version. Too bad, however, its size blocks access to the other USB ports of the RPI if it is installed directly. No other choice but to connect it with a USB male/female cable, which finally allows to install it in a more suitable place for communication with the sensors.
I will therefore move towards this model, hoping that it will allow me to finalize my migration without other problems.

I will come back to give you the result.

[solarboy]

You absolutely should use and extension cable or even better a powered USB hub. I've read that the USB chipset on the pi interferes with the RF of the Z-stick. Good luck with the migration.

[Michel13]

I've read the same. I have ordered a 2m extension cable with the stick. I should receive them on friday.

[Michel13]

Hello,
As promised I come back to give you a final report.

I received the Z-Wave key and the USB extension cable today.
But before putting the key into service, I thought it was worth trying to install the sensors with the UZB connected to the extension cable in order to eliminate any possible interference with the RPI4.
And then, everything started working without any problem!
The sensors were included without difficulty, I restarted the RPI to check that they did not disappear as had happened to me above.
Everything works perfectly.
Thanks solarboy for your recommendation.

So it was clearly an interference problem between the UZB and the RPI4 and I did not imagine that it could be to this extent. It is all the more surprising since it was mounted directly on the RPI3 and I have never had any malfunctions between the two.

The most important thing in the end is that I know the cause of the problem and above all that it is resolved.

Now I just have to return the unused Z-Stick which is no longer useful.

Thanks again to all of you for your support.

[solarboy]

Brilliant news, now time for the fun bit.