Easiest way to migrate Openzwave to ZWave-JS-UI

[aukesp]

@aukesp this is more a docker issue than anything else I presume.

Ahha, I was reading these post and thought that maybe more people has got these problems during migration

[andrehj]

I have installed zwave-js-ui (via snap) to replace OZW and run into the following problems:

1. Main problem: When trying to browse to http://<my pi ip-address>:8091/ I get a "cannot connect error"

What I have done:

Code: Select all

pi@raspberrypi:~ $ sudo snap install zwave-js-ui
zwave-js-ui v8.6.2 from Giaever.online (giaever-online) installed
pi@raspberrypi:~ $ sudo snap connect zwave-js-ui:raw-usb
pi@raspberrypi:~ $ sudo snap connect zwave-js-ui:hardware-observe

So far so good. I also enabled the service:

Code: Select all

pi@raspberrypi:~ $ sudo zwave-js-ui.enable
Missing plug: «serial-port»
Service enabled!
- If you have turned OFF «log to file», follow logs with:
  $ snap logs zwave-js-ui -f
- If you have turned ON «log to file», follow logs with:
  $ tail -f /var/snap/zwave-js-ui/49/*.log

Strange is the message "Missing plug: «serial-port»", but is that a problem? I only need zwave-js-ui to read my Z-wave USB-stick.

When asking help, I see a lot of instructions, but I have no idea which of them I need or where to set those "server configuration values"...

Code: Select all

pi@raspberrypi:~ $ zwave-js-ui.help
Missing plug: «serial-port»
zwave-js-ui - v8.6.2

IMPORTANT! The DAEMON/SERVICE is disabled by default after installation.
You have to manually 'daemonize' it with executing the command

$ zwave-js-ui.enable

Set your configuration as descrived below

Server configuration values: {
        "server": {
                "force-disable-ssl": false,
                "host": "localhost",
                "port": 8091,
                "ssl": false
        }
}

- server.ssl: Use secure communication
- server.host: IP address to bind to, e.g 127.0.0.1
- server.port: Port to reach the web interface

Session configuration values: {
        "session": {
                "cookie-secure": "",
                "secret": "c6b3a0b0-93ed-11ed-b424-035baff9c03b"
        }
}

- session.secret: Used as secret for session. If not provided a default one is used.
- session.cookie-secure: Set the cookie secure option. See: https://github.com/expressjs/session#cookiesecure

Session configuration values: {
        "mqtt": {
                "name": ""
        }
}

- mqtt.name: The name used as client name when connecting to the mqtt server.

Set options with:         $ snap set zwave-js-ui param=key
For example:              $ snap set zwave-js-ui server.host=0.0.0.0

Other settings can be set in the UI after start.

If you have turned OFF «log to file», follow the log(s) with
  $ sudo snap logs zwave-js-ui -f

OR if you have turned ON «log to file», tail the logs with
  $ tail -f /var/snap/zwave-js-ui/49/*.log

Other commands
Disable the daemon:       $ zwave-js-ui.disable
Restart the daemon:       $ zwave-js-ui.restart

Info on the serial-port plug:
 This plug requires the experimental hotplug feature in snapd.
 Enable it with
  $ sudo snap set system experimental.hotplug=true

 And restart snapd with:
  $ sudo systemctl restart snapd

 Now list available serial slots with:
  $ sudo snap iterface serial-port

 And connect it to zwave-js-ui with:
  $ sudo snap connect zwave-js-ui:serial-port <slot name>

At the moment I'm stuck. How can I make zwave-js-ui work and access it with a browser at http://<my pi ip-address>:8091/ ?

[MikeyMan]

@andrehj funny seeing you here

i followed this guide. I remember following the steps under 4 several times before it worked.

[JackD]

Since it was announced that Open-Zwave is depreciated I find it stressful to update Domoticz.
Therefore I stayed a while on beta14599, but a few days ago I updated to 14957 because there where a few helpful things changed.
And I am happily to say that 14957 did not broke my Open-Zwave, at least not for my system/setup.

I really hoop that Open-Zwave keeps working, I don't like to put in a lot of work just to keep my fully functioning system working.
It is not a hobby of mine to tinker around with Domoticz and other tools, I just want a system that works and keeps working with a minimal of effort.

But for now I am good......

[andrehj]

i followed this guide. I remember following the steps under 4 several times before it worked.

Thanks for your reply, but which guide do you mean?

[MikeyMan]

i followed this guide. I remember following the steps under 4 several times before it worked.

Thanks for your reply, but which guide do you mean?

Oops... This one

https://community.home-assistant.io/t/i ... -pi/486484

[andrehj]

Oops... This one
https://community.home-assistant.io/t/i ... -pi/486484

Thanks! That guide looks very promising! After everything (hopefully) works, I'll write a full guide here.

[HansF]

Since it was announced that Open-Zwave is depreciated I find it stressful to update Domoticz.
Therefore I stayed a while on beta14599, but a few days ago I updated to 14957 because there where a few helpful things changed.
And I am happily to say that 14957 did not broke my Open-Zwave, at least not for my system/setup.

I really hoop that Open-Zwave keeps working, I don't like to put in a lot of work just to keep my fully functioning system working.
It is not a hobby of mine to tinker around with Domoticz and other tools, I just want a system that works and keeps working with a minimal of effort.

But for now I am good......

Same here. I fully agree with this.

[andrehj]

Thanks! That guide looks very promising! After everything (hopefully) works, I'll write a full guide here.

Today I successfully finished my upgrade. Pi is now running Domoticz 2022.2 with ZWave-JS-UI (via SNAP) and Bullseye OS.
Since there is no complete guide for this upgrade process, I decided to make my own.
The basics steps I made were:
1. Backup Z-Wave data
2. Install Mosquitto
3. Install Snap
4. Install Z-Wave JS UI
5. Disable OpenZwave gateway
6. Configure Z-Wave JS UI
7. Install MQTT Auto Discover Client gateway in Domoticz
8. Replace old Z-Wave devices
9. Delete old Z-wave controller
10. Upgrade Domoticz to 2022.2
11. Upgrade Pi from Buster to Bullseye

According viewtopic.php?p=296307#p296307 the upgrade to bullseye should be done after the Domoticz upgrade to 2022.2, but in step 11 I ran into some minor conflicts which made me have to redo two of the installation steps of Mosquitto (2.5) and Snap (3.3b). Maybe it is better to upgrade your Pi to Bullseye as step 1 instead of 11.

In detail, there were many, many more steps:

Code: Select all

1. Backup Z-Wave data from Domoticz
		Most imporant data is:
		1. Security Network Key
		2. List of NodeID's and node names 
		3. The USB address of your Z-Wave usb stick.

2. Install Mosquitto according https://randomnerdtutorials.com/how-to-install-mosquitto-broker-on-raspberry-pi/ (en https://www.domoticz.com/wiki/MQTT)
	
		1. Update and upgrade system of Pi:
		sudo apt update && sudo apt upgrade

		2. Install Mosquitto:
		sudo apt install -y mosquitto mosquitto-clients
		
		3. Make Mosquitto auto start when the Raspberry Pi boots
		sudo systemctl enable mosquitto.service
		
		4. Start-up and version check:
		mosquitto -v  (Gives version 1.5.7, version 2.xx will be installed with Bullseye)
	
		5. Enable remote access without authentication:
		sudo nano /etc/mosquitto/mosquitto.conf
		Add these lines at the end:
		listener 1883
		allow_anonymous true
		Use CTRL-X to exit and save.
		
		6. Restart Mosquitto:
		sudo systemctl restart mosquitto
		
		7. Test status of Mosquitto:
		sudo systemctl status mosquitto
		
		8. Check version:
		mosquitto -h | grep version    (again version 1.5.7)
		
3.  Install Snap according https://snapcraft.io/docs/installing-snap-on-raspbian en https://community.home-assistant.io/t/install-and-setup-zwave-js-ui-with-snap-on-a-raspberry-pi/486484
		1. Update system:
		sudo apt update
		
		2. Install snap
		sudo apt install snapd
		
		3. Work around a bug which seems to exist only on Raspberry Pi OS 32-bit system: https://stackoverflow.com/questions/42443273/raspberry-pi-libarmmem-so-cannot-open-shared-object-file-error/50958615#50958615 en https://github.com/zwave-js/zwave-js-ui/discussions/1216
		3a
		sudo nano /etc/ld.so.preload
		Comment out this line:
		(Change /usr/lib/arm-linux-gnueabihf/libarmmem.so to #/usr/lib/arm-linux-gnueabihf/libarmmem.so)
		3b
		sudo visudo
		add :/snap/bin to the end of the secure_path, BUT before the closing quote ". Note the :-separator!
				
		4. Reboot Pi
		sudo reboot
		
		5. After this, install the core snap in order to get the latest snapd.
		sudo snap install --channel=stable core
		
		6. Enable hotplug support and restart:
		sudo snap set system experimental.hotplug=true
		sudo systemctl restart snapd
		
		7. To test your system, install the hello-world snap and make sure it runs correctly:
		sudo snap install hello-world
		hello-world
		
4.  Install Z-Wave JS UI according https://www.domoticz.com/wiki/Zwave-JS-UI en https://zwave-js.github.io/zwave-js-ui/#/getting-started/other-methods and https://community.home-assistant.io/t/install-and-setup-zwave-js-ui-with-snap-on-a-raspberry-pi/486484
		1. Install zwave-js-ui
		sudo snap install --channel=stable zwave-js-ui
		
		2. Give the package access to use USB devices and observe hardware:
		sudo snap connect zwave-js-ui:raw-usb
		
		3. Needed for the program to list available devices in the UI:
		sudo snap connect zwave-js-ui:hardware-observe
		
		4. Open help to see some info about commands:
		zwave-js-ui.help	
		
		5. To access the UI from outside your Pi:
		sudo snap set zwave-js-ui server.host=0.0.0.0
		
		6. Then we need to connect the serial-port to the plug zwave-js-ui. First list available serial slots:
		sudo snap interface serial-port
		
		7. Then use this command to connect the correct serial slot to zwave-js-ui (use correct core: from list you got in previous step):
		sudo snap connect zwave-js-ui:serial-port core:aeotecz-stickgen5zw0
		
		8.Turn service on:
		sudo zwave-js-ui.enable

5.	Disable (not delete!) OpenZwave gateway (USB stick) in Domoticz (menu Setup - hardware)
		Idea is that the old devices are not deleted.
		Sadly this did not work. All my old devices disappeared. Maybe I should have upgraded Domoticz from 2022.1 to 2022.2 first. See also https://www.domoticz.com/forum/viewtopic.php?p=297183&sid=8601aab4dc1990f1d0199e8caced281b#p297183
	
6.	Configure Z-Wave JS UI (from https://www.domoticz.com/wiki/Zwave-JS-UI)

		1. Browse to http://192.168.178.41:8091 
		
		2. Menu settings → MQTT: 
		Retain on
		
		3. Menu settings → Gateway:
		Use nodes names on
		Ignore location on
		Include Node info on
		
		4. Menu settings → Home Assistant:
		MQTT Discovery on
		Retained discovery on (not mentioned in text, but that's according picture)
		
		5. Menu settings → Z-Wave
		Get the key from Domoticz (mine was just "0102030405060708090A0B0C0D0E0F10") and enter as S2 Unauthenticated. 
		Enter the right Serial Port address. In my case: /dev/ttyUSB-ZwaveStick
		I also had to enter something at S0 legacy to get rid of an error message regarding the security status of a Qubino ZMNHDD. I just pressed the button to generate a random key. Result was 643294277C1E50FAEA0C497F116F9897.
		
		6. Menu settings 
		Save settings (Press "save" at the bottom)

		7. Menu Control Panel:
		Give all nodes a useful name based on their Node ID and the info you saved in step 1

7. 	Install MQTT Auto Discover Client gateway in Domoticz (see https://www.domoticz.com/wiki/MQTT#Add_hardware_.22MQTT_Auto_Discovery_Client_Gateway.22)
		
		1. Domoticz → Menu Setup → Hardware 
		Name = MQTT Auto Discovery
		Type = MQTT Auto Discover Client Gateway with LAN interface
		Data Timeout: Disabled
		Remote Address: localhost
		Username: Leave empty when mqtt broker has no user accounts setup else give mqtt broker account name
		Password: Leave empty when mqtt broker has no user accounts setup else give mqtt broker account password
		Auto Discovery Prefix: homeassistant
		
		2. Make sure it is enabled and press "add".

		3. New devices are created in Domoticz by MQTT AutoDiscover
		(After renaming the devices in Z-Wave JS UI I should have gotten nice names, but sadly that didn't work...)
		
		4. Domoticz → Menu Setup → Devices
		Manually rename all new devices to something usefull

8.	Replace old Z-Wave devices with new ones
		If you want to keep the history logging of the old devices then you need to replace the old device with the new one if identical devices are created. Use the replace function in Edit device, see wiki https://www.domoticz.com/wiki/Managing_Devices#Replace_device Devices that were replaced will get the old name and old IDX, no need to change scripts.
	
9. 	Delete old Z-wave controller in Domoticz.
		1. Domoticz → Menu Setup → Hardware
		Select old Z-wave controller (Aeotec USB stick) and press Delete.
		
		2. Reboot pi
		sudo reboot 

10. Upgrade Domoticz to 2022.2 according https://www.domoticz.com/forum/viewtopic.php?t=39291
		1. Backup version 2022.1
		sudo service domoticz stop
		sudo cp -R /home/pi/domoticz /home/pi/domoticz_2022.1  (may take while...)
		sudo service domoticz start

		2. Make sure you're on the Stable Release Channel
		Menu Setup → Settings → Software Updates → Release Channel → Stable
		
		3. Update to 2022.2 (called #14606 in dialogue box...)
		Menu Setup → Check for updates → OK
		
11. Upgrade Pi to Bullseye according https://pimylifeup.com/upgrade-raspberry-pi-os-bullseye/ and https://linuxhint.com/upgrade-raspberry-pi-os-buster-to-bullseye/
		1. Backup your Pi.
		
		1. Check your current version:
		lsb_release -a
		
		2. Update Raspberry Pi Repositories List
		sudo apt update
		
		3. Upgrade Packages on Raspberry Pi
		sudo apt dist-upgrade -y
		
		4. Upgrade Firmware
		sudo rpi-update
		sudo reboot
		
		5. Edit Raspberry Pi Source Lists
		In /etc/apt/sources.list replace all instances of "buster" by "bullseye".
		sudo nano /etc/apt/sources.list or use WinSCP
		Do the same for any other sources lists in the /etc/apt/sources.list.d directory:
		sudo nano /etc/apt/sources.list.d/raspi.list
		sudo nano /etc/apt/sources.list.d/vscode.list
		
		6. Update package lists
		sudo apt update
		
		7. Install missing packages:
		sudo apt install -y libgcc-8-dev gcc-8-base
		(enter yes if asked if services have to be automatically restarted)
		
		8. Upgrade to Bullseye
		sudo apt full-upgrade
		
			Ran into Error:
			E: Failed to fetch http://mirror.serverius.net/raspbian/raspbian/pool/main/p/protobuf-c/libprotobuf-c1_1.3.3-1+b2_armh              f.deb  Undetermined Error [IP: 5.255.95.70 80]
			E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?
			Decided to rerun command, second time it worked OK.
			
			Installer reported change conflict in 3 files. Backed up all three and let installler use new versions.
			/etc/default/useradd
			/etc/sudoers
			/etc/mosquitto/mosquitto.conf

			Decided not to do anything about useradd
			Fixed /etc/sudoers with command sudo visudo (see snap configuration above)
			Fixed conflict in /etc/mosquitto/mosquitto.conf by adding the two missing lines (see mosquitto configuration above)
		
		9. Fix error in /etc/systemd/system/dhcpcd.service.d/wait.conf
		Check if file exists:
		sudo stat /etc/systemd/system/dhcpcd.service.d/wait.conf
		Edit the file
		sudo nano /etc/systemd/system/dhcpcd.service.d/wait.conf
		Replace
		ExecStart=/usr/lib/dhcpcd5/dhcpcd -q -w
		by
		ExecStart=/usr/sbin/dhcpcd -q -w
		Exit and save.
		
		10. Modify Boot config
		sudo nano /boot/config.txt
		In section [pi4] comment out dtoverlay=vc4-fkms-v3d (put a # in front of the line)
		In section [all] add the line dtoverlay=vc4-kms-v3d
		Exit and save.
		
		11. Reboot Pi.
		sudo reboot
		
		12. Check version to see if upgrade was successful:
		lsb_release -a
		
		13. Cleanup (if everything works)
		sudo apt autoclean

		14. Reinstall dev version of Python plugin to get plugins working
		sudo apt install python3-dev

		15. Restart Domoticz
		sudo service domoticz restart

I hope this guide is useful for anyone who still has to do the same upgrade. If you like it, please let me know.

[MikeyMan]

Nice work @andrehj !

I had some challenges with python and DHCP after bullseye update. Otherwise its running smooth.

[andrehj]

I had some challenges with python and DHCP after bullseye update. Otherwise its running smooth.

That DHCP error, was that the one I described in step 11.9 above? (Fix error in /etc/systemd/system/dhcpcd.service.d/wait.conf)

[MikeyMan]

I had some challenges with python and DHCP after bullseye update. Otherwise its running smooth.

That DHCP error, was that the one I described in step 11.9 above? (Fix error in /etc/systemd/system/dhcpcd.service.d/wait.conf)

Yup, exactly that.

[nitpicker]

How to find the Z-Wave Security Network Key?

[walawa75]

Thank you @andrehj, you are finaly the only man giving some simple instructions to do it !! All went very fine and all devices were detected and all is working like a charm. Much better manager instead of the ugly openZwave.

[walawa75]

I had some challenges with python and DHCP after bullseye update. Otherwise its running smooth.

That DHCP error, was that the one I described in step 11.9 above? (Fix error in /etc/systemd/system/dhcpcd.service.d/wait.conf)

This file does not exists in my PI. Do I have to create it ? Is that a problem if this file is not there ?

[MikeyMan]

I'd try and create it, see if it works.
Mine looks as follows:

Code: Select all

[Service]
ExecStart=
ExecStart=/usr/sbin/dhcpcd -q -w

I assume i could delete the first line, but didn't fiddle around with it anymore

[tjabas]

i usually make backups of my sdcard, and i also have a spare eaotech gen5 usb stick which i use as a backup,
can i start and install the new z-wave-JS-UI on my domoticz backup sd card and config this in my own pace, it seems like its quite some programming and renaming all the nodes in the new JS-UI, do this configuration also change the node information on my aeotech stick?
or can i simply change the sd card back to my backup and the old open-zwave will work again?
and then when i have more time continue to edit the JS-UI?

hope you know what i mean.
thanks

[mgugu]

This should work, as far as you do not pair or remove devices.

[Benneton]

Wondering if there is a somewhat more detailed manual:
The basics steps I made were:
1. Backup Z-Wave data
2. Install Mosquitto
3. Install Snap
4. Install Z-Wave JS UI
5. Disable OpenZwave gateway
6. Configure Z-Wave JS UI
7. Install MQTT Auto Discover Client gateway in Domoticz
8. Replace old Z-Wave devices
9. Delete old Z-wave controller
10. Upgrade Domoticz to 2022.2
11. Upgrade Pi from Buster to Bullseye

Step 10 & 11 that is covered running on that, and my z-wave is still running with OZW and it appears to work okay.
Step 2 is no issue, have Mosquitto running to scrape my ECU-3 with Node Red.

So how do you do step 1?
Step 3 and 4 I most likely will manage as there are how to's here and there.
Step 5 to 9 how does that work?
Is there a dummy guide as it is getting more and more complex to maintain my Domoticz setup.

Thanks
Bernard

[elzorrovega]

Hello,

Below is a link to a document with my personal migration experience.

I hope this helps other to sweat less.

regards,

El Zorro

P.S. Hope Dropbox link works

https://www.dropbox.com/s/b46opxj6ccvoo ... .docx?dl=0