Any tips or suggestions on RPI3 to RPI4 upgrade?

[Domoberry]

Hi Forum,
(I updated this post few times, so below is becoming more and more a high level description of the steps taken )
Seeking some of the forum's collective knowledge to smoothen the upgrade process for my Domoticz from an RPI3 to an RPI4.
Starting point is a properly running 2023.2 release on Bullseye with Docker ZwaveJS/MQTT AD and Aeotec Gen5 Zwave USB dongle, Nest-via-Google thermostat, Energy meter via Modbus, MQTT client (several Tasmota devices), Buienradar, Nefit Easy thermostat, Philips Hue, Solar Edge PV system, and P1-smartmeter.
The new hardware is an RPI4 (4GB), new 32 GB SD card, new power supply, new Aeotec Gen5-plus Zwave USB dongle.
I'm planning the following steps, but since this is obviously a first, tips welcome!
0 perform full update/upgrade on old RPI3 and create a full back up image of the current RPI3 SD card (done)
1 install Bullseye on RPI4, Pi OS Lite 64bit version, headless with SSH, used Raspberry Imager tool, set user/password, usual Raspi Config stuff, reserved IP address via router's DHCP (done and OK),
2 back up Aeotec Gen5 stick and upload to Gen5plus stick (use the Aeotec Windows tool), ensure this works on RPI3 (done and OK)
2b what I probably should have done was make a backup of the RPI3 ZwaveJS configuration and reinstate it on the RPI3 instance (forgot...)
3 install latest stable Domoticz, using the default bash script (done and OK)
3a: needed to install python-dev to get Domoticz plugin system running (done and OK)
3b ensure Mosquitto has its config updated to allow for external MQTT clients (done and OK)
4 install the various Domoticz plugins files and any related libraries needed for my Domoticz hardware (Pip3, GIT, Mosquitto, NodeJS, NPM, PM2, Easy-Server, minimalmodbus, pyserial, docker, docker-compose, as well as some dedicated plugin.py files in their respective folders) (done and OK)
5 get ZwaveJS and Aeotec Gen5-plus running, using edited yml file and Docker Compose, eventually working. Address port 3000 conflict with Nefit Easy server. (done and OK)
6 complete setting for Zwave via ZwaveJS-UI and copy the html file for the custom ZwaveJS-UI Domoticz menu (Done and OK)
7 backup the Domoticz database on the (soon to be) old RPI3 system using Domoticz UI (done and OK)
8 upload the Domoticz database to the RPI4 system (done and OK) (with the upload of the old database many things reappeared as before, inclusing my LUA and DzVents scripts, really good!)
9 check if all Domoticz hardware devices are OK (Done and OK) (see (8), only needed to correct MQTT IP addresses obviously)
10 say thanks to the RPI3 for many years of service, there is a new task awaiting (Pi-star) (done!)
11 move the USB devices (Zwave stick, USB-Modbus and USB-P1) to RPI4. (there was no issue continuing using the persistemt USB IDs from IRPI3. Done and OK)
12 Use the Domoticz "replace-device" function to correct here and there. (needed in some cases, mainly due to Zwave devices reappearing as new devices. Done)

Thanks in advance!

[HvdW]

The easiest way to clone for me is rpi-clone.
Did you try to just put a cloned SD card in your new rpi4?
Before you boot the rpi4 beware to halt your rpi3 for a while.
If you're lucky the only things you have to arrange is switch some USB ports.

[Domoberry]

That sounds very tempting. My consideration for the 'clean' approach was that after messing around for 6 years with Domoticz on the current RPI3, it would be best to do a fresh setup as there is probably a lot of 'abandoned engineering' and other orphaned code laying around.

[edited to original question at the start of this post]

[eddieb]

I did this a couple of months ago ...
The RPI3 kernel does not want to run on a RPI4.
So I needed to increase the size of the /boot partition and install a RPI4 kernel on it.
There are instructions for this floating around.
I all did this on the current SDCard. (made some backups of course)
After that I did put the SDCard in the RPI4 and I was back in the air

[HvdW]

Doing a fresh install to hop from Buster to Bullseye on RPI3 is easy.

I tried several times to copy my old Domoticz install from a Buster system to a Bullseye system on my PI 3B+.
It failed.
What I did.
On a fresh Bullseye I installed SYSFS for GPIO (some switches)
I copied /usr/local/bin/sbfspot.3 and /etc/init.d/domoticz.sh
Then I installed Domoticz and hardware RFXtfx, SBFspot and P1 smart meter.
Next step was to copy domoticz.db from the Buster sd card to the Bullseye sd card.

My RPI with Bullseye starts up nicely.
Connecting with PuTTY and sudo service domoticz status shows that Domoticz is running smoothly.
However trying to connect with the browser renders a blank page. Connection not possible.
Even after clearing cache it still refuses connection.

What did I miss here.

[Domoberry]

Checking sudo service domoticz.sh status for normal status (not 'exited') and clearing cache seem the logical first steps.
I found this https://www.domoticz.com/wiki/Troubleshooting document helpful in similar cases.

I have also updated the original first post with the steps I have taken so far. Not fully done yet .

[HvdW]

Thanks @Domberry
I have got it up and running now, but it was an obstacle course

- One thing that was interfering with the setup was that I had connected a USB drive using fstab
A raspberry pi doesn't like an USB drive that is not connected and should be there according /etc/fstab
- Another thing is that one way or the other I wasn't able to transfer domoticz.db to the new SD card.
This sounds very strange but it happened 3 times.
- I encountered other problems with file permissions 755 for directories and 644 for files. Many had changed.
- I encountered problems with ownership 1001:1001 instead of pi:pi

That's it for the moment.

My advise: when changing from buster to bullseye or when changing from RPI3 to RPI4 do consider a complete new setup.
The thing I wanted is to retain old Gas and Power data 3 years back, that's where the trouble began.

[Domoberry]

At the moment I have my previous Domoticz 2023.2 setup running again on an RPI4, Pi OS lite, 64bit.
I have updated the original post above with the high level steps. Can't say it was a 'walk in the park', which is largely linked to my experience level in Pi OS and Domoticz . Most challenges I had with the transfer of the Zwave part of my system. I probably should have made use of the backup/restore feature in ZwaveJS-UI.
I continue to use an SD card. I do replace the SD card every year or so, while at the same time making a backup of the card (image). Branded 32GB class 10 SD cards are friendly priced.
Reinstating the Domoticz.db file on the new RPI4 was a truly positive experience. Once you have all 'non Domoticz' changes in place (libraries, programs, plugin files, etc), it revieves the old system including the historic device data. I used the 'Restore' function in the Domoticz UI.
Reinstalling Domoticz from scratch (except reusing the Domoticz databse) on a freshly setup RPI would be recommended from my point of view.
Thanks all!